Exceptions

    exception scrapy.exceptions.CloseSpider(reason=’cancelled’)[source]

    This exception can be raised from a spider callback to request the spider to be closed/stopped. Supported arguments:

    • Parameters

      reason () – the reason for closing

    For example:

    DontCloseSpider

    exception scrapy.exceptions.DontCloseSpider

    This exception can be raised in a spider_idle signal handler to prevent the spider from being closed.

    exception scrapy.exceptions.DropItem

    The exception that must be raised by item pipeline stages to stop processing an Item. For more information see Item Pipeline.

    IgnoreRequest

    This exception can be raised by the Scheduler or any downloader middleware to indicate that the request should be ignored.

    exception scrapy.exceptions.NotConfigured[source]

    This exception can be raised by some components to indicate that they will remain disabled. Those components include:

    • Item pipelines

    • Downloader middlewares

    • Spider middlewares

    The exception must be raised in the component’s method.

    NotSupported

    exception scrapy.exceptions.NotSupported[source]

    New in version 2.2.

    exception scrapy.exceptions.StopDownload(fail=True)

    Raised from a bytes_received or signal handler to indicate that no further bytes should be downloaded for a response.

    The fail boolean parameter controls which method will handle the resulting response:

    • If fail=True (default), the request errback is called. The response object is available as the response attribute of the exception, which is in turn stored as the value attribute of the received Failure object. This means that in an errback defined as def errback(self, failure), the response can be accessed though failure.value.response.

    In both cases, the response could have its body truncated: the body contains all bytes received up until the exception is raised, including the bytes received in the signal handler that raises the exception. Also, the response object is marked with "download_stopped" in its Response.flags attribute.

    Note

    fail is a keyword-only parameter, i.e. raising or StopDownload(True) will raise a .