PaginationMixin¶
PaginationMixin
is a mixin with a group of decorators
to control the logic of requesting the next page.
It has an interesting flow, which could be summarized as:
- At the listing parsing method, every item page request is yielded. Each request is marked to be associated with the current response and any pagination requests is enqueued.
- At the item parsing method, the matching logic is applied and each valid item and its related request is registered.
- After comparing the yielded requests at step 1 and the requests which yielded valid items at step 2, the mixin decides to dequeue the next page request only if every request yielded a valid item.
To understand better its working, please review the examples.
-
class
scrapy_mosquitera.mixin.
PaginationMixin
(*args, **kwargs)¶ -
static
deregister_response
(fn)¶ Deregister response from the registry.
It’s a decorator.
-
static
enqueue_next_page_requests
(fn)¶ Enqueue next page requests to be only requested if they meet the conditions.
It’s a decorator.
-
static
register_requests
(fn)¶ Register requests yielded from
fn
in the registry using as key its parent response id.It’s a decorator.
-
static