We really need to be able to take advantage of PBs paging capabilities. But they are broken. Here is a discussions of the problems:
Overview of PB Paging
=====================
There are two sides involved, the sender and the receiver.
There are three basic Twisted classes involved:
* Pager: Instantiated on sender side with reference to remote collector.
* CallbackPageCollector?: Instantiated on receiver side.
* Broker: Manages things on both sides.
First, a collector is created on the receiver. It has two methods::
def remote_gotPage(self, page):
self.pages.append(page)
def remote_endedPaging(self):
self.callback(self.pages)
The first methods, remote_gotPage, is called by the sender to send a page. The second,
remote_endedPaging, is called by the sender when there are no more pages.
Second, the sender creates a Pager with a reference to the remote collector. This causes the Pager to be registered with the collectors broker in registerPageProducer. This appends the pager to a list of producers (pageProducers). Also, if there is exactly one pageProducer in this list, the Broker is registered as a producer with its transport.
From this point on, it is really the Producer machinery that runs the show. The main method that gets triggered is::
def resumeProducing(self):
for pageridx in xrange(len(self.pageProducers)-1, -1, -1):
pager = self.pageProducers[pageridx]
pager.sendNextPage()
if not pager.stillPaging():
del self.pageProducers[pageridx]
if not self.pageProducers:
self.transport.unregisterProducer()
The important thing here is the this calls sendNextPage when needed. This looks like this::
def sendNextPage(self):
self.collector.callRemote("gotPage", self.nextPage())
This is one source of dangling deferreds. These should probably be collected::
def sendNextPage(self):
d = self.collector.callRemote("gotPage", self.nextPage())
self.deferreds.append(d)
Then the stillPaging method gets called to see if the Pager still has pages. Here is that method::
def stillPaging(self):
if not self._stillPaging:
self.collector.callRemote("endedPaging")
if self.callback is not None:
self.callback(*self.callbackArgs, **self.callbackKeyword)
return self._stillPaging
This calls endedPaging, which create another dangling deferred.
