http client should limit the number of queued retries to 1 by default for the same url and request data

[opalmer]

In cases where an http request is being retried multiple times the request may end up being retried more than once. Meaning you could have a request that retries which spawns a retry which then spawns a retry, etc etc. This behavior does not always happen but when it does it's likely a problem in pyfarm.agent.http.core.client, more specifically inside of Request.retry we should be returning the existing request rather than a new one each time.

I found this bug while working with this bit of code:

pyfarm-agent/pyfarm/jobtypes/core/jobtype.py

Line 323 in cf9608d

def download_complete(response):

All you have to do to reproduce the problem is create POST an assignment with a job type that does not exist. If you let it run for a bit it will eventually complete but you can see it making the same request over and over again.

A solution to this may be to return a regular Deferred object in place of the request. Then we attach the callbacks for that deferred object to the single request. When the request completes you'd end up with the results propagating to the dummy deferred objects.