[Bug]: Download errors stop the program with no retries

[Tschrock]

Version

4.4.7

What happened?

I'm trying to download a lot of submissions, and I often come back after a while to find the download command errored and stopped halfway through. Most of the time it's a gateway timeout or a network related error, though the last example I have is an encoding error. A lot of these are transient errors that would resolve if the request was retried, but instead of retrying or continuing, these errors short-circut everything and stop the program.

Examples of errors I've seen:

Error: An error occurred during download: ChunkedEncodingError(ProtocolError("Connection broken: InvalidChunkLength(got length b'', 0 bytes read)", InvalidChunkLength(got length b'', 0 bytes read))).

Error: An error occurred during download: ConnectionError(ProtocolError('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))).

How to reproduce the bug?

You can easily reproduce this by

• start a user download
• unplug the network
• wait a few minutes
• reconnect the network

I'd expect that the download would pause for a moment and then retry after a few seconds, and once retries are exhausted move on to the next one. Instead it just prints an error and exits.

Relevant log output

...

3/47 0030522212 Happy Birthday, Pilz!                                                [ DOWNLOAD ]
Error: An error occurred during download: ChunkedEncodingError(ProtocolError("Connection broken: InvalidChunkLength(got length b'', 0 bytes read)", InvalidChunkLength(got length b'', 0 bytes read))).

Added users         : 2
Modified users      : 1
Added userpages     : 2

...

[MatteoCampinoti94]

It already performs a retry when an error occurs, but not on major errors like that.

# falocalrepo download users --help

--retry INTEGER         Retry downloads.  [default: 1; 1<=x<=5]

PS: The program is now at version 4.5.0 :)