Skip to content
This repository has been archived by the owner on Nov 15, 2024. It is now read-only.

Commit

Permalink
Seek the write buffer to zero after flushing
Browse files Browse the repository at this point in the history
While `.truncate(0)` is called on the buffer object
after writing, this won't reset the current position
to zero. Subsequent writes will therefore just keep
extending the buffer and writing more and more data.
The fix is pretty simple -- `.seek(0)` after truncating.

It looks like this will come up any time an avro file
grows past 64,000 bytes (i.e., SYNC_INTERVAL bytes).
  • Loading branch information
JohnEmhoff committed Feb 21, 2020
1 parent 6a6c32d commit d6d76f1
Showing 1 changed file with 2 additions and 1 deletion.
3 changes: 2 additions & 1 deletion src/spavro/datafile.py
Original file line number Diff line number Diff line change
Expand Up @@ -208,7 +208,8 @@ def _write_block(self):
self.writer.write(self.sync_marker)

# reset buffer
self.buffer_writer.truncate(0)
self.buffer_writer.truncate(0)
self.buffer_writer.seek(0)
self.block_count = 0

def append(self, datum):
Expand Down

0 comments on commit d6d76f1

Please sign in to comment.