System.IO.InvalidDataException on TransactWrite

[alexey62soft]

Hello everyone!

I'm using EfficienDynamoDb v0.9.16 (runtime - .NET 6) like that:

var updateOperation = _dbContext.TransactWrite()
    .WithClientRequestToken(Guid.NewGuid().ToString())
    .WithItems(CreateTransactItems(/* data is here */))
    .ExecuteAsync(cx);

try
{
    await updateOperation;
}
catch (TransactionCanceledException tcex)
{
    /* Reprocessing is here */
}

And recently I started receive the following error is DynamoDB responds with error:

System.IO.InvalidDataException: The archive entry was compressed using an unsupported compression method.
at System.IO.Compression.Inflater.Inflate(FlushCode flushCode)
at System.IO.Compression.Inflater.ReadInflateOutput(Byte* bufPtr, Int32 length, FlushCode flushCode, Int32& bytesRead)
at System.IO.Compression.Inflater.InflateVerified(Byte* bufPtr, Int32 length)
at System.IO.Compression.DeflateStream.CopyToStream.WriteAsyncCore(ReadOnlyMemory`1 buffer, CancellationToken cancellationToken)
at System.Net.Http.HttpConnection.CopyToContentLengthAsync(Stream destination, Boolean async, UInt64 length, Int32 bufferSize, CancellationToken cancellationToken)
at System.Net.Http.HttpConnection.ContentLengthReadStream.CompleteCopyToAsync(Task copyTask, CancellationToken cancellationToken)
at System.IO.Compression.DeflateStream.CopyToStream.CopyFromSourceToDestinationAsync()
at EfficientDynamoDb.Internal.ErrorHandler.ProcessErrorAsync(DynamoDbContextMetadata metadata, HttpResponseMessage response, CancellationToken cancellationToken)
at EfficientDynamoDb.Internal.ErrorHandler.ProcessErrorAsync(DynamoDbContextMetadata metadata, HttpResponseMessage response, CancellationToken cancellationToken)
at EfficientDynamoDb.Internal.ErrorHandler.ProcessErrorAsync(DynamoDbContextMetadata metadata, HttpResponseMessage response, CancellationToken cancellationToken)
at EfficientDynamoDb.Internal.HttpApi.SendAsync(DynamoDbContextConfig config, HttpContent httpContent, CancellationToken cancellationToken)
at EfficientDynamoDb.DynamoDbContext.TransactWriteItemsAsync(BuilderNode node, CancellationToken cancellationToken)
at /* application code mentioned above*/

Can you help me understand the reason for the error? Is that associated with the library itself?

Update: setting DecompressionMethods.None here resolves the problem.

[alexey62soft]

@firenero, maybe you can take a look on the error?

[firenero]

@alexey62soft Seems like something is wrong with gzip decompression during DDB error processing. As you noticed, we use automatic decompression so getting an error there is unusual. But since setting AutomaticDecompressions to None there is definitely something wrong with compression. There are several things we can do to troubleshoot it:

1. Do you only get this exception when processing DynamoDB error? Could you try running a query or scan request with large response body (larger than 4KB) to test if decompression works for successful operations?
2. Do you get this error consistently with some requests or it's random and can't be reproduced reliably?
3. If you can reproduce it, could you try to put breakpoint to this line and confirm that the response stream is indeed gzip compressed?

[alexey62soft]

@firenero First of all, thank you for your response, and sorry for the delay!

Do you only get this exception when processing DynamoDB errors? Could you try running a query or scan request with a large response body (larger than 4KB) to test if decompression works for successful operations?

I used the query operation that has the response with "Content-Length: 29896 bytes" when "AutomaticDecompression = DecompressionMethods.None". With "AutomaticDecompression = DecompressionMethods.GZip", I still received a successful response with "Content-Length: 1143 bytes" and "Content-Encoding: gzip". No errors - everything was processed correctly.

Do you get this error consistently with some requests or it's random and can't be reproduced reliably?

I realized that issue just a couple of days ago, but can't prove the issue itself appears recently. Anyway I can notice consistent behavior - it always failed on error responses with large "Content-Length" - I used the "ReturnValuesOnConditionCheckFailure.AllOld" setting for transaction items in the code.

If you can reproduce it, could you try to put a breakpoint to this line and confirm that the response stream is indeed gzip compressed?

Yes, I was able to reproduce that locally using different configurations. I assume the reason of the error - AWS API request compression support. In particular, AWS DynamoDB API does not compress error responses (we are having same Content-Length) but provides different checksum (CRC32).

AutomaticDecompression - DecompressionMethods.None

Response.RequestMessage.Headers:

1. X-Amz-Date,[20241224T150420Z]
2. x-amz-security-token,[...]
3. Host,[dynamodb.us-east-1.amazonaws.com]
4. Authorization,[...]
5. Transfer-Encoding,[chunked]

Response.Headers:

1. Date,[Tue, 24 Dec 2024 14:34:43 GMT]
2. Connection,[keep-alive]
3. x-amzn-RequestId,[...]
4. x-amz-crc32,[872488556]

Response.Content.Type: System.Net.Http.HttpConnectionResponseContent

Response.Content.Headers:

1. Content-Type,[application/x-amz-json-1.0]
2. Content-Length,[30479]

Response.Stream: System.Net.Http.HttpConnection.ContentLengthReadStream

Result: Processing is fine. The library was able to process the BadRequest error response.

AutomaticDecompression = DecompressionMethods.GZip

Response.Request.Headers:

1. X-Amz-Date,[20241224T145815Z]
2. x-amz-security-token,[...]
3. Host,[dynamodb.us-east-1.amazonaws.com]
4. Authorization,[...]
5. Transfer-Encoding,[chunked]
6. Accept-Encoding,[gzip]

Response.Headers:

1. Date,[Tue, 24 Dec 2024 14:27:50 GMT]
2. Connection,[keep-alive]
3. x-amzn-RequestId,[...]
4. x-amz-crc32,[3431636590]

Response.Content.Type: System.Net.Http.DecompressionHandler.GZipDecompressedContent

Response.Content.Headers:

1. Content-Type,[application/x-amz-json-1.0]

Response.Content.OriginalContent.Headers:

1. Content-Type,[application/x-amz-json-1.0]
2. Content-Length,[30479]
3. Content-Encoding,[gzip]

Response.Stream: System.IO.Compression.GZipStream

Result: Processing was failed with System.IO.InvalidDataException provided in the issue description. The library was NOT able to process the BadRequest error response.

AutomaticDecompression = DecompressionMethods.None, but Accept-Encoding = gzip (manually provided on the request in EfficientDynamoDb.Internal.HttpApi)

Response.RequestMessage.Headers:

1. Accept-Encoding,[gzip]
2. X-Amz-Date,[20241224T150946Z]
3. x-amz-security-token,[...]
4. Host,[dynamodb.us-east-1.amazonaws.com]
5. Authorization,[...]
6. Transfer-Encoding,[chunked]

Response.Headers:

1. Date,[Tue, 24 Dec 2024 15:10:04 GMT]
2. Connection,[keep-alive]
3. x-amzn-RequestId,[...]
4. x-amz-crc32,[3431636590]

Response.Content.Type: System.Net.Http.HttpConnectionResponseContent

Response.Content.Headers:

1. Content-Type,[application/x-amz-json-1.0]
2. Content-Length,[30479]
3. Content-Encoding,[gzip]

Response.Stream: System.Net.Http.HttpConnection.ContentLengthReadStream

Result: Processing is fine. The library processed the BadRequest error response without additional decompression.