Configurable MessageDecoder Buffer Size

[zoewangg]

Expected Behavior

Allow users to configure MessageDecoder initial buffer size

Current Behavior

Currently, the initial buffer size in MessageDecoder is 2MB and there's no way to configure it.

aws-sdk-java-v2/flow/src/main/java/software/amazon/eventstream/MessageDecoder.java

Line 18 in a00d993

private static final int INITIAL_BUFFER_SIZE = 2048 * 1024;

Your Environment

• AWS Java SDK version used: 2.0.0-preview-13-SNAPSHOT

[millems]

I don't see any upvotes on this. Is this still something that customers want?

[fommil]

There's also no way for the buffer to reduce in size if it is ever increased through a call to buf = ByteBuffer.allocate(currentPrelude.getTotalLength()); (looks like that's defined over in https://github.com/awslabs/aws-eventstream-java/blob/master/src/main/java/software/amazon/eventstream/MessageDecoder.java)

I see 10MB buffers regularly, which is a per-shard overhead when using the KCL (via EventStreamAsyncResponseTransformer).

[fommil]

See awslabs/aws-eventstream-java#4 for a proof of concept PR to the eventstream repo allowing the buffer to grow AND shrink.

I drop this into my project so that it monkey patches the AWS libraries.