Reuse sigv4 chunk buffer

[RanVaknin]

Background

Synchronous S3 aws-chunked uploads process payloads in 128 KiB chunks so they can be signed and checksummed without loading the entire request into memory. However, ChunkedEncodedInputStream allocated a new 128 KiB array for every chunk even though only one chunk is consumed at a time, creating unnecessary allocation and CPU overhead that grows with payload size.

Changes

1. New benchmark added - Sigv4ChunkedSigningBenchmark signs and fully consumes an in memory S3 PUT payload (excluding network so that SDK signing and encoding costs remain narrow and accurate). It covers 64 KiB, 1 MiB, and 16 MiB payloads across signed payload, signed checksum trailer, and unsigned checksum trailer modes.

Capturing JFRs from the benchmark showed that ChunkedEncodedInputStream.getChunk accounted for 51%-74% of self CPU and 88%-100% of sampled allocation weight in the profiles. The existing code allocated a new 128 KiB array for every chunk and another array when probing for EOF, so a 16 MiB request allocated 16 MiB of temporary chunk arrays even though only one chunk is consumed at a time.

2. Reusing chunk buffer - This optimization target changes ChunkedEncodedInputStream to lazily allocates one chunk array per stream, refills it only after the previous chunk is exhausted, and releases it at EOF, read failure, or close.

From a customer perspective there is no new API or behavior. Synchronous S3 customers using aws-chunked uploads benefit through less allocations and consequently less CPU spent allocating and zeroing arrays, with the largest effect on large payloads and unsigned checksum trailer uploads (where crpyto work doesn't dominate).

Results

Payload	Mode	Baseline ops/s	Candidate ops/s	Delta	Baseline B/op	Candidate B/op	Allocation delta
64 KiB	Signed payload	12790.760	15086.183	+17.9%	286540	155460	-45.7%
64 KiB	Signed checksum trailer	12062.976	14085.332	+16.8%	295328	164192	-44.4%
64 KiB	Unsigned checksum trailer	29173.288	44726.624	+53.3%	279056	147936	-47.0%
1 MiB	Signed payload	1159.483	1322.558	+14.1%	1258366	209843	-83.3%
1 MiB	Signed checksum trailer	1127.180	1278.246	+13.4%	1267259	218548	-82.8%
1 MiB	Unsigned checksum trailer	5370.273	11216.463	+108.9%	1202080	153368	-87.2%
16 MiB	Signed payload	73.385	82.749	+12.8%	17916296	1141989	-93.6%
16 MiB	Signed checksum trailer	71.826	80.489	+12.1%	17928455	1145979	-93.6%
16 MiB	Unsigned checksum trailer	368.025	773.295	+110.1%	17024794	245609	-98.6%

Before (64KiB unsigned trailer case)

Memory:

After (64KiB unsigned trailer case)

Memory:

Canary

30min canary runtime showed improvements across the main resource metrics. The optimization branch used ~50% less host CPU, 12.8% less JVM CPU, 9.4% less host memory, and completed 15.9% more operations per CPU second.

HeapAfterGC was the one result that looked worse. Further investigation showed that this metric retains the value from the most recent garbage collection. Because the PR performed far fewer garbage collections, the two values represented different points in the garbage collection cycle and were not directly comparable. Forced GC measurements showed that the PR’s retained heap remained stable rather than growing over time, so there was no evidence of a memory leak.

Metric	Direction	Magnitude	p-value	Candidate	Baseline
HeapAfterGC_p50	higher	126.1%	6.53e-84	2.500e+07	1.105e+07
HeapAfterGC_p90	higher	126.1%	6.53e-84	2.500e+07	1.105e+07
cpu_usage_user_p50	lower	54.1%	2.14e-19	0.7828	1.71
cpu_usage_user_p90	lower	54.1%	2.14e-19	0.7828	1.71
SystemCpuLoad_p50	lower	49.9%	3.37e-19	1.21	2.41
SystemCpuLoad_p90	lower	49.9%	3.37e-19	1.21	2.41
cpu_usage_active_p50	lower	50.2%	4.96e-19	1.22	2.44
cpu_usage_active_p90	lower	50.2%	4.96e-19	1.22	2.44
cpu_usage_system_p50	lower	48.1%	8.75e-17	0.2385	0.4596
cpu_usage_system_p90	lower	48.1%	8.75e-17	0.2385	0.4596
OpsPerCpuSecond_p50	higher	15.9%	1.94e-16	404.8	349.2
OpsPerCpuSecond_p90	higher	15.9%	1.94e-16	404.8	349.2
CpuUsage_p50	lower	12.8%	5.08e-16	5.41	6.21
CpuUsage_p90	lower	12.8%	5.08e-16	5.41	6.21
mem_used_p50	lower	9.4%	4.19e-07	2.799e+09	3.088e+09
mem_used_p90	lower	9.4%	4.19e-07	2.799e+09	3.088e+09
mem_used_percent_p50	lower	9.4%	4.19e-07	16.99	18.74
mem_used_percent_p90	lower	9.4%	4.19e-07	16.99	18.74
mem_available_p50	higher	2.2%	4.28e-07	1.331e+10	1.302e+10
mem_available_p90	higher	2.2%	4.28e-07	1.331e+10	1.302e+10

[github-actions]

This pull request has been closed and the conversation has been locked. Comments on closed PRs are hard for our team to see. If you need more assistance, please open a new issue that references this one.