gzip and snappy compression not working when min Produce RPC version is set to 3 on broker side #4956
Comments
|
As discussed offline, this also affects lz4. And zstd also has the same type of logic bug - it checks for a specific version instead of checking that the supported version is greater or equal than the minimum version required. We should fix all 4 at the same time. |
3 tasks
7 tasks
ijuma
added a commit
to apache/kafka
that referenced
this issue
Feb 2, 2025
…returns produce v0 (#18727) Return produce v0-v2 as supported versions in `ApiVersionsResponse`, but disable support for it everywhere else. Since clients pick the highest supported version by both client and broker during version negotiation, this solves the problem with minimal tech debt (even though it's not ideal that `ApiVersionsResponse` becomes inconsistent with the actual protocol support). Add one test for the socket server handling (in `ProcessorTest`) and one test for the client behavior (in `ProduceRequestTest`). Adjust a couple of api versions tests to verify the new behavior. Finally, include a few clean-ups in `ApiKeys`, `Protocol`, `ProduceRequest`, `ProduceRequestTest` and `BrokerApiVersionsCommandTest`. Reference to related librdkafka issue: confluentinc/librdkafka#4956 Reviewers: Jun Rao <[email protected]>, Chia-Ping Tsai <[email protected]>, Stanislav Kozlovski <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
With the KIP-896 support in AK, gzip and snappy compression not working when min Produce RPC version is set to 3.
Test in 0077.c fails when the broker supports v3 as min version. The produce request falls back to producing without compression in librdkafka.
Logs shows something like this:
The text was updated successfully, but these errors were encountered: