Kafka Share Groups and Parallelizing Consumption - Part 2: Producer Batches and share.acquire.mode (opens in new tab)

In the last post we used simulated consumer processing time to reveal how important it is to set an appropriate value for max.poll.records. The rule of thumb was a value somewhat lower than: group.share.partition.max.record.locks / number of consumers per partition But there’s more to parallel consumption than max.poll.records. The size of producer batches also plays a role when using the default share.acquire.mode (batch_optimized).