Introducing Flink's Native S3 FileSystem: Built for Performance, Designed for Production (opens in new tab)

Apache Flink relies on the underlying filesystem for much of its work: reading and writing application data, materializing streaming sinks, and storing checkpoints and savepoints for recovery. For years, S3 support in Flink meant choosing between two Hadoop-based plugins, each with its own trade-offs and configuration quirks. With Flink 2.3, there is a better option. Today we’re introducing flink-s3-fs-native, a ground-up, Hadoop-free S3 filesystem built specifically for Flink. It ships as an...