Hacker News with Generative AI: Query Engines

Query Engines: Gatekeepers of the Parquet File Format (duckdb.org)
TL;DR: Mainstream query engines do not support reading newer Parquet encodings, forcing systems like DuckDB to default to writing older encodings, thereby sacrificing compression.
Apache DataFusion (apache.org)
DataFusion is an extensible query engine written in Rust that uses Apache Arrow as its in-memory format.
Apache DataFusion: Fast, Embeddable, Modular Analytic Query Engine [pdf] (nerdnetworks.org)