Hacker News with Generative AI: Query Engines

Query Engines: Push vs. Pull (2021) (justinjaffray.com)
People talk a lot about “pull” vs. “push” based query engines, and it’s pretty obvious what that means colloquially, but some of the details can be a bit hard to figure out.
Query Engines, Databases, Software Engineering
101 points by tanelpoder 12 days ago | 9 comments
Query Engines: Gatekeepers of the Parquet File Format (duckdb.org)
TL;DR: Mainstream query engines do not support reading newer Parquet encodings, forcing systems like DuckDB to default to writing older encodings, thereby sacrificing compression.
Database Systems, Data Formats, Performance, Compression, Query Engines
32 points by tosh 87 days ago | 1 comments
Apache DataFusion (apache.org)
DataFusion is an extensible query engine written in Rust that uses Apache Arrow as its in-memory format.
Query Engines, Apache, Rust, Data Processing
156 points by thebuilderjr 104 days ago | 47 comments
Apache DataFusion: Fast, Embeddable, Modular Analytic Query Engine [pdf] (nerdnetworks.org)
Apache DataFusion, Big Data, Query Engines, Data Processing, Open Source
12 points by hambandit 109 days ago | 0 comments