Hacker News with Generative AI: Data Warehousing

Logical Replication from Postgres to Iceberg (crunchydata.com)
Operational and analytical workloads have historically been handled by separate database systems, though they are starting to converge. We built Crunchy Data Warehouse to put PostgreSQL at the frontier of analytics systems, using modern technologies like Iceberg and a hybrid query engine.

PostgreSQL, Data Warehousing, Analytics, Iceberg

8 points by theanirudh 4 days ago | 1 comments

OLAP Hierarchical Aggregation with DuckDB SQL Recursive Common Table Expressions (medium.com)
Aggregation for dimensional hierarchies doesn’t require costly Business Intelligence (BI) tools. You can use recursive SQL techniques to express your hierarchical data in relational form, allowing for easy and fast aggregation along multiple levels and dimensions.

SQL, Databases, Data Analysis, Data Warehousing, Recursive SQL

7 points by tanelpoder 16 days ago | 1 comments

Apache iceberg the Hadoop of the modern-data-stack? (det.life)
In the early 2010s, Apache Hadoop dominated the big data conversation. Organizations raced to adopt it, seeing it as the cornerstone for scalable, distributed storage and processing. Today, Apache Iceberg is emerging as a cornerstone for data lakes and lakehouses in the modern data stack.

Apache Iceberg, Data Warehousing, Big Data, Apache Hadoop

115 points by samrohn 51 days ago | 65 comments

SAP Databricks (databricks.com)

SAP, Data Warehousing, Cloud Computing

12 points by ClosedPistachio 72 days ago | 0 comments

Apache Iceberg (apache.org)
Iceberg is a high-performance format for huge analytic tables.

Data Warehousing, Big Data, Apache

207 points by jacobmarble 93 days ago | 66 comments

Use Cases for ChDB, a Powerful In-Memory OLAP SQL Engine (runportcullis.co)
Clickhouse is quickly becoming a crowd favorite real-time data warehouse platform for organizations looking to take advantage of blazing fast query speeds in OLAP scenarios that power mission-critical applications and embedded analytics.

Databases, Data Warehousing, Real-time Data, OLAP, SQL

9 points by auxten 130 days ago | 2 comments

From Zero to Terabytes: Building SaaS Analytics with ClickHouse (crisp.chat)
At Crisp, we help businesses manage all their customer conversations in one place—whether through chat, email, WhatsApp, or other channels - through a help desk platform. As our customers' needs grew, they asked for more detailed insights into their customer support, like response times and team performance.

Data Warehousing, SaaS, Analytics, ClickHouse

14 points by baptistejamin 172 days ago | 7 comments

DataChain: DBT for Unstructured Data (github.com/iterative)
DataChain is a modern Pythonic data-frame library designed for artificial intelligence.

Python, Data Science, Artificial Intelligence, Data Warehousing

163 points by shcheklein 173 days ago | 26 comments

Dbt – Incremental but Incomplete (tobikodata.com)
Earlier this month, dbtTM launched microbatch incremental models in version 1.9, a highly requested feature since the experimental insert_by_period was introduced back in 2018. While it's certainly a step in the right direction, it has been a long time coming.

Software, Data Engineering, Data Warehousing

67 points by captaintobs 193 days ago | 18 comments

I spent 5 hours learning how ClickHouse built their internal data warehouse (vutr.substack.com)
My name is Vu Trinh, and I am a data engineer.

Data Warehousing, Databases, ClickHouse, Data Engineering, Substack

47 points by markhneedham 214 days ago | 5 comments

6 Powerful Databricks Alternatives for Data Lakes and Lakehouses (definite.app)
Databricks has established itself as a leader in the data lake and lakehouse space, offering a powerful platform for big data processing and analytics.

Data Lakes, Data Warehousing, Big Data, Cloud Computing, Analytics

13 points by TenMatrix 228 days ago | 2 comments

Pinot for Low-Latency Offline Table Analytics (uber.com)

Data Warehousing, Data Analytics, Apache Pinot

27 points by PeterCorless 239 days ago | 2 comments

Why Did Databricks Open-Source Unity Catalog? (medium.com)

Open Source, Databricks, Data Warehousing, Cloud Computing

19 points by Merick 242 days ago | 2 comments

Launch HN: Roe AI (YC W24) – AI-powered data warehouse to query multimodal data (ycombinator.com)

Artificial Intelligence, Data Warehousing, YC, Startups

60 points by richardmeng 260 days ago | 35 comments

ClickHouse acquires PeerDB for native Postgres CDC integration (peerdb.io)