Hacker News with Generative AI: Data Replication

Stop syncing everything (sqlsync.dev)
Partial replication sounds easy—just sync the data your app needs, right? But choosing an approach is tricky: logical replication precisely tracks every change, complicating strong consistency, while physical replication avoids that complexity but requires syncing every change, even discarded ones. What if your app could combine the simplicity of physical replication with the efficiency of logical replication? That’s the key idea behind Graft, the open-source transactional storage engine I’m launching today.

Databases, Open Source, Software, Data Replication

656 points by neilk 26 days ago | 126 comments

Reliably replicating data between Postgres and ClickHouse (benjaminwootton.com)
In this series of articles we are going to demonstrate how to reliably replicate data from PostgreSQL to ClickHouse.

Databases, PostgreSQL, ClickHouse, Data Replication

97 points by saisrirampur 65 days ago | 42 comments

Show HN: OLake[open source] Fastest database to Iceberg data replication tool (ycombinator.com)
Hi HN,<p>Today we’re excited to introduce OLake (github.com/datazip-inc/olake, 130+ and growing fast), an open-source tool built to help you replicate Database (MongoDB, for now, mysql and postgres under development) data into Data Lakehouse at faster speed without any hassle of managing Debezium or kafka (at least 10x faster than Airbyte and Fivetran at fraction of the cost, refer docs for benchmarks - https://olake.io/docs/connectors/mongodb/benchmarks).

Open Source, Data Replication, Database, Data Lakehouse, Benchmarking

14 points by pkhodiyar 75 days ago | 3 comments

Replited: Replicate SQLite to anywhere (S3\FTP\webdav\Google Drive\Dropbox,etc.) (github.com/lichuang)
Replicate SQLite to every where(S3\ftp\webdav\google drive\dropbox,etc)

Databases, Cloud Storage, Software, Data Replication

28 points by thunderbong 202 days ago | 2 comments

ClickHouse Data Modeling for Postgres Users (clickhouse.com)
Last month, we acquired PeerDB, a company that specializes in Postgres CDC. PeerDB makes it fast and simple to replicate data from Postgres to ClickHouse. A common question from PeerDB users is how to model their data in ClickHouse after the replication process to maximize the benefits of ClickHouse.