Postgres to ClickHouse: Data Modeling Tips V2
(clickhouse.com)
It is becoming increasingly common for customers to use Postgres and ClickHouse together, with Postgres powering transactional workloads and ClickHouse powering analytics.
It is becoming increasingly common for customers to use Postgres and ClickHouse together, with Postgres powering transactional workloads and ClickHouse powering analytics.
Making Postgres scale
(pgdog.dev)
Postgres scales. No other two words that I’ve ever heard of, produced more controversy. At least in the circles I hang out in, in the company basement where infrastructure elves make the Rails app go brrr. A lot of people believe, against all odds, and marketing campaigns by Big NoSQL, that technology you know is better than the devil you just heard pitched at the Engineering Leadership meeting.
Postgres scales. No other two words that I’ve ever heard of, produced more controversy. At least in the circles I hang out in, in the company basement where infrastructure elves make the Rails app go brrr. A lot of people believe, against all odds, and marketing campaigns by Big NoSQL, that technology you know is better than the devil you just heard pitched at the Engineering Leadership meeting.
Did u ever read so hard u accidentally wrote?
(danslimmon.com)
Owning a production Postgres database is never boring.
Owning a production Postgres database is never boring.
Show HN: PG-Capture – a better way to sync Postgres with Algolia (or Elastic)
(onrender.com)
Schema-based Change-Data-Capture for Postgres
Schema-based Change-Data-Capture for Postgres
Postgres as a Graph Database: (Ab)Using PgRouting
(supabase.com)
pgRouting is a Postgres extension. It's often used for finding the “shortest path” between two locations, however it's a hidden gem in Postgres and can be used for basic graph functionality.
pgRouting is a Postgres extension. It's often used for finding the “shortest path” between two locations, however it's a hidden gem in Postgres and can be used for basic graph functionality.
The Great Re-shard: adding Postgres capacity (again) with zero downtime (2023)
(notion.com)
Earlier this year, we swapped out Notion’s live database cluster for a larger one without taking downtime.
Earlier this year, we swapped out Notion’s live database cluster for a larger one without taking downtime.
A Language Server for Postgres
(github.com/supabase-community)
A collection of language tools and a Language Server Protocol (LSP) implementation for Postgres, focusing on developer experience and reliable SQL tooling.
A collection of language tools and a Language Server Protocol (LSP) implementation for Postgres, focusing on developer experience and reliable SQL tooling.
A major Postgres upgrade with zero downtime
(instantdb.com)
Right before Christmas we discovered that our Aurora Postgres instance needed a major version upgrade. We found a great essay by the Lyft team, showing how they ran their upgrade with about 7 minutes of downtime.
Right before Christmas we discovered that our Aurora Postgres instance needed a major version upgrade. We found a great essay by the Lyft team, showing how they ran their upgrade with about 7 minutes of downtime.
"Just Use Postgres" Book
(manning.com)
You probably don’t need a collection of specialty databases. Just use Postgres instead!
You probably don’t need a collection of specialty databases. Just use Postgres instead!
Reads Causing Writes in Postgres
(jesipow.com)
It is good practice to regularly inspect the statements running in the hot path of your Postgres instance. One way to do this is to examine the pg_stat_statements view, which shows various statistics about the SQL statements executed by the Postgres server.
It is good practice to regularly inspect the statements running in the hot path of your Postgres instance. One way to do this is to examine the pg_stat_statements view, which shows various statistics about the SQL statements executed by the Postgres server.
Shrinking a Postgres Table
(johnnunemaker.com)
Ok folks, this is kind of a weird one. I'm going to put it in the "you won't ever need this, but if you do, you are going to be glad I wrote this up for ya" category.
Ok folks, this is kind of a weird one. I'm going to put it in the "you won't ever need this, but if you do, you are going to be glad I wrote this up for ya" category.
Django and Postgres for the Busy Rails Developer
(andyatkinson.com)
About 10 years ago I wrote a post PostgreSQL for the Busy MySQL Developer, as part of switching from MySQL to Postgres for my personal and professional projects, wherever I could.
About 10 years ago I wrote a post PostgreSQL for the Busy MySQL Developer, as part of switching from MySQL to Postgres for my personal and professional projects, wherever I could.
Running Durable Workflows in Postgres Using DBOS
(supabase.com)
Michael Stonebraker is the inventor of Postgres and a Turing Award winner. His latest venture is DBOS, a three-year joint research project between Stanford and MIT. The DBOS team have built a Durable Workflow engine using Postgres. It's one of the of the more elegant designs I've seen, leveraging the features of Postgres to keep it lightweight and fast.
Michael Stonebraker is the inventor of Postgres and a Turing Award winner. His latest venture is DBOS, a three-year joint research project between Stanford and MIT. The DBOS team have built a Durable Workflow engine using Postgres. It's one of the of the more elegant designs I've seen, leveraging the features of Postgres to keep it lightweight and fast.
Show HN: Performing a zero downtime major version upgrade on RDS Aurora Postgres
(medplum.com)
Medplum is built on Postgres. Until recently, our hosted Medplum service was using an Amazon Web Services (AWS) RDS Aurora Postgres cluster running version 12.16. Since v12 is rather outdated and nearing the end of its standard support window on RDS, it was time to plan our upgrade to the newest version available on RDS, v16.4.
Medplum is built on Postgres. Until recently, our hosted Medplum service was using an Amazon Web Services (AWS) RDS Aurora Postgres cluster running version 12.16. Since v12 is rather outdated and nearing the end of its standard support window on RDS, it was time to plan our upgrade to the newest version available on RDS, v16.4.
Keyset cursors, not offsets, for Postgres pagination
(sequinstream.com)
We're building Sequin, a Postgres CDC tool to streams and queues like Kafka, SQS, HTTP endpoints, and more. Efficient, correct pagination is paramount for our backfill process, so we use keyset cursors extensively.
We're building Sequin, a Postgres CDC tool to streams and queues like Kafka, SQS, HTTP endpoints, and more. Efficient, correct pagination is paramount for our backfill process, so we use keyset cursors extensively.
Shrinking a Postgres Table
(johnnunemaker.com)
Ok folks, this is kind of a weird one. I'm going to put it in the "you won't ever need this, but if you do, you are going to be glad I wrote this up for ya" category.
Ok folks, this is kind of a weird one. I'm going to put it in the "you won't ever need this, but if you do, you are going to be glad I wrote this up for ya" category.
Show HN: Neon-PostgREST – Build a Supabase-Like API/SDK DevX over Neon
(github.com/zuplo-samples)
Are you using/considering Neon (or your own hosted Postgres) - but wish you had the ability to interact with your database via API/SDK like Supabase? Look no further! This project shows you exactly how to recreate the Supabase API and developer experience. Try the Demo.
Are you using/considering Neon (or your own hosted Postgres) - but wish you had the ability to interact with your database via API/SDK like Supabase? Look no further! This project shows you exactly how to recreate the Supabase API and developer experience. Try the Demo.
What I wish someone told me about Postgres
(challahscript.com)
I’ve been working professionally for the better part of a decade on web apps and, in that time, I’ve had to learn how to use a lot of different systems and tools.
I’ve been working professionally for the better part of a decade on web apps and, in that time, I’ve had to learn how to use a lot of different systems and tools.
Show HN: BemiDB – Postgres read replica optimized for analytics
(github.com/BemiHQ)
BemiDB is a Postgres read replica optimized for analytics, It consists of a single binary that seamlessly connects to a Postgres database, replicates the data in a compressed columnar format, and allows you to run complex queries using its Postgres-compatible analytical query engine.
BemiDB is a Postgres read replica optimized for analytics, It consists of a single binary that seamlessly connects to a Postgres database, replicates the data in a compressed columnar format, and allows you to run complex queries using its Postgres-compatible analytical query engine.
Neon Authorize: Simplifying row-level security for Postgres
(neon.tech)
Today we’re launching Neon Authorize, a tool that aims to simplify the usage of Postgres row-level security policies while enabling new deployment models for app developers.
Today we’re launching Neon Authorize, a tool that aims to simplify the usage of Postgres row-level security policies while enabling new deployment models for app developers.
Graph Retrieval Using Postgres Recursive CTEs
(sheshbabu.com)
Did you know you can use Postgres as a graph database for certain usecases?
Did you know you can use Postgres as a graph database for certain usecases?
Pg_parquet: An extension to connect Postgres and parquet
(crunchydata.com)
Today, we’re excited to release pg_parquet - an open source Postgres extension for working with Parquet files.
Today, we’re excited to release pg_parquet - an open source Postgres extension for working with Parquet files.
How we build forms in LiveView and LiveSvelte
(sequinstream.com)
We're Sequin, an open source message stream built on Postgres. We have a console for managing tables and streams, and that console is built with LiveView + LiveSvelte.
We're Sequin, an open source message stream built on Postgres. We have a console for managing tables and streams, and that console is built with LiveView + LiveSvelte.
Putting a full power search engine in Ecto
(moosie.us)
ParadeDB is a set of extensions that add pretty amazing search and analytics features to Postgres. In particular, ParadeDB embeds Tantivy as an extension via pgrx.
ParadeDB is a set of extensions that add pretty amazing search and analytics features to Postgres. In particular, ParadeDB embeds Tantivy as an extension via pgrx.
Optimizing Postgres table layout for maximum efficiency
(r.ena.to)
When modeling a Postgres database, you probably don’t give much thought to the order of columns in your tables. After all, it seems like the kind of thing that wouldn’t affect storage or performance. But what if I told you that simply reordering your columns could reduce the size of your tables and indexes by 20%? This isn’t some obscure database trick — it’s a direct result of how Postgres aligns data on disk.
When modeling a Postgres database, you probably don’t give much thought to the order of columns in your tables. After all, it seems like the kind of thing that wouldn’t affect storage or performance. But what if I told you that simply reordering your columns could reduce the size of your tables and indexes by 20%? This isn’t some obscure database trick — it’s a direct result of how Postgres aligns data on disk.
Deep Dive into Postgres Write-Ahead Logs
(artie.com)
Write-ahead logs are one of four components required for logical replication. They are used to ensure data durability, meaning your data is consistent and protected against data loss. Even if a server machine fails, write-ahead logs ensure your changes persist.
Write-ahead logs are one of four components required for logical replication. They are used to ensure data durability, meaning your data is consistent and protected against data loss. Even if a server machine fails, write-ahead logs ensure your changes persist.
A few secure, random bytes without `pgcrypto`
(brandur.org)
In Postgres it’s common to see the SQL random() function used to generate a random number, but it’s a pseudo-random number generator, and not suitable for cases where real randomness is required critical. Postgres also provides a way of getting secure random numbers as well, but only through the use of the pgcrypto extension, which makes gen_random_bytes available.
In Postgres it’s common to see the SQL random() function used to generate a random number, but it’s a pseudo-random number generator, and not suitable for cases where real randomness is required critical. Postgres also provides a way of getting secure random numbers as well, but only through the use of the pgcrypto extension, which makes gen_random_bytes available.