Hacker News with Generative AI: Database

Abusing DuckDB-WASM by making SQL draw 3D graphics (Sort Of) (hey.earth)
I had this slightly crazy idea: Could I ditch most of the conventional JavaScript game loop and rendering logic and build a 3D game engine where SQL queries did the heavy lifting?
Database, Graphics, JavaScript, WebAssembly, Game Development
198 points by tanelpoder 4 days ago | 32 comments
Everything You Need to Know About Incremental View Maintenance (materializedview.io)
Incremental view maintenance has been a hot topic lately.
Database, Data Management, Optimization, Performance
9 points by riccomini 7 days ago | 6 comments
New Apache Cassandra Release Saves 400% IOPS (simplyblock.io)
On April 10, 2025, the Apache Software Foundation released version 5.0.4 of Apache Cassandra, bringing significant performance optimizations for all users—but especially for those relying on remotely attached storage like Amazon EBS. The standout feature in this release is an overhaul of the compaction algorithm aimed at slashing IOPS usage while increasing overall throughput.
Database, Apache Cassandra, Performance Optimization, Storage
5 points by noctarius 13 days ago | 0 comments
Faster interpreters in Go: Catching up with C++ (planetscale.com)
The SQL evaluation engine that ships with Vitess, the open-source database that powers PlanetScale, was originally implemented as an AST evaluator that used to operate directly on the SQL AST generated by our parser. Over this past year, we've gradually replaced it with a Virtual Machine which, despite being written natively in Go, performs similarly to the original C++ evaluation code in MySQL.
Go, Programming Languages, Performance, Database, Open Source
252 points by ksec 20 days ago | 56 comments
Show HN: AGX – Open-Source Data Exploration for ClickHouse (The New Standard?) (github.com/agnosticeng)
agx is a desktop application built with Tauri and SvelteKit that provides a modern interface for exploring and querying data using ClickHouse's embedded database engine (chdb).
Open Source, Data Exploration, ClickHouse, Database, Desktop Applications
48 points by didierfranc 38 days ago | 3 comments
Preview: Amazon S3 Tables and Lakehouse in DuckDB (duckdb.org)
TL;DR: We are happy to announce a new preview feature that adds support for Apache Iceberg REST Catalogs, enabling DuckDB users to connect to Amazon S3 Tables and Amazon SageMaker Lakehouse with ease.
Database, Data Storage, Cloud Computing, Preview
177 points by hn1986 38 days ago | 47 comments
Ask HN: Alternatives to Vector DB? (ycombinator.com)
A while back I was looking for a vector database that would work across Windows / Mac / Linux platforms. Some of the options required specific processors like Intel. I am curious if there are any alternatives to a Vector DB that can run cross platform and are easy to setup?
Database, Cross-Platform, Alternatives, Open Source, Vector Databases
21 points by tmaly 39 days ago | 37 comments
Selective async commits in PostgreSQL – balancing durability and performance (shayon.dev)
I was recently looking into some workloads that generate a lot of I/O and CPU contention on some very high-write code paths and came across synchronous_commit (https://www.postgresql.org/docs/current/wal-async-commit.html). It can be very tempting to turn this off globally because the performance gains in terms of I/O, CPU, and TPS (transactions per second) are very hard to overlook. I noticed I/O completely gone, CPU down 20% (at peak), and a 30% increase in TPS.
Database, Performance, PostgreSQL
109 points by shayonj 40 days ago | 37 comments
Nubmq: A high performant key-value store engine built from first principles (ycombinator.com)
Nubmq: A high performant key-value store engine built from first principles
Database, Key-Value Stores, Performance
39 points by nubskr 42 days ago | 18 comments
What if we just didn't decompress it? (spiraldb.com)
Vortex is unique in the way it evaluates filter and projection expressions by supporting full compute push-down, in many cases avoiding decompression entirely.
Database, Compression, Optimization, Performance
12 points by gatesn 51 days ago | 2 comments
An Experimental Study of Bitmap Compression vs. Inverted List Compression (dl.acm.org)
Bitmap compression has been studied extensively in the database area and many efficient compression schemes were proposed, e.g., BBC, WAH, EWAH, and Roaring. Inverted list compression is also a well-studied topic in the information retrieval community and many inverted list compression algorithms were developed as well, e.g., VB, PforDelta, GroupVB, Simple8b, and SIMDPforDelta.
Database, Information Retrieval, Compression, Algorithms, Experimental Studies
32 points by westurner 57 days ago | 6 comments
EdgeDB is now Gel and Postgres is the future (geldata.com)
We have news! EdgeDB is rebranding as Gel, more on that below.
Database, Software, Rebranding
165 points by mmastrac 59 days ago | 123 comments
Do It Yourself Database CDN with Embedded Replicas (turso.tech)
Imagine you have a user in Singapore, and your database is in the US. Every time the user makes a request, it has to travel halfway around the world, which can lead to high latency and poor performance.
Database, Performance, Cloud, Networking, CDN
19 points by emschwartz 73 days ago | 2 comments
Show HN: OLake[open source] Fastest database to Iceberg data replication tool (ycombinator.com)
Hi HN,<p>Today we’re excited to introduce OLake (github.com/datazip-inc/olake, 130+ and growing fast), an open-source tool built to help you replicate Database (MongoDB, for now, mysql and postgres under development) data into Data Lakehouse at faster speed without any hassle of managing Debezium or kafka (at least 10x faster than Airbyte and Fivetran at fraction of the cost, refer docs for benchmarks - https://olake.io/docs/connectors/mongodb/benchmarks).
Open Source, Data Replication, Database, Data Lakehouse, Benchmarking
14 points by pkhodiyar 74 days ago | 3 comments
SQL pipe syntax available in public preview in BigQuery (cloud.google.com)
Pipe query syntax is an extension to GoogleSQL that supports a linear query structure designed to make your queries easier to read, write, and maintain. You can use pipe syntax anywhere you write GoogleSQL.
SQL, BigQuery, Cloud Computing, Programming Languages, Database
247 points by marcyb5st 75 days ago | 113 comments
A New Postgres Block Storage Layout for Full Text Search (paradedb.com)
Database, PostgreSQL
13 points by craigkerstiens 83 days ago | 1 comments
One Billion JSON Challenge (clickhouse.com)
We took on Gunnar Morling’s One Billion Row Challenge almost exactly a year ago, testing how quickly a 1-billion-row text file could be aggregated.
Database, Performance, Data Analysis
15 points by pradeepchhetri 86 days ago | 0 comments
Adding concurrent read/write to DuckDB with Arrow Flight (definite.app)
We've been thinking a lot about latency, streaming and (near) real-time analytics lately. At Definite, we deal with a lot of data pipelines. In most cases (e.g. ingesting Stripe data), our customers are fine with batch processing (e.g. every hour). But as we've grown, we've seen more and more need for near real-time pipelines (e.g. ingesting events or CDC from Postgres).
Database, Data Pipelines, Real-Time Analytics, Streaming Data, Arrow Flight
94 points by mritchie712 87 days ago | 32 comments
Composable SQL (borretti.me)
SQL could be improved somewhat by introducing composable query fragments with statically-typed interfaces. I begin by explaining two areas (testing and reusing business logic) where SQL does very poorly. Then I explain my solution, and how it addresses the problems.
SQL, Programming, Database, Testing, Business Logic
271 points by earnestinger 90 days ago | 143 comments
An experiment of adding recommendation engine to your app using pgvector search (silk.us)
The reason I am excited about the latest genAI and vector search developments is that you can “plug in” these new technologies into your existing applications and their data, without having to re-platform or re-engineer your entire stack first. You can keep your existing application code and database design exactly as it is and then just add new genAI & vector search features where it makes sense.
Generative AI, Vector Search, Application Development, Database, Recommendation Engines
76 points by tanelpoder 93 days ago | 7 comments
The Mythical IO-Bound Rails App (byroot.github.io)
When the topic of Rails performance comes up, it is commonplace to hear that the database is the bottleneck, so Rails applications are IO-bound anyway, hence Ruby performance doesn’t matter that much, and all you need is a healthy dose of concurrency to make your service scale.
Rails, Performance Optimization, Database, Ruby, Concurrency
8 points by todsacerdoti 93 days ago | 2 comments
The missing tier for query compilers (scattered-thoughts.net)
Database query engines used to be able to assume that disk latency was so high that the overhead of interpreting the query plan didn't matter. Unfortunately these days a cheap nvme ssd can supply data much faster than a query interpreter can process it.
Database, Query Optimizers, Performance, Computer Architecture
5 points by eatonphil 102 days ago | 1 comments
Database Release and End-to-End Testing: ClickHouse Database Cloning (notion.site)
Database, Testing, Software, Cloud Computing
25 points by LittleCat38 125 days ago | 2 comments
How bloom filters made SQLite 10x faster (avi.im)
This is the fascinating story of how researchers used Bloom filters cleverly to make SQLite 10x faster for analytical queries.
Database, Performance, Optimization, SQLite, Bloom Filters
504 points by avinassh 125 days ago | 121 comments
Reads Causing Writes in Postgres (jesipow.com)
It is good practice to regularly inspect the statements running in the hot path of your Postgres instance. One way to do this is to examine the pg_stat_statements view, which shows various statistics about the SQL statements executed by the Postgres server.
Database, Postgres, Performance Optimization, Monitoring, SQL
224 points by thunderbong 125 days ago | 28 comments
ScyllaDB – Why We're Moving to a Source Available License (scylladb.com)
ScyllaDB has decided to focus on a single release stream – ScyllaDB Enterprise. Starting with the ScyllaDB Enterprise 2025.1 release (ETA February 2025):
Open Source, Software, Business, Licensing, Database
82 points by campers 128 days ago | 30 comments
VectorChord: Store 400k Vectors for $1 in PostgreSQL (pgvecto.rs)
We’re pleased to announce our new vector search extension for PostgreSQL, providing a highly cost-effective way to manage large vectors. Using VectorChord, you can achieve a QPS of 131 with 0.95 precision on 100 million 768-dimensional vectors for the top 10 queries. This setup costs only $250 monthly and can be hosted on a single machine.
Database, Vector Search, PostgreSQL, Open Source, Cost-Effective
156 points by gaocegege 142 days ago | 42 comments
Supabase AI Assistant v2 (supabase.com)
Today we are releasing Supabase Assistant v2 in the Dashboard - a global assistant with several new abilities:
AI, New Releases, Software, Database
11 points by samuba 143 days ago | 0 comments
Introducing integrated inference: Embed, rerank, and retrieve data with one API (pinecone.io)
We’re excited to announce expanded inference capabilities alongside our core vector database to make it even easier and faster to build high-quality, knowledgeable AI applications with Pinecone.
Artificial Intelligence, Database, APIs, Vector Databases
1 points by jimminyx 144 days ago | 0 comments
Show HN: WeSQL – An Innovative MySQL That Stores All Data on S3 (github.com/wesql)
WeSQL is an innovative MySQL distribution that adopts a compute-storage separation architecture, with storage backed by S3 (and S3-compatible systems). It can run on any cloud, ensuring no vendor lock-in.
Database, Cloud Computing, Open Source, Storage, MySQL
14 points by west0n 151 days ago | 5 comments