Hacker News with Generative AI: Databases

Building Databases over a Weekend (denormalized.io)
In this post we take you on a walkthrough on how you can use DataFusion to build your own developer experience for databases.

Databases, Developer Tools, DataFusion

67 points by ambrood 6 hours ago | 8 comments

SQL, Homomorphisms and Constraint Satisfaction Problems (philipzucker.com)
Database queries are a pretty surprisingly powerful tool that can solve seemingly intractable problems.

Databases, Programming, Computer Science

126 points by xlinux 14 hours ago | 12 comments

Loading the World OpenStreetMap Import in Under 4 Hours (crunchydata.com)
The OpenStreetMap (OSM) database builds almost 750GB of location data from a single file download. OSM notoriously takes a full day to run.

OpenStreetMap, Databases, Data Import, Performance Optimization

16 points by todsacerdoti 22 hours ago | 0 comments

Open Riak – open, modern Riak fork (github.com/OpenRiak)
Promoting an open, modern Riak

Open Source, Databases, Software, Riak

216 points by amarsahinovic 1 day ago | 70 comments

Show HN: Fireproof – local-first database with Git-like encrypted sync (fireproof.storage)
Quickly add live data to any app or page.

Databases, Software, Open Source, Collaboration, Encryption

90 points by jchanimal 2 days ago | 41 comments

Show HN: Embed an SQLite database in your PostgreSQL table (github.com/frectonz)
Embed an SQLite database in your PostgreSQL table. AKA multitenancy has been solved.

Databases, PostgreSQL, SQLite, Multitenancy, Software

172 points by frectonz 2 days ago | 108 comments

SQLite vs. PostgreSQL Performance [video] (youtube.com)

Databases, Performance, Video, PostgreSQL, SQLite

28 points by thunderbong 2 days ago | 7 comments

The Fastest Redis Ever (redis.io)
We’re happy to announce the second milestone of Redis 8, our most advanced and performant offering yet, available for you to try in Community Edition (CE) today.

Redis, Performance, Databases, Open Source

14 points by lsferreira42 2 days ago | 3 comments

Usql: Universal command-line interface for SQL databases (github.com/xo)
Universal command-line interface for SQL databases

SQL, Databases, Tools

38 points by thunderbong 4 days ago | 2 comments

Representing filesystems in databases efficiently with Hierarchical Ordering (danthegoodman.substack.com)
The way the we interface with files in a file system and rows in a database are fundamentally not the same.

Databases, Data Structures, File Systems, Optimization

9 points by dangoodmanUT 5 days ago | 0 comments

Optimizers: The Low-Key MVP (duckdb.org)
TL;DR: The query optimizer is an important part of any analytical database system as it provides considerable performance improvements compared to hand-optimized queries, even as the state of your data changes.

Databases, Optimization, Performance

119 points by tosh 5 days ago | 13 comments

Building Observability with ClickHouse (cmtops.dev)
This is a summary of my experience in searching for and implementing an optimal observability stack at my workplace.

Observability, ClickHouse, Databases, DevOps

58 points by valyala 6 days ago | 20 comments

SQLite Index Visualization (mrsuh.com)
After learning about indexes, I understood their basic structure, but I wanted to dig deeper — to explore the data structure, understand the algorithm, and learn how the index data is stored on disk.

Databases, Data Structures, Optimization, SQLite

294 points by mrsuh 7 days ago | 25 comments

Don't Do This with PostgreSQL (postgresql.org)
Don't use trust authentication over any TCP/IP method (e.g. host, hostssl) in any production environment.

PostgreSQL, Security, Databases

64 points by sh_tomer 9 days ago | 15 comments

What I wish someone told me about Postgres (challahscript.com)
I’ve been working professionally for the better part of a decade on web apps and, in that time, I’ve had to learn how to use a lot of different systems and tools.

Databases, Postgres, Web Development

483 points by todsacerdoti 9 days ago | 190 comments

The CVM Algorithm (buttondown.com)
Everything you need to know about query planning can be understood from this query:

Algorithms, Databases, Query Optimization

13 points by todsacerdoti 9 days ago | 1 comments

Evolving a NoSQL Database Schema (karmanivero.us)
In a NoSQL environment, Entity Manager organizes the physical distribution of data to support efficient query operations.

NoSQL, Databases, Data Management, Optimization

83 points by smitty1e 10 days ago | 20 comments

PostgreSQL Conference Europe 2024 Talks (youtube.com)

PostgreSQL, Conferences, Europe, Databases

3 points by davidgomes 11 days ago | 0 comments

SQLite is not a single connection database (igorstechnoclub.com)
I think the most common misconception about SQLite which drives people away from that database is that it's a single connection database.

Databases, SQLite, Misconceptions

77 points by Igor_Wiwi 12 days ago | 38 comments

SQLite does not do checksums (avi.im)
SQLite does not do checksums by default. I learned this from Alex Miller. What does this mean? If there is disk corruption, the database or application won’t be able to know that the database is ‘corrupt’.

Databases, SQLite, Data Integrity

132 points by todsacerdoti 12 days ago | 98 comments

JIT in ClickHouse (2022) (clickhouse.com)
In this post, I will describe what JIT compilation is, how LLVM infrastructure can be used for JIT compilation, and how JIT compilation works in ClickHouse.

Databases, JIT Compilation, ClickHouse

12 points by thunderbong 13 days ago | 0 comments

Userland Disk I/O: Filesystems, file IO, and durability (transactional.blog)
In database land, most databases open(2) their WAL and data files with O_DIRECT so that write(2)/writev(2)/pwritev(2) perform unbuffered IO, maintain their own page cache, and utilize fdatasync() for durability.

Operating Systems, File Systems, Databases

30 points by thunderbong 13 days ago | 2 comments

Show HN: BemiDB – Postgres read replica optimized for analytics (github.com/BemiHQ)
BemiDB is a Postgres read replica optimized for analytics, It consists of a single binary that seamlessly connects to a Postgres database, replicates the data in a compressed columnar format, and allows you to run complex queries using its Postgres-compatible analytical query engine.

Databases, Analytics, Postgres, Open Source, Software

209 points by exAspArk 14 days ago | 118 comments

Optimize Database Performance in Ruby on Rails and ActiveRecord (appsignal.com)
In Rails, we're more likely to use SQL databases than other frameworks.

Databases, Performance Optimization, ActiveRecord

83 points by amalinovic 15 days ago | 58 comments

GoToSocial WASM-based SQLite driver and BSD (tumfatig.net)
I started using GoToSocial (the fast, fun and small ActivityPub server) in 2022 on OpenBSD. Because it was nearly the only OpenBSD-native ActivityPub options at that time, because it was light and because it could use the SQLite database engine.

Web Development, Databases, Open Source, Social Media

67 points by jaypatelani 15 days ago | 1 comments

Build Your Own Database from Scratch in Go (build-your-own.org)
Understand databases from the bottom up by building your own, in small steps, and with simple Golang code.

Databases, Go, Programming, Tutorials

7 points by saintfiends 16 days ago | 0 comments

DuckDB-GSheets: Read and Write Google Sheets in Pure SQL (github.com/evidence-dev)

Databases, Google Sheets, SQL

11 points by archiewood 17 days ago | 1 comments

Do you need Redis? PostgreSQL does queuing, locking, and pub/sub (2021) (atomicobject.com)

Databases, PostgreSQL, Redis, Queuing, Locking

197 points by fanf2 17 days ago | 129 comments

pg_flo – Stream, transform, and re-route PostgreSQL data in real-time (pgflo.io)
The easiest way to move and transform data between PostgreSQL databases

PostgreSQL, Data Processing, Real-time Data, Databases, Software

239 points by shayonj 18 days ago | 54 comments

Leveraging the lightcone around the source of truth with Postgres (benoitessiambre.com)
To summarize, the postgres-centrism hypothesis says that integrated, entropy-minimizing systems that reduce dependency distances, and anchor logic near the source of truth, leveraging the efficiencies of proximity, often better aligns with fundamentals of computing, physics, information theory, thermodynamics and intelligence.

Databases, Software Architecture, Information Theory

9 points by BenoitEssiambre 18 days ago | 5 comments