Hacker News with Generative AI: SQLite

Optimizing a large SQLite database for reading (jacobfilipp.com)
I recently needed to speed up a simple read query on a large SQLite file (620Mb), for my grocery price CSV export tool.

Databases, SQLite, Performance Optimization, Data Analysis

5 points by hinkley 4 days ago | 1 comments

Virtual Tables in SQLite with Go (misfra.me)
SQLite’s virtual tables let you expose external data sources as if they were normal tables. The go-sqlite3 package includes an example of how to register a custom module, but it doesn’t cover cases where you need to initialize the module differently per connection.

Databases, Programming, Go, SQLite

4 points by preetamjinka 17 days ago | 0 comments

Build your own SQLite in Rust, Part 5: Evaluating queries (sylver.dev)
In the previous posts, we've explored the SQLite file format and built a simple SQL parser. It's time to put these pieces together and implement a query evaluator!

Rust, Databases, Programming, SQLite, Query Optimization

216 points by todsacerdoti 17 days ago | 27 comments

Searchcode.com’s SQLite database is probably 6 terabytes bigger than yours (boyter.org)
searchcode.com’s SQLite database is probably one of the largest in the world, at least for a public facing website. It’s actual size is 6.4 TB. Which is probably 6 terabytes bigger than yours.

Databases, SQLite, Search Engines, Technology

242 points by polyrand 20 days ago | 95 comments

Show HN: SQLite disk page explorer (github.com/QuadrupleA)
A small GUI application built in redbean that lets you explore your SQLite databases "page by page" the way SQLite sees them.

Software, Databases, SQLite, GUI

288 points by QuadrupleA 30 days ago | 41 comments

Build your own SQLite, Part 4: reading tables metadata (sylver.dev)
As we saw in the opening post, SQLite stores metadata about tables in a special "schema table" starting on page 1. We've been reading records from this table to list the tables in the current database, but before we can start evaluating SQL queries against user-defined tables, we need to extract more information from the schema table.

Databases, SQLite, Programming, Tutorials

135 points by thunderbong 33 days ago | 11 comments

GarminDB (github.com/tcgoetz)
Python scripts for parsing health data into and manipulating data in a SQLite database. SQLite is a light weight database that doesn't require a server.

Python, Databases, Health Data, SQLite

326 points by haltcatchfire 34 days ago | 72 comments

SQLook – A free online SQLite database manager with a Windows 2000 interface (sqlook.com)
SQLook is a modern web-based SQLite database manager with a nostalgic Windows 2000 interface.

Databases, Web Development, User Interfaces, Nostalgic Design, SQLite

178 points by gringow 42 days ago | 72 comments

Supercharge SQLite with Ruby Functions (julik.nl)
An interesting twist in my recent usage of SQLite was the fact that I noticed my research scripts and the database intertwine more. SQLite is unique in that it really lives in-process, unlike standalone database servers. There is a feature to that which does not get used very frequently, but can be indispensable in some situations.

Ruby, SQLite, Databases, Programming, Performance

159 points by julik 44 days ago | 43 comments

We will rewrite SQLite. And we are going all-in (turso.tech)
Two years ago, we made a bet: SQLite, the most successful database in history, could be even better.

Databases, SQLite, Open Source

17 points by mpereira 46 days ago | 3 comments

Show HN: Postgres-SQLite: SQLite as first-class types in Postgres (github.com/michelp)
postgres-sqlite is a PostgreSQL extension that introduces SQLite databases as a first-class data type within PostgreSQL.

Databases, PostgreSQL, SQLite, Extensions

15 points by michelpp 63 days ago | 2 comments

Speeding Up SQLite Inserts (julik.nl)
In my work I tend to reach for SQLite more and more. The type of work I find it useful for most these days is quickly amalgamating, dissecting, collecting and analyzing large data sets. As I have outlined in my Euruko talk on scheduling, a key element of the project was writing a simulator. That simulator outputs metrics - lots and lots of metrics, which resemble what our APM solution collects.

Databases, Performance Optimization, SQLite

14 points by julik 65 days ago | 3 comments

Fun facts about SQLite (avi.im)
SQLite is the most deployed and most used database. There are over one trillion (1000000000000 or a million million) SQLite databases in active use.

Databases, SQLite, Fun Facts

393 points by avinassh 68 days ago | 191 comments

How bloom filters made SQLite 10x faster (avi.im)
This is the fascinating story of how researchers used Bloom filters cleverly to make SQLite 10x faster for analytical queries.

Database, Performance, Optimization, SQLite, Bloom Filters

504 points by avinassh 76 days ago | 121 comments

Show HN: NoSQL, but it's SQLite (github.com)
SQLite-backed key-value store with JS-like object manipulation and automatic JSON serialization.

Databases, NoSQL, JavaScript, SQLite

98 points by vsroy 77 days ago | 39 comments

Cr-SQLite – Multi-writer and CRDT support for SQLite (github.com/vlcn-io)
CR-SQLite is a run-time loadable extension for SQLite and libSQL. It allows merging different SQLite databases together that have taken independent writes.

Databases, SQLite, Software, Open Source

30 points by apitman 82 days ago | 2 comments

Limbo: A complete rewrite of SQLite in Rust (turso.tech)
Today we are announcing a more ambitious experiment: what could we achieve, if we were to completely rewrite SQLite in a memory-safe language (Rust)? With the Limbo project, now available at github.com/tursodatabase/limbo, we are now trying to answer that question.

Rust, Databases, SQLite, Programming Languages, Open Source

373 points by avinassh 88 days ago | 232 comments

Sensible SQLite Defaults (briandouglas.ie)
SQLite is cool now. DHH uses it, Laravel defaults to it. Here is a list of sensible defaults when using sqlite.

Databases, SQLite, Software Development

223 points by thunderbong 93 days ago | 67 comments

SQLite Index Visualization: Search (mrsuh.com)
In the previous post, I explained how to extract data from SQLite Indexes and make it visual. This time, I'll try to show what a search inside an Index looks like.

SQLite, Databases, Visualization, Search

15 points by mrsuh 97 days ago | 0 comments

Big Endian's Guide to SQLite Storage (jabid.in)
I wanted to learn how databases like SQLite store data under the hood, so I decided to write some code to inspect the database file. SQLite famously stores the entire database in a single file, and the file format is very well documented. Here is one diagram1 to get started instead of the roughly 13,848 words in that document.

Databases, SQLite, Programming, Data Storage, Software

38 points by marcobambini 97 days ago | 0 comments

SQLite-utils: a tool for manipulating SQLite databases (datasette.io)
CLI tool and Python library for manipulating SQLite databases

Databases, SQLite, Tools, Python

9 points by mpbart 104 days ago | 0 comments

SQLiteStudio: Create, edit, browse SQLite databases (sqlitestudio.pl)
It's a hotfix release to address urgent problem of "black SQL code line" that appeared in 3.4.5. It also gets two more issues resolved.

SQLite, Databases, Software, Bug Fixes

626 points by thunderbong 104 days ago | 123 comments

Sqlpkg – The SQLite Extension Hub (sqlpkg.org)
Find SQLite extensions using the search box above. You can download and install them manually, or use the sqlpkg package manager.

SQLite, Database, Tools, Package Management

5 points by thunderbong 105 days ago | 0 comments

SQLite’s Use of Tcl (2017) (tcl.tk)
SQLite is a TCL extension that has escaped into the wild.

Databases, Programming Languages, TCL, SQLite

138 points by fanf2 106 days ago | 44 comments

Show HN: Embed an SQLite database in your PostgreSQL table (github.com/frectonz)
Embed an SQLite database in your PostgreSQL table. AKA multitenancy has been solved.

Databases, PostgreSQL, SQLite, Multitenancy, Software

180 points by frectonz 110 days ago | 110 comments

SQLite vs. PostgreSQL Performance [video] (youtube.com)

Databases, Performance, Video, PostgreSQL, SQLite

28 points by thunderbong 110 days ago | 7 comments

SQLite Index Visualization (mrsuh.com)
After learning about indexes, I understood their basic structure, but I wanted to dig deeper — to explore the data structure, understand the algorithm, and learn how the index data is stored on disk.

Databases, Data Structures, Optimization, SQLite

295 points by mrsuh 115 days ago | 26 comments

SQLite is not a single connection database (igorstechnoclub.com)
I think the most common misconception about SQLite which drives people away from that database is that it's a single connection database.

Databases, SQLite, Misconceptions

77 points by Igor_Wiwi 119 days ago | 38 comments

SQLite does not do checksums (avi.im)
SQLite does not do checksums by default. I learned this from Alex Miller. What does this mean? If there is disk corruption, the database or application won’t be able to know that the database is ‘corrupt’.

Databases, SQLite, Data Integrity

132 points by todsacerdoti 119 days ago | 98 comments

SQLite Rsync: Database Remote-Copy Tool for SQLite (sqlite.org)
The following command causes REPLICA to become a copy of ORIGIN:

Databases, Tools, SQLite, Remote Syncing

75 points by Peter5 132 days ago | 11 comments