Hacker News with Generative AI: Memory Management

Loading Pydantic models from JSON without running out of memory (pythonspeed.com)
You have a large JSON file, and you want to load the data into Pydantic. Unfortunately, this uses a lot of memory, to the point where large JSON files are very difficult to read. What to do?
Python, Data Processing, Memory Management, Optimization, Pydantic
120 points by itamarst 1 day ago | 43 comments
Memory Consistency Models: A Tutorial (jamesbornholt.com)
There are, of course, only two hard things in computer science: cache invalidation, naming things, and off-by-one errors. But there is another hard problem lurking amongst the tall weeds of computer science: seeing things in order. Whether it be sorting, un-sorting, or tweeting, seeing things in order is a challenge for the ages.
Computer Science, Programming, Memory Management
57 points by tanelpoder 4 days ago | 5 comments
Writing into Uninitialized Buffers in Rust (sunfishcode.online)
Uninitialized buffers in Rust are a long-standing question, for example:
Rust, Programming, Security, Memory Management
126 points by luu 4 days ago | 118 comments
A Retrospective on Region-Based Memory Management (2004) [pdf] (elsman.com)
Computer Science, Memory Management, Programming Languages, Research Papers, Technical Papers
6 points by earthicus 5 days ago | 0 comments
Rust Devs Think We're Hopeless; Let's Prove Them Wrong (With C++ Memory Leaks) (babaei.net)
Rust Devs Think We’re Hopeless; Let’s Prove Them Wrong (with C++ Memory Leaks)!
Programming Languages, Rust, C++, Memory Management, Software Development
27 points by zdw 12 days ago | 27 comments
Hazard Pointers in C++26 (modernescpp.com)
Hazard pointers provide garbage collection in C++ and solve the ABA problem.
C++, Programming Languages, Garbage Collection, Memory Management
12 points by ibobev 17 days ago | 3 comments
Beyond the 1 MB barrier in DOS (blogsystem5.substack.com)
In “From 0 to 1 MB in DOS”, I presented an overview of all the ways in which DOS and its applications tried to maximize the use of the 1 MB address space inherited from the 8086—even after the 80286 introduced support for 16 MB of memory and the 80386 opened the gates to 4 GB.
Operating Systems, DOS, Memory Management
14 points by gone35 18 days ago | 3 comments
Understanding Memory Management, Part 5: Fighting with Rust (educatedguesswork.org)
This is the fifth post in my planned multipart series on memory management. You will probably want to go back and read Part I, which covers C, parts II and III, which cover C++, and part IV, which introduces Rust memory management. In part IV, we got through the basics of Rust memory management up through smart pointers. In this post I want to look at some of the gymnastics you need to engage in to do serious work in Rust.
Programming, Rust, Memory Management
147 points by Curiositry 20 days ago | 103 comments
Memory leak proof every C program (tedunangst.com)
Memory leaks have plagued C programs for as long as the language has existed.
Programming, C Programming, Memory Management
8 points by iio7 23 days ago | 3 comments
The Linux Memory Manager (nostarch.com)
The Linux memory management subsystem hasn’t had a definitive reference since 2004.
Linux, Operating Systems, Memory Management
18 points by sohkamyung 24 days ago | 2 comments
Nofl: A Precise Immix (arxiv.org)
Can a memory manager be built with fast bump-pointer allocation, single-pass heap tracing, and a low upper bound on memory overhead?
Memory Management, Programming Languages, Computer Science, Algorithms
8 points by todsacerdoti 28 days ago | 0 comments
How ZGC allocates memory for the Java heap (joelsiks.com)
This post explores how ZGC, one of the garbage collectors in the OpenJDK, allocates memory for the Java heap, focusing on enhancements introduced in JDK-8350441 with the Mapped Cache. A garbage collector does much more than just collect garbage - and that’s what I want to unpack in this post. Whether you’re a Java nerd yearning for details, a GC enthusiast, or just curious about how ZGC uses memory behind the scenes, this deep dive is for you.
Java, Garbage Collection, OpenJDK, Memory Management, Performance
86 points by lichtenberger 30 days ago | 18 comments
Understanding Memory Management, Part 5: Fighting with Rust (educatedguesswork.org)
This is the fifth post in my planned multipart series on memory management. You will probably want to go back and read Part I, which covers C, parts II and III, which cover C++, and part IV, which introduces Rust memory management. In part IV, we got through the basics of Rust memory management up through smart pointers. In this post I want to look at some of the gymnastics you need to engage in to do serious work in Rust.
Rust, Memory Management, Programming Languages
4 points by ibobev 30 days ago | 0 comments
Understanding Memory Management, Part 5: Fighting with Rust (educatedguesswork.org)
This is the fifth post in my planned multipart series on memory management. You will probably want to go back and read Part I, which covers C, parts II and III, which cover C++, and part IV, which introduces Rust memory management. In part IV, we got through the basics of Rust memory management up through smart pointers. In this post I want to look at some of the gymnastics you need to engage in to do serious work in Rust.
Programming, Rust, Memory Management, Software Development
6 points by ekr____ 33 days ago | 0 comments
WebAssembly: How to Allocate Your Allocator (nullprogram.com)
An early, small hurdle diving into WebAssembly was allocating my allocator. On a server or desktop with virtual memory, the allocator asks the operating system to map fresh pages into its address space (sbrk, anonymous mmap, VirtualAlloc), which it then dynamically allocates to different purposes. In an embedded context, dynamic allocation memory is typically a fixed, static region chosen at link time.
WebAssembly, Programming, Memory Management
45 points by ingve 34 days ago | 3 comments
Pahole: Analysing Memory Layout of Complex Data Structures with Ease (pramodkumbhar.com)
Sunday, 5th November 2023: Putting together this blog post feels like a positive stride! As I mentioned in the previous post, (core-to-core latency tool), I'm aiming to integrate more consistent writing into my routine. While it took a month to pen this down, it's progress from the previous year 😇. Hoping that the upward trend will continue...!
Memory Management, Software Development, Programming, Data Structures, Linux
19 points by todsacerdoti 35 days ago | 0 comments
Avoiding memory fragmentation in Rust with jemalloc (kerkour.com)
Rust, Memory Management, Programming Languages
7 points by redcannon218 37 days ago | 0 comments
Battle of the Mallocators (blogspot.com)
If you use RocksDB and want to avoid OOM then use jemalloc or tcmalloc and avoid glibc malloc. That was true in 2015 and remains true in 2025 (see here).
Memory Management, Performance Optimization, Software Development, C/C++
27 points by qianli_cs 41 days ago | 11 comments
Mistakes and cool things to do with arena allocators (zylinski.se)
When programming in Odin you can use arena allocators. If you use an arena allocator combined with a dynamic array, then there are a couple of pitfalls that may not be apparent at first. Let’s look at what arenas are, how you can run into trouble when naively using them with dynamic arrays and what you can do instead.
Programming, Odin, Memory Management
78 points by skeptrune 43 days ago | 8 comments
Golang sync.Pool is not a silver bullet (wundergraph.com)
When it comes to performance optimization in Go, sync.Pool often appears as a tempting solution. It promises to reduce memory allocations and garbage collection pressure by reusing objects. But is it always the right choice? Let's dive deep into this fascinating topic.
Go, Performance Optimization, Programming, Memory Management
4 points by jensneuse 51 days ago | 0 comments
Implementing a spellchecker on 64 kB of RAM in 70s led to compression algorithm (pcgamer.com)
Algorithms, Computer Science, History, Programming, Memory Management
3 points by amichail 52 days ago | 0 comments
Understanding Memory Management, Part 4: Rust Ownership and Borrowing (educatedguesswork.org)
This is the fourth post in my planned multipart series on memory management. Part I covers the basics of memory allocation and how it works in C, and parts II and III covered the basics of C++ memory management, including RAII and smart pointers.
Programming, Memory Management, Rust, C++, C
5 points by ekr____ 53 days ago | 0 comments
Vramfs: Vram Based Filesystem for Linux (github.com/Overv)
Unused RAM is wasted RAM, so why not put some of that VRAM in your graphics card to work?
Linux, Operating Systems, Graphics Cards, Memory Management
130 points by signa11 55 days ago | 41 comments
Damon Self-Tuned Memory Tiering Shows Nice Improvement for Linux Servers (phoronix.com)
Linux developer SeongJae Park has posted a set of patches for the Linux kernel's wonderful DAMON code to provide for self-tuned memory tiering that "just works" and is racking up some nice performance wins.
Linux, Memory Management, Performance Optimization, Kernel Development
5 points by rbanffy 64 days ago | 0 comments
Stupid Smart Pointers in C (kevinalbs.com)
Managing memory in C is difficult and error prone. C++ solves this with smart pointers like std::unique_ptr and std::shared_ptr. This article demonstrates a proof-of-concept (aka stupid) smart pointer in C with very little code. Along the way we'll look at the layout of the 32-bit x86 call stack and write assembly in a C program.
C, Programming, Memory Management, Software Development, Pointers
146 points by seansh 67 days ago | 164 comments
Understanding Memory Management, Part 3: C++ Smart Pointers (educatedguesswork.org)
This is the third post in my planned multipart series on memory management. In part I we covered the basics of memory allocation and how it works in C, and in part II we covered the basics of C++ memory management, including the RAII idiom for memory management. In this post, we'll be looking at a powerful technique called "smart pointers" that lets you use RAII-style idioms but for pointers rather than objects.
C++, Memory Management, Programming, Smart Pointers
5 points by ekr____ 73 days ago | 0 comments
Understanding Memory Management, Part 2: C++ and RAII (educatedguesswork.org)
This is the second post in my planned multipart[1] series on memory management. In part I we covered the basics of memory allocation and how it works in C, where the programmer is responsible for manually allocating and freeing memory. In this post, we'll start looking at memory management in C++, which provides a number of much fancier affordances.
C++, Memory Management, Programming, Software Development
40 points by ibobev 75 days ago | 18 comments
The case of the vanishing CPU: A Linux kernel debugging story (clickhouse.com)
A mysterious CPU spike in ClickHouse Cloud on GCP led to months of debugging, revealing a deeper issue within the Linux kernel’s memory management.
Linux Kernel, Debugging, Cloud Computing, Memory Management
47 points by manish_gill 78 days ago | 5 comments
Zig: A good memory allocator in 200 lines of code (github.com/ziglang)
An allocator that is designed for ReleaseFast optimization mode, with multi-threading enabled.
Programming Languages, Optimization, Memory Management
14 points by signa11 78 days ago | 1 comments
A good memory allocator in 200 lines of code (github.com/ziglang)
An allocator that is designed for ReleaseFast optimization mode, with multi-threading enabled.
Programming, Memory Management, Optimization, Zig Language
8 points by todsacerdoti 79 days ago | 0 comments