Lossless LLM compression for efficient GPU inference via dynamic-length float
(arxiv.org)
Large Language Models (LLMs) have grown rapidly in size, creating significant challenges for efficient deployment on resource-constrained hardware.
Large Language Models (LLMs) have grown rapidly in size, creating significant challenges for efficient deployment on resource-constrained hardware.
I used simple rules to make DFAs that kinda match accepted physics models
(keweizhou1996-df477.web.app)
I USED SIMPLE RULES TO MAKE DETERMINISTIC FINITE AUTOMATAS THAT RESONATE WITH OUR (AS OF 2025) MOST ACCEPTED MODELS OF PHYSICS
I USED SIMPLE RULES TO MAKE DETERMINISTIC FINITE AUTOMATAS THAT RESONATE WITH OUR (AS OF 2025) MOST ACCEPTED MODELS OF PHYSICS
Shortest-possible walking tour to 81,998 bars in South Korea
(uwaterloo.ca)
We have solved a traveling salesman problem (TSP) to walk to 81,998 bars in South Korea.
We have solved a traveling salesman problem (TSP) to walk to 81,998 bars in South Korea.
A Computational Proof of the Highest-Scoring Boggle Board
(danvk.org)
Exciting news! This is the best possible Boggle board:
Exciting news! This is the best possible Boggle board:
Algebraic Semantics for Machine Knitting
(uwplse.org)
As programming languages researchers, we’re entitled to a certain level of mathematical rigor behind the languages we write and analyze.
As programming languages researchers, we’re entitled to a certain level of mathematical rigor behind the languages we write and analyze.
Data Compression Nerds Hate This One Trick [video]
(media.ccc.de)
How one guy in his bedroom (kind of) beat all of PNG's combined multi-decade effort in one year, and why that's strange.
How one guy in his bedroom (kind of) beat all of PNG's combined multi-decade effort in one year, and why that's strange.
Flat origami is Turing complete (2023)
(arxiv.org)
"Flat origami" refers to the folding of flat, zero-curvature paper such that the finished object lies in a plane.
"Flat origami" refers to the folding of flat, zero-curvature paper such that the finished object lies in a plane.
Pydrofoil: Accelerating Sail-based instruction set simulators
(arxiv.org)
We present Pydrofoil, a multi-stage compiler that generates instruction set simulators (ISSs) from processor instruction set architectures (ISAs) expressed in the high-level, verification-oriented ISA specification language Sail.
We present Pydrofoil, a multi-stage compiler that generates instruction set simulators (ISSs) from processor instruction set architectures (ISAs) expressed in the high-level, verification-oriented ISA specification language Sail.
Regex Isn't Hard (2023)
(timkellogg.me)
Regex gets a bad reputation for being very complex. That’s fair, but I also think that if you focus on a certain core subset of regex, it’s not that hard. Most of the complexity comes from various “shortcuts” that are hard to remember. If you ignore those, the language itself is fairly small and portable across programming languages.
Regex gets a bad reputation for being very complex. That’s fair, but I also think that if you focus on a certain core subset of regex, it’s not that hard. Most of the complexity comes from various “shortcuts” that are hard to remember. If you ignore those, the language itself is fairly small and portable across programming languages.
Sparsely-Gated Mixture of Experts (MoE)
(thegreenplace.net)
In transformer models, the attention block is typically followed by a feed forward layer (FF), which is a simple fully-connected NN with a hidden layer and nonlinearity.
In transformer models, the attention block is typically followed by a feed forward layer (FF), which is a simple fully-connected NN with a hidden layer and nonlinearity.
A curated blog for learning LLM internals: tokenize, attention, PE, and more
(ycombinator.com)
I've been diving deep into the internals of Large Language Models (LLMs) and started documenting my findings.
I've been diving deep into the internals of Large Language Models (LLMs) and started documenting my findings.
The Art of Assembly Language (2010)
(plantation-productions.com)
"The Art of Assembly Language Programming" is now hard. This text is now available in published form from "No Starch Press" (http://www.nostarch.com). Please check out their website for more details.
"The Art of Assembly Language Programming" is now hard. This text is now available in published form from "No Starch Press" (http://www.nostarch.com). Please check out their website for more details.
Efficient E-Matching for Super Optimizers
(vortan.dev)
Modern theorem provers and optimizing compilers are built on an interesting concept: the ability to recognize when two things are equal, even if they look completely different.
Modern theorem provers and optimizing compilers are built on an interesting concept: the ability to recognize when two things are equal, even if they look completely different.
Microsoft’s “1‑bit” AI model runs on a CPU only, while matching larger systems
(arstechnica.com)
Future AI might not need supercomputers thanks to models like BitNet b1.58 2B4T.
Future AI might not need supercomputers thanks to models like BitNet b1.58 2B4T.
From GnuGo to AlphaGo Zero – A Roadmap for Solving Difficult Problems
(moderndescartes.com)
What do difficult problems like digital assistants, self-driving cars, theorem-proving systems, and compilers have in common? They all have a certain level of fuzziness in their problem specification and a large solution space that makes it incredibly difficult to find optimal solutions.
What do difficult problems like digital assistants, self-driving cars, theorem-proving systems, and compilers have in common? They all have a certain level of fuzziness in their problem specification and a large solution space that makes it incredibly difficult to find optimal solutions.
C++ creator Bjarne Stroustrup reimagines how coding is taught
(stanforddaily.com)
At a Computer Science Education Seminar talk Thursday, C++ creator Bjarne Stroustrup shared his thoughts on the role of C++ and what needs fixing in programming education.
At a Computer Science Education Seminar talk Thursday, C++ creator Bjarne Stroustrup shared his thoughts on the role of C++ and what needs fixing in programming education.
The Halting Problem is a terrible example of NP-Harder
(buttondown.com)
In computation complexity, NP is the class of all decision problems (yes/no) where a potential proof (or "witness") for "yes" can be verified in polynomial time. For example, "does this set of numbers have a subset that sums to zero" is in NP. If the answer is "yes", you can prove it by presenting a set of numbers.
In computation complexity, NP is the class of all decision problems (yes/no) where a potential proof (or "witness") for "yes" can be verified in polynomial time. For example, "does this set of numbers have a subset that sums to zero" is in NP. If the answer is "yes", you can prove it by presenting a set of numbers.
Quare FreeBSD?
(wordpress.com)
I really wanted to make this article short … but I failed miserably. At least I tried to organize it well so one may get back to it after ‘some’ reading because its not a short lecture. I wanted to title it Why FreeBSD? but when you type that into your favorite duck.com search engine there are so many similar articles. I wanted it to have distinguished and unique name so I used Latin word for ‘why‘ which is ‘quare‘.
I really wanted to make this article short … but I failed miserably. At least I tried to organize it well so one may get back to it after ‘some’ reading because its not a short lecture. I wanted to title it Why FreeBSD? but when you type that into your favorite duck.com search engine there are so many similar articles. I wanted it to have distinguished and unique name so I used Latin word for ‘why‘ which is ‘quare‘.
How to Optimize Rust for Slowness: Inspired by New Turing Machine Results
(medium.com)
Everyone talks about making Rust programs faster [1, 2, 3], but what if we pursue the opposite goal? Let’s explore how to make them slower — absurdly slower. Along the way, we’ll examine the nature of computation, the role of memory, and the scale of unimaginably large numbers.
Everyone talks about making Rust programs faster [1, 2, 3], but what if we pursue the opposite goal? Let’s explore how to make them slower — absurdly slower. Along the way, we’ll examine the nature of computation, the role of memory, and the scale of unimaginably large numbers.
Lifetimes of Cryptographic Hash Functions
(valerieaurora.org)
I've written some cautionary articles on using cryptographic hashes to create content-based addresses (compare-by-hash). This page brings together everything I've written and keeps an updated table of the status of popular cryptographic hash functions.
I've written some cautionary articles on using cryptographic hashes to create content-based addresses (compare-by-hash). This page brings together everything I've written and keeps an updated table of the status of popular cryptographic hash functions.
Teaching keeps you young, or so says Professor Brian Kernighan
(dailyprincetonian.com)
After attending Princeton as a graduate student in the Department of Electrical Engineering in 1969, Brian Kernighan returned to the University in 2000 as a professor in the computer science department after spending time at Bell Labs, a renowned computer science research center.
After attending Princeton as a graduate student in the Department of Electrical Engineering in 1969, Brian Kernighan returned to the University in 2000 as a professor in the computer science department after spending time at Bell Labs, a renowned computer science research center.