Hacker News with Generative AI: Information Theory

How much information is in DNA? (dynomight.substack.com)
While answering how much information is in DNA may seem straightforward, it actually requires a wild odyssey through information theory and molecular biology.
Science, Biology, Information Theory
89 points by crescit_eundo 15 days ago | 65 comments
What Is Entropy? (jasonfantl.com)
People say many things about entropy: entropy increases with time, entropy is disorder, entropy increases with energy, entropy determines the arrow of time, etc.. But I have no idea what entropy is, and from what I find, neither do most other people. This is the introduction I wish I had when first told about entropy, so hopefully you find it helpful.
Science, Physics, Information Theory
288 points by jfantl 39 days ago | 113 comments
Information Converted to Energy (2010) (physicsworld.com)
Physicists in Japan have shown experimentally that a particle can be made to do work simply by receiving information, rather than energy.
Physics, Information Theory, Energy
6 points by teleforce 39 days ago | 0 comments
On Zero Sum Games (The Informational Meta-Game) (rohan.ga)
Zero-sum games are fundamentally informational contests.
Game Theory, Information Theory, Strategy
43 points by ocean_moist 91 days ago | 25 comments
The Maxwell-Heaviside Equations Explained by the Theory of Informatons (researchgate.net)
Physics, Mathematics, Information Theory
22 points by keepamovin 141 days ago | 7 comments
What Is Entropy? A Measure of Just How Little We Know (quantamagazine.org)
Exactly 200 years ago, a French engineer introduced an idea that would quantify the universe’s inexorable slide into decay. But entropy, as it’s currently understood, is less a fact about the world than a reflection of our growing ignorance. Embracing that truth is leading to a rethink of everything from rational decision-making to the limits of machines.
Physics, Information Theory, Science, Decision Making
6 points by rbanffy 161 days ago | 0 comments
Huffman Coding (wikipedia.org)
In computer science and information theory, a Huffman code is a particular type of optimal prefix code that is commonly used for lossless data compression.
Computer Science, Information Theory, Data Compression
5 points by tosh 177 days ago | 3 comments
An alternative construction of Shannon entropy (rkp.science)
TL;DR: Shannon’s entropy formula is usually justified by showing it satisfies key mathematical criteria, or by computing how much space is needed to encode a variable. But one can also construct Shannon’s formula starting purely from the simpler notion of entropy as a (logarithm of a) count—of how many different ways a distribution could have emerged from a sequence of samples.
Information Theory, Mathematics, Statistics, Probability
126 points by rkp8000 191 days ago | 11 comments
Leveraging the lightcone around the source of truth with Postgres (benoitessiambre.com)
To summarize, the postgres-centrism hypothesis says that integrated, entropy-minimizing systems that reduce dependency distances, and anchor logic near the source of truth, leveraging the efficiencies of proximity, often better aligns with fundamentals of computing, physics, information theory, thermodynamics and intelligence.
Databases, Software Architecture, Information Theory
9 points by BenoitEssiambre 201 days ago | 5 comments
Machine learning and information theory concepts towards an AI Mathematician (arxiv.org)
The current state-of-the-art in artificial intelligence is impressive, especially in terms of mastery of language, but not so much in terms of mathematical reasoning.
Artificial Intelligence, Machine Learning, Information Theory, Mathematics
109 points by marojejian 223 days ago | 19 comments
Introduction to Information Theory – Edward Witten [video] (youtube.com)
Information Theory, Mathematics, Video, Physics, Edward Witten
15 points by Anon84 278 days ago | 3 comments
What Is Entropy? (wordpress.com)
Physics, Science, Information Theory
345 points by ainoobler 305 days ago | 200 comments
What Is Entropy? (wordpress.com)
Science, Physics, Information Theory
6 points by mathgenius 307 days ago | 1 comments
Data Compression Explained (2011) (mattmahoney.net)
Data Compression, Computer Science, Information Theory
26 points by TheAlchemist 347 days ago | 1 comments
A Mathematical Theory of Communication [pdf] (math.harvard.edu)
Mathematics, Communication, Information Theory
298 points by luu 388 days ago | 52 comments