Hacker News with Generative AI: Neural Networks

µPC: Scaling Predictive Coding to 100 Layer Networks (arxiv.org)
The biological implausibility of backpropagation (BP) has motivated many alternative, brain-inspired algorithms that attempt to rely only on local information, such as predictive coding (PC) and equilibrium propagation. However, these algorithms have notoriously struggled to train very deep networks, preventing them from competing with BP in large-scale settings. Indeed, scaling PC networks (PCNs) has recently been posed as a challenge for the community (Pinchetti et al., 2024).
Artificial Intelligence, Machine Learning, Deep Learning, Neural Networks
32 points by frozenseven 17 days ago | 0 comments
Traditional and Neural Order-Independent Transparency (tobias-franke.eu)
Order independent transparency (OIT) is a technique in computer graphics that allows for accurate rendering of transparent objects without the need to sort them in a specific order based on their depth.
Computer Graphics, Transparency, Neural Networks
8 points by ibobev 19 days ago | 0 comments
Transformer neural net learns to run Conway's Game of Life just from examples (sidsite.com)
We find that a highly simplified transformer neural network is able to compute Conway’s Game of Life perfectly, just from being trained on examples of the game.
Artificial Intelligence, Machine Learning, Neural Networks
69 points by montebicyclelo 21 days ago | 35 comments
Continuous Thought Machines (sakana.ai)
Neural networks (NNs) were originally inspired by biological brains, yet they remain significantly distinct from their biological counterparts.
Artificial Intelligence, Neural Networks, Machine Learning
315 points by hardmaru 27 days ago | 36 comments
Towards the Cutest Neural Network (kevinlynagh.com)
I recently needed to use a microcontroller to estimate the pose (translation and orientation) of an object using readings from six different sensors.
Neural Networks, Robotics, Machine Learning
121 points by surprisetalk 37 days ago | 24 comments
World Emulation via Neural Network (madebyoll.in)
I turned a forest trail near my apartment into a playable neural world.
Neural Networks, Artificial Intelligence, Simulation, Gaming, Nature
250 points by treesciencebot 43 days ago | 46 comments
Show HN: Dosidicus – A digital pet with a simple neural network (github.com/ViciousSquid)
What if a Tamagotchi had a neural network and could learn stuff?
Neural Networks, AI, Pets, Software, Creative Projects
74 points by vicioussquid 46 days ago | 16 comments
Show HN: I'm 15 and built a neural network from scratch in C++,just math (github.com/muchlakshay)
A C++ implementation of a Multilayer Perceptron (MLP) neural network using Eigen, supporting multiple activation and loss functions, mini-batch gradient descent, and backpropagation for training.
Machine Learning, C++, Neural Networks
8 points by muchlakshay 46 days ago | 2 comments
SDFs from Unoriented Point Clouds Using Neural Variational Heat Distances (arxiv.org)
We propose a novel variational approach for computing neural Signed Distance Fields (SDF) from unoriented point clouds.
Machine Learning, Computer Vision, Neural Networks, Distance Fields
38 points by haxiomic 50 days ago | 5 comments
NoProp: Training neural networks without back-propagation or forward-propagation (arxiv.org)
The canonical deep learning approach for learning requires computing a gradient term at each layer by back-propagating the error signal from the output towards each learnable parameter.
Neural Networks, Machine Learning, Deep Learning, AI Research, Computer Science
161 points by belleville 55 days ago | 49 comments
NNN: Next-Generation Neural Networks for Marketing Mix Modeling (arxiv.org)
We present NNN, a Transformer-based neural network approach to Marketing Mix Modeling (MMM) designed to address key limitations of traditional methods.
Machine Learning, Marketing, Neural Networks, Artificial Intelligence
25 points by tmulc 60 days ago | 3 comments
Neural Graffiti – Liquid Memory Layer for LLMs (github.com/babycommando)
This is an experimental layer that merges ideas from liquid neural networks with static transformer models, using a simple but powerful "plug-in": the Spray Layer.
Generative AI, Machine Learning, Neural Networks
107 points by vessenes 61 days ago | 25 comments
A Neural Network in 11 lines of Python (2015) (iamtrask.github.io)
Summary: I learn best with toy code that I can play with. This tutorial teaches backpropagation via a very simple toy example, a short python implementation.
Neural Networks, Python, Machine Learning, Programming, Tutorials
7 points by williamtrask 65 days ago | 2 comments
(2016) Interactive Neural Network Art (otoro.net)
<h4>Interactive Neural Network Art</h4>
Art, Artificial Intelligence, Neural Networks
44 points by vinhnx 69 days ago | 6 comments
The Matrix Calculus You Need for Deep Learning (explained.ai)
Most of us last saw calculus in school, but derivatives are a critical part of machine learning, particularly deep neural networks, which are trained by optimizing a loss function.
Deep Learning, Machine Learning, Mathematics, Neural Networks
67 points by Anon84 70 days ago | 3 comments
Tied Crosscoders: Tracing How Chat LLM Behavior Emerges from Base Model (lesswrong.com)
We are interested in model-diffing: finding what is new in the chat model when compared to the base model. One way of doing this is training a crosscoder, which would just mean training an SAE on the concatenation of the activations in a given layer of the base and chat model. When training this crosscoder, we find some latents whose decoder vector mostly helps reconstruct the base model activation and does not affect the reconstruction for the chat model activation.
Generative AI, Machine Learning, AI Research, Neural Networks
7 points by aranguri 76 days ago | 0 comments
CHM Releases AlexNet Source Code (computerhistory.org)
In partnership with Google, CHM has released the source code to AlexNet, the neural network that in 2012 kick-started today’s prevailing approach to AI. It is available as open source here.
Artificial Intelligence, Open Source, Computer Science, Neural Networks, History
12 points by tosh 79 days ago | 0 comments
Neurosymbolic Decision Trees (arxiv.org)
Neurosymbolic (NeSy) AI studies the integration of neural networks (NNs) and symbolic reasoning based on logic.
Artificial Intelligence, Machine Learning, Neural Networks, Symbolic Reasoning, Decision Trees
42 points by PaulHoule 80 days ago | 0 comments
Deep Learning Is Not So Mysterious or Different (arxiv.org)
Deep neural networks are often seen as different from other model classes by defying conventional notions of generalization.
Deep Learning, Artificial Intelligence, Machine Learning, Generalization, Neural Networks
485 points by wuubuu 82 days ago | 126 comments
Arbitrary-Scale Super-Resolution with Neural Heat Fields (therasr.github.io)
Thera is the first arbitrary-scale super-resolution method with a built-in physical observation model.
Super-Resolution, Image Processing, Computer Vision, Artificial Intelligence, Neural Networks
151 points by 0x12A 84 days ago | 56 comments
Francis Crick – The Recent Excitement About Neural Networks [pdf] (1989) (wordpress.com)
Neural Networks, Artificial Intelligence, History, Science
5 points by sim04ful 88 days ago | 0 comments
Deriving Muon (jeremybernste.in)
We recently proposed Muon: a new neural net optimizer. Muon has garnered attention for its excellent practical performance: it was used to set NanoGPT speed records leading to interest from the big labs.
Machine Learning, Optimization, Neural Networks
7 points by jxmorris12 92 days ago | 3 comments
Get Started with Neural Rendering Using Nvidia RTX Kit (Vulkan) (nvidia.com)
Neural rendering is the next era of computer graphics.  By integrating neural networks into the rendering process, we can take dramatic leaps forward in performance, image quality, and interactivity to deliver new levels of immersion.
Computer Graphics, Neural Networks, Vulkan, Nvidia, Rendering
29 points by pjmlp 94 days ago | 0 comments
Analogies Between Startups and Neural Networks (mikealche.com)
I’m consulting for a company that the had a crucial team member taking “extended vacations”. Ouch!
Startups, Neural Networks, Business
10 points by yoouareperfect 101 days ago | 0 comments
Computer Simulation of Neural Networks Using Spreadsheets (2018) (arxiv.org)
The article substantiates the necessity to develop training methods of computer simulation of neural networks in the spreadsheet environment.
Neural Networks, Computer Simulation, Spreadsheets
40 points by benbreen 104 days ago | 1 comments
Z-Ant: An Open-Source SDK for Neural Network Deployment on Microprocessors (github.com/ZIGTinyBook)
Zant (Zig-Ant) is an open-source SDK designed to simplify deploying Neural Networks (NN) on microprocessors.
Open Source, Neural Networks, Microprocessors, Software Development
10 points by jedisct1 111 days ago | 0 comments
AI Designed Computer Chips That the Human Mind Can't Understand (popularmechanics.com)
A new neural network process has designed wireless chips that can outperform existing ones.
Artificial Intelligence, Computer Hardware, Neural Networks
4 points by givinguflac 126 days ago | 2 comments
An overview of gradient descent optimization algorithms (2016) (ruder.io)
Gradient descent is one of the most popular algorithms to perform optimization and by far the most common way to optimize neural networks.
Machine Learning, Optimization Algorithms, Neural Networks
135 points by skidrow 135 days ago | 26 comments
An Introduction to Neural Ordinary Differential Equations [pdf] (diposit.ub.edu)
Neural Networks, Machine Learning, Differential Equations, Mathematics
79 points by gballan 149 days ago | 10 comments
NNCP: Lossless Data Compression with Neural Networks (bellard.org)
NNCP is an experiment to build a practical lossless data compressor with neural networks.
Neural Networks, Data Compression, Machine Learning
10 points by ksec 159 days ago | 0 comments