Hacker News with Generative AI: Deep Learning

PyGraph: Robust Compiler Support for CUDA Graphs in PyTorch (arxiv.org)
CUDA Graphs -- a recent hardware feature introduced for NVIDIA GPUs -- aim to reduce CPU launch overhead by capturing and launching a series of GPU tasks (kernels) as a DAG. However, deploying CUDA Graphs faces several challenges today due to the static structure of a graph. It also incurs performance overhead due to data copy. In fact, we show a counter-intuitive result -- deploying CUDA Graphs hurts performance in many cases.
CUDA, PyTorch, Performance Optimization, GPU Programming, Deep Learning
81 points by mfiguiere 1 day ago | 8 comments
Three things everyone should know about Vision Transformers (arxiv.org)
After their initial success in natural language processing, transformer architectures have rapidly gained traction in computer vision, providing state-of-the-art results for tasks such as image classification, detection, segmentation, and video analysis.
Computer Vision, Transformers, Artificial Intelligence, Deep Learning
67 points by reqo 2 days ago | 16 comments
Double Descent Demystified: size of smallest non-zero singular value of X (arxiv.org)
Double descent is a surprising phenomenon in machine learning, in which as the number of model parameters grows relative to the number of data, test error drops as models grow ever larger into the highly overparameterized (data undersampled) regime.
Machine Learning, Deep Learning, Mathematical Concepts
8 points by highfrequency 2 days ago | 1 comments
Improving Deep Learning with a Little Help from Physics (quantamagazine.org)
Rose Yu has a plan for how to make AI better, faster and smarter — and it’s already yielding results.
Deep Learning, Artificial Intelligence, Physics, Machine Learning
10 points by marsh_mellow 2 days ago | 0 comments
Show HN: Keep your PyTorch model in VRAM by hot swapping code (github.com/valine)
This is an example of how to hotswap PyTorch training code without unloading your model weights from VRAM.
Machine Learning, PyTorch, Deep Learning, Software, Programming
77 points by valine 5 days ago | 7 comments
Sparsely-Gated Mixture of Experts (MoE) (thegreenplace.net)
In transformer models, the attention block is typically followed by a feed forward layer (FF), which is a simple fully-connected NN with a hidden layer and nonlinearity.
Transformers, Deep Learning, Machine Learning, Computer Science
15 points by mfrw 6 days ago | 0 comments
A curated blog for learning LLM internals: tokenize, attention, PE, and more (ycombinator.com)
I've been diving deep into the internals of Large Language Models (LLMs) and started documenting my findings.
Machine Learning, Deep Learning, Computer Science
6 points by zljdanceholic 6 days ago | 1 comments
Packing Input Frame Context in Next-Frame Prediction Models for Video Generation (lllyasviel.github.io)
Diffuse thousands of frames at full fps-30 with 13B models using 6GB laptop GPU memory. Finetune 13B video model at batch size 64 on a single 8xA100/H100 node for personal/lab experiments. Personal RTX 4090 generates at speed 2.5 seconds/frame (unoptimized) or 1.5 seconds/frame (teacache). No timestep distillation. Video diffusion, but feels like image diffusion.
Video Generation, Deep Learning, Computer Vision, Artificial Intelligence
269 points by GaggiX 7 days ago | 27 comments
Show HN: I built a deep learning engine from scratch in Python (github.com/whitegra)
It implements deep learning architecture and training logic without relying on NumPy, PyTorch, or any external libraries. Every operation—tensor arithmetic, backpropagation, attention, and optimization—is executed through hand-written, minimal Python logic.
Deep Learning, Python, Machine Learning, Software Development, Open Source
30 points by gmwhitebox_dev 9 days ago | 3 comments
Understanding Some Limits of DeepSeek (ycombinator.com)
Recently I asked deepseek about how to use javascript to extract and makes computations with moodle in education. I note that the program did not consider two crucial points: 1) It is of utmost importance that the answer and the grade of the answer should be in the same row. 2) Don't modify the answer of the student. In this case the answer are ten letters in response to a test with ten questions.
Deep Learning, Education, Software Development, Programming
9 points by hazymemory 10 days ago | 0 comments
The path to open-sourcing the DeepSeek inference engine (github.com/deepseek-ai)
A few weeks ago, during Open Source Week, we open-sourced several libraries. The response from the community has been incredibly positive - sparking inspiring collaborations, productive discussions, and valuable bug fixes. Encouraged by this, we’ve decided to take another step forward: contributing our internal inference engine back to the open-source community.
Open Source, Deep Learning, Inference Engine, Artificial Intelligence
550 points by Palmik 12 days ago | 63 comments
NoProp: Training neural networks without back-propagation or forward-propagation (arxiv.org)
The canonical deep learning approach for learning requires computing a gradient term at each layer by back-propagating the error signal from the output towards each learnable parameter.
Neural Networks, Machine Learning, Deep Learning, AI Research, Computer Science
161 points by belleville 12 days ago | 49 comments
Universal photonic artificial intelligence acceleration (nature.com)
Over the past decade, photonics research has explored accelerated tensor operations, foundational to artificial intelligence (AI) and deep learning1,2,3,4, as a path towards enhanced energy efficiency and performance5,6,7,8,9,10,11,12,13,14.
Photonic Computing, Artificial Intelligence, Hardware, Science, Deep Learning
14 points by westurner 16 days ago | 6 comments
Tom and Jerry One-Minute Video Generation with Test-Time Training (test-time-training.github.io)
Adding TTT layers into a pre-trained Transformer enables it to generate one-minute videos with strong temporal consistency and motion smoothness.
Generative AI, Computer Vision, Machine Learning, Deep Learning, Video Generation
80 points by walterbell 18 days ago | 18 comments
Deep Learning, Deep Scandal (garymarcus.substack.com)
Deep learning is indeed finally hitting a wall, in the sense of reaching a point of diminishing results. That’s been clear for months. One of the clearest signs of this is the saga of the just-released Llama 4, the latest failed billion (?) dollar attempt by one of the majors to create what we might call GPT-5 level AI.
Deep Learning, Artificial Intelligence, Generative AI
37 points by imichael 18 days ago | 21 comments
AI masters Minecraft: DeepMind program finds diamonds without being taught (nature.com)
An artificial intelligence (AI) system has for the first time figured out how to collect diamonds in the hugely popular video game Minecraft — a difficult task requiring multiple steps — without being shown how to play.
Artificial Intelligence, Video Games, Minecraft, Deep Learning
6 points by bglazer 22 days ago | 0 comments
AI image recognition detects bubble-like structures in the universe (phys.org)
To learn more about the deepest reaches of our own galaxy and the mysteries of star formation, Japanese researchers have created a deep learning model.
Artificial Intelligence, Astronomy, Deep Learning, Universe
110 points by PaulHoule 26 days ago | 78 comments
The Matrix Calculus You Need for Deep Learning (explained.ai)
Most of us last saw calculus in school, but derivatives are a critical part of machine learning, particularly deep neural networks, which are trained by optimizing a loss function.
Deep Learning, Machine Learning, Mathematics, Neural Networks
67 points by Anon84 28 days ago | 3 comments
Self-Supervised Learning from Images with JEPA (2023) (arxiv.org)
This paper demonstrates an approach for learning highly semantic image representations without relying on hand-crafted data-augmentations.
Self-Supervised Learning, Computer Vision, Image Recognition, Artificial Intelligence, Deep Learning
40 points by Brysonbw 28 days ago | 10 comments
How DeepSeek Rewrote the Transformer [video] (youtube.com)
Transformers, Deep Learning, Computer Vision, Artificial Intelligence, Video
5 points by chetangole 29 days ago | 0 comments
Physics-Based Deep Learning v4 (arxiv.org)
This document is a hands-on, comprehensive guide to deep learning in the realm of physical simulations.
Deep Learning, Physics, Machine Learning
5 points by __rito__ 29 days ago | 1 comments
Optimizing ML training with metagradient descent (arxiv.org)
A major challenge in training large-scale machine learning models is configuring the training process to maximize model performance, i.e., finding the best training setup from a vast design space.
Machine Learning, Optimization, Deep Learning, Training, Meta-Learning
83 points by ladberg 31 days ago | 13 comments
VGGT: Visual Geometry Grounded Transformer (github.com/facebookresearch)
Computer Vision, Transformers, Artificial Intelligence, Deep Learning, Facebook Research
190 points by xnx 32 days ago | 42 comments
DeepSeek V3 is now the highest scoring non-reasoning model (twitter.com)
Something went wrong, but don’t fret — let’s give it another shot.
Deep Learning, Artificial Intelligence, Benchmarking
14 points by aurareturn 32 days ago | 3 comments
The Original 2012 AlexNet Is Open Source Now (github.com/computerhistory)
This package contains the original 2012 AlexNet code.
Computer Vision, Deep Learning, Open Source, Machine Learning, History
3 points by kwiktrip 32 days ago | 0 comments
Attention is NOT all you need (twitter.com)
Something went wrong, but don’t fret — let’s give it another shot.
Artificial Intelligence, Deep Learning
6 points by eunos 32 days ago | 0 comments
MIT 6.S191: Deep Generative Modeling [video] (youtube.com)
Deep Learning, Artificial Intelligence, Education, Generative AI, Computer Science
7 points by mfiguiere 33 days ago | 0 comments
Deepseek V3-0324 (huggingface.co)
This model is not currently available via any of the supported Inference Providers.
Deep Learning, New Releases
18 points by bananaflag 33 days ago | 2 comments
Mac Studio M3 Ultra can run Deepseek R1 671B in memory using <200W (techradar.com)
Mac, Computers, Deep Learning, Hardware, Power Consumption
7 points by mizzao 34 days ago | 1 comments
PyTorch Internals: Ezyang's Blog (ezyang.com)
This post is a long form essay version of a talk about PyTorch internals, that I gave at the PyTorch NYC meetup on May 14, 2019.
Deep Learning, Python, PyTorch, Open Source, Machine Learning
443 points by Anon84 35 days ago | 24 comments