Hacker News with Generative AI: Deep Learning

Titans: Learning to Memorize at Test Time (arxiv.org)
Over more than a decade there has been an extensive research effort on how to effectively utilize recurrent models and attention.
Deep Learning, Memory
147 points by bicepjai 2 days ago | 32 comments
Ε, a Nuisance No More (zna.do)
For a while now I have been advocating for tuning ε in various parts of the modern deep learning stack, and in this post I’ll explain why.
Deep Learning, Machine Learning, Optimization, Artificial Intelligence
3 points by jxmorris12 3 days ago | 0 comments
Titans: Learning to Memorize at Test Time (arxiv.org)
Over more than a decade there has been an extensive research effort on how to effectively utilize recurrent models and attention.
Machine Learning, Deep Learning, Memory, Attention
104 points by birriel 4 days ago | 13 comments
Intrinsic Dimensions: How Learning in Large Models Is Driven by a Few Parameters (medium.com)
Learned over-parameterized models inherently exist within a low intrinsic dimension {Li et al.¹ and Aghajanyan et al.³}. To understand this concept better, let’s delve into the following questions:
Generative AI, Machine Learning, Deep Learning
3 points by gautambt 11 days ago | 0 comments
Show HN: DeepFace – A lightweight deep face recognition library for Python (github.com/serengil)
DeepFace is a lightweight face recognition and facial attribute analysis (age, gender, emotion and race) framework for python.
Python, Deep Learning, Face Recognition, Computer Vision
257 points by serengil 15 days ago | 46 comments
1.58-Bit Flux (chenglin-yang.github.io)
We present 1.58-bit FLUX, the first successful approach to quantizing the state-of-the-art text-to-image generation model, FLUX.1-dev, using 1.58-bit weights (i.e., values in {-1, 0, +1}) while maintaining comparable performance for generating 1024 x 1024 images.
Machine Learning, Text-to-Image Generation, Image Generation, Deep Learning, Computer Science
11 points by quaintdev 16 days ago | 3 comments
Tenstorrent Wormhole Series (corsix.org)
A company called Tenstorrent design and sell PCIe cards for AI acceleration. At the time of writing, they've recently started shipping their Wormhole n150s and Wormhole n300s cards.
Hardware, AI, Computer Vision, Deep Learning, Machine Learning
9 points by FL33TW00D 17 days ago | 0 comments
Beyond Gradient Averaging in Parallel Optimization (arxiv.org)
We introduce Gradient Agreement Filtering (GAF) to improve on gradient averaging in distributed deep learning optimization.
Machine Learning, Optimization, Distributed Systems, Deep Learning
96 points by shinryudbz 18 days ago | 41 comments
Hallucination of closed repeat proteins containing central pockets (2023) (nature.com)
Inspired by these proteins, we devised a deep-learning-based approach to broadly exploring the space of closed repeat proteins starting from only a specification of the repeat number and length.
Proteins, Deep Learning, Science, Biology, AI
8 points by bookofjoe 20 days ago | 0 comments
The Structure of Neural Embeddings (seanpedersen.github.io)
A small collection of insights on the structure of embeddings (latent spaces) produced by deep neural networks.
Machine Learning, Deep Learning, Neural Networks, Embeddings
74 points by sean_pedersen 21 days ago | 4 comments
DeepSeek-V3 (deepseek.com)
Deep Learning, Software, New Releases
8 points by mromanuk 22 days ago | 0 comments
DeepSeek-v3 Technical Report [pdf] (github.com/deepseek-ai)
Deep Learning, Computer Vision, Research Papers
26 points by pongogogo 22 days ago | 2 comments
Exploring LoRA – Part 1: The Idea Behind Parameter Efficient Fine-Tuning (medium.com)
Pre-trained large language models undergo extensive training on vast data from the internet, resulting in exceptional performance across a broad spectrum of tasks. Nonetheless, in most real-world scenarios, there arises a necessity for the model to possess expertise in a particular, specialized domain.
Generative AI, Machine Learning, Deep Learning, Fine-tuning
166 points by aquastorm 26 days ago | 15 comments
No More Adam: Learning Rate Scaling at Initialization Is All You Need (arxiv.org)
In this work, we question the necessity of adaptive gradient methods for training deep neural networks.
Machine Learning, Deep Learning, Optimization, Neural Networks
91 points by jinqueeny 31 days ago | 28 comments
FastVideo: a lightweight framework for accelerating large video diffusion models (github.com/hao-ai-lab)
FastVideo is a lightweight framework for accelerating large video diffusion models.
Deep Learning, Video Processing, Computer Vision, Generative AI, Software
110 points by zhisbug 31 days ago | 24 comments
Veo 2: Our video generation model (deepmind.google)
Veo creates videos with realistic motion and high quality output, up to 4K. Explore different styles and find your own with extensive camera controls.
Video Generation, Artificial Intelligence, Deep Learning, Google, Computer Vision
587 points by mvoodarla 33 days ago | 327 comments
Sequence to sequence learning with neural networks: what a decade (youtube.com)
Deep Learning, Neural Networks, Artificial Intelligence, Machine Learning
89 points by dspoka 35 days ago | 31 comments
Founder who built Snap's AI launches a snappy new take on video chatbots (techcrunch.com)
A deep learning scientist whose last startup was acquired by Snap to build its My AI chatbot has raised seed funding for his latest venture: a platform for building and operating real-time, video-based conversational AI agents.
Artificial Intelligence, Startups, Video Chat, Conversational AI, Deep Learning
12 points by jeffspinny 37 days ago | 2 comments
From Unemployment to Lisp: Running GPT-2 on a Teen's Deep Learning Compiler (github.com/hikettei)
This repository is still in the early stages of development. Additionally, it includes many experimental approaches. Please consider this as a place to experiment with my ideas. Do not use it in a product under any circumstances.
Generative AI, Programming Languages, Deep Learning, Open Source
116 points by AymanB 39 days ago | 6 comments
Ways to Use Torch.compile (ezyang.com)
On the surface, the value proposition of torch.compile is simple: compile your PyTorch model and it runs X% faster. But after having spent a lot of time helping users from all walks of life use torch.compile, I have found that actually understanding how this value proposition applies to your situation can be quite subtle! In this post, I want to walk through the ways to use torch.compile, and within these use cases, what works and what doesn't.
PyTorch, Deep Learning, Optimization, Machine Learning, Performance
31 points by mfiguiere 39 days ago | 6 comments
Convolutional Neural Network Visualization [video] (youtube.com)
Machine Learning, Computer Vision, Deep Learning, Video
6 points by simonjgreen 42 days ago | 0 comments
FlashAttention on a Napkin:A Diagrammatic Approach to Deep Learning IO-Awareness (arxiv.org)
Optimizing deep learning algorithms currently requires slow, manual derivation, potentially leaving much performance untapped.
Deep Learning, Optimization, Computer Science, Artificial Intelligence
5 points by mpweiher 43 days ago | 0 comments
Google DeepMind team develops AI weather model (technologyreview.com)
Google DeepMind has unveiled an AI model that’s better at predicting the weather than the current best systems.
Artificial Intelligence, Weather Forecasting, Deep Learning
3 points by rrrrrrrrrrrryan 44 days ago | 0 comments
Google's DeepMind tackles weather forecasting, with great performance (arstechnica.com)
By some measures, AI systems are now competitive with traditional computing methods for generating weather forecasts.
Artificial Intelligence, Weather Forecasting, Deep Learning, Google
5 points by Gedxx 44 days ago | 0 comments
Laser: Attention with Exponential Transformation (arxiv.org)
Transformers have had tremendous impact for several sequence related tasks, largely due to their ability to retrieve from any part of the sequence via softmax based dot-product attention.
Transformers, Machine Learning, Attention Mechanisms, Deep Learning
13 points by PaulHoule 50 days ago | 0 comments
A Deep Dive into DDPMs (magic-with-latents.github.io)
DDPMs - Part 3
Generative AI, Machine Learning, Deep Learning
25 points by mariuz 52 days ago | 2 comments
The Machine and Deep Learning Compendium (gitbook.io)
Hi! When I created the Machine & Deep Learning Compendium, it was a personal list of resources curated in a private Google document, for my own education.
Machine Learning, Deep Learning, Resources, Education
6 points by reqo 53 days ago | 0 comments
Grant Sanderson: Visualizing transformers and attention [video] (youtube.com)
Machine Learning, Deep Learning, Artificial Intelligence, Video, Computer Science
33 points by skadamat 59 days ago | 0 comments
Using uv with PyTorch (astral.sh)
The PyTorch ecosystem is a popular choice for deep learning research and development. You can use uv to manage PyTorch projects and PyTorch dependencies across different Python versions and environments, even controlling for the choice of accelerator (e.g., CPU-only vs. CUDA).
Deep Learning, PyTorch, Software, Tools, Python
170 points by charliermarsh 59 days ago | 54 comments
PyTorch 101: Understanding Graphs, Automatic Differentiation and Autograd (digitalocean.com)
PyTorch is one of the foremost python deep learning libraries out there. It’s the go to choice for deep learning research, and as each days passes by, more and more companies and research labs are adopting this library.
Deep Learning, Python, Machine Learning, PyTorch
55 points by teleforce 60 days ago | 3 comments