Hacker News with Generative AI: Scaling

DeepSeek-V3: Scaling Challenges and Reflections on Hardware for AI Architectures (arxiv.org)
The rapid scaling of large language models (LLMs) has unveiled critical limitations in current hardware architectures, including constraints in memory capacity, computational efficiency, and interconnection bandwidth.

Generative AI, Hardware, Scaling

4 points by nsoonhui 7 days ago | 0 comments

Sharding Mastodon, Part 1 (pgdog.dev)
Redirecting…

Mastodon, Social Media, Distributed Systems, Scaling

4 points by levkk 23 days ago | 0 comments

Vertical Sharding Sucks (pgdog.dev)
Vertical sharding, sometimes called functional sharding, takes tables out of your main database and puts them somewhere else. Most of the time, it’s another Postgres database. This reduces load on the main DB and gives your app some breathing room to grow.

Database Design, Performance Optimization, Scaling, Postgres

20 points by samokhvalov 42 days ago | 13 comments

The Great Re-shard: adding Postgres capacity (again) with zero downtime (2023) (notion.com)
Earlier this year, we swapped out Notion’s live database cluster for a larger one without taking downtime.

Databases, Postgres, Downtime, Scaling, Technology

30 points by unchar1 88 days ago | 8 comments

Scaling with PostgreSQL without boiling the ocean (shayon.dev)
“Postgres was great when we started but now that our service is being used heavily we are running into a lot of ‘weird’ issues”

Databases, PostgreSQL, Performance, Scaling

40 points by plaur782 103 days ago | 14 comments

Value-Based Deep RL Scales Predictably (arxiv.org)
Scaling data and compute is critical to the success of machine learning. However, scaling demands predictability: we want methods to not only perform well with more compute or data, but also have their performance be predictable from small-scale runs, without running the large-scale experiment.

Machine Learning, Deep Learning, Artificial Intelligence, Scaling, Predictability

68 points by bearseascape 105 days ago | 3 comments

How to scale your model: A systems view of LLMs on TPUs (jax-ml.github.io)
Training LLMs often feels like alchemy, but understanding and optimizing the performance of your models doesn't have to. This book aims to demystify the science of scaling language models on TPUs: how TPUs work and how they communicate with each other, how LLMs run on real hardware, and how to parallelize your models during training and inference so they run efficiently at massive scale.

Machine Learning, Systems Architecture, TPUs, Scaling

185 points by mattjjatgoogle 108 days ago | 30 comments

S1: Simple Test-Time Scaling (github.com/simplescaling)
This repository provides an overview of all resources for the paper "s1: Simple test-time scaling".

Machine Learning, Computer Vision, Research Papers, Open Source, Scaling

40 points by t55 109 days ago | 3 comments

How we scaled Slack to support 1000s of developers (railway.com)
Railway makes software infrastructure for humans. Our pitch is simple. You give us a docker image or GitHub repo. We deploy and scale it, no friction.

Software Development, Scaling, Infrastructure, Cloud Computing, DevOps

173 points by eckles 119 days ago | 118 comments

Bottleneck Dirty Webs (staysaasy.com)
Delegation, specialization, and federation are critical to scaling companies. But scaling doesn’t mean stepping back from everything. Especially for unsavory, cross-functional, time intensive tasks, leaders should position themselves as bottlenecks - owners that feel pressure when the work grows too much, forcing them to find ways to push back on the growth in time and effort.

Management, Scaling, Leadership, Startups

4 points by Garbage 131 days ago | 0 comments

Ask HN: What are your experiences with scaling a company? (ycombinator.com)
Let's say your company has 20 employees (markets, products, dev) and one stable product they offer. The plan is to introduce another new product and maybe add new members to the team.

Startups, Scaling, Business Growth, Product Development, Hiring

70 points by surrTurr 165 days ago | 38 comments

Facebook's Little Red Book (map.cv)
In 2012, Facebook was facing a challenge as it hit a billion users: rapid scaling was outpacing their ability to maintain focus on the big picture. Narratives became fragmented, and with them, the essence of what tied the company to Zuckerberg's vision began to fade.

Social Media, Facebook, Business, Scaling

544 points by heshiebee 172 days ago | 283 comments

The Practical Guide to Scaling Django (slimsaas.com)
Most Django scaling guides focus on theoretical maximums. But real scaling isn’t about handling hypothetical millions of users - it’s about systematically eliminating bottlenecks as you grow. Here’s how to do it right, based on patterns that work in production.

Django, Web Development, Scaling, Performance Optimization, Software Architecture

149 points by rbanffy 189 days ago | 34 comments

Data movement bottlenecks to large-scale model training: Scaling past 1e28 FLOP (epochai.org)
Data movement bottlenecks limit LLM scaling beyond 2e28 FLOP, with a "latency wall" at 2e31 FLOP. We may hit these in ~3 years. Aggressive batch size scaling could potentially overcome these limits.

Machine Learning, Computer Science, Scaling

12 points by jasondavies 202 days ago | 1 comments

Possible futures for the Ethereum protocol, part 2: The Surge (eth.limo)
At the beginning, Ethereum had two scaling strategies in its roadmap. One (eg. see this early paper from 2015) was "sharding": instead of verifying and storing all of the transactions in the chain, each node would only need to verify and store a small fraction of the transactions. This is how any other peer-to-peer network (eg. BitTorrent) works too, so surely we could make blockchains work the same way.

Ethereum, Blockchain, Scaling, Decentralized Networks, Cryptocurrency

95 points by bpierre 219 days ago | 52 comments

Upgrading Uber's MySQL Fleet (uber.com)

Databases, MySQL, Scaling, Uber, Technology

236 points by benocodes 221 days ago | 198 comments

What can we do to make games scale? (twitter.com)

Game Development, Scaling, Optimization

4 points by LorenDB 229 days ago | 2 comments

Does It Scale (Down)? (bugsink.com)
It’s 2024, and software is in a ridiculous state.

Software Engineering, Scaling, Trends

11 points by vanschelven 229 days ago | 3 comments

How we built ngrok's data platform (ngrok.com)
At ngrok, we manage an extensive data lake with an engineering team of one (me!).