Hacker News with Generative AI: Scalability

Scaling to users requires Synapse Pro (element.io)
Since launching Synapse Pro to support nation-scale Matrix deployments, we’ve had a lot of questions asking how it differs from vanilla “community” AGPL Synapse.

Software, Open Source, Scalability

13 points by gmemstr 18 hours ago | 13 comments

PostgreSQL Meets ScyllaDB's Lightning Speed and Monstrous Scalability (medium.com)
PostgreSQL’s inability to handle our growing write-intensive workloads and real-time demands made it clear that a change was necessary.

Databases, Performance, Scalability, PostgreSQL, ScyllaDB

7 points by sebg 22 days ago | 2 comments

400TB Single Cluster: OceanBase Powers Kwai`s Core Business (oceanbase.github.io)
Kwai is a short video app boasting more than 10 million daily active users. How does it efficiently process highly concurrent user requests? Kwai once deployed multiple MySQL clusters in the backend to support high traffic with large data storage and satisfactory performance. What are the weak points of this conventional sharding solution? What pushed Kwai to select distributed databases and eventually deploy OceanBase Database?

Databases, Scalability, Distributed Systems

44 points by jinqueeny 24 days ago | 10 comments

Managing Large-Scale Redis Clusters on K8s – Kuaishou's Approach (kubeblocks.io)
As a popular short-form video application, Kuaishou relies heavily on Redis to deliver low-latency responses to its users.

Redis, Kubernetes, Scalability, Performance, Microservices

26 points by west0n 54 days ago | 5 comments

Docker Compose Isn't Enough (tealok.tech)
Docker Compose creates problems as you scale a single server to many applications. We should learn from it and build something better.

Docker, Microservices, Scalability, DevOps

126 points by hartspear 66 days ago | 198 comments

Virtual Texturing (playerunknownproductions.net)
Building a game engine that supports earth-sized environments requires us to pay attention to the scalability of our engine. For example, the amount of data existing on the planet cannot be loaded into RAM or VRAM. Every system we design needs to take into account the assumption that data is not loaded and needs to be streamed in or in some cases even needs to be generated first.

Game Development, Virtual Reality, Scalability

4 points by carlos-menezes 67 days ago | 0 comments

Why software only moves forward (swizec.com)
At scale there are no rollbacks and no cut-overs. Your software only moves forward.

Software Development, Software Architecture, Scalability

92 points by speckx 74 days ago | 64 comments

How do you deploy in 10 seconds? (bearblog.dev)
This post describes my lessons learned after 10 years running production environments in sizes ranging from "just getting started" to a "Series F" company with 4-9 SLAs.

Deployment, Software Engineering, Scalability, Production Environments

62 points by mpweiher 82 days ago | 54 comments

Building a better and scalable system for data migrations (yorickpeterse.com)
Recently I read about a new Rust web framework called rwf. I was especially curious about its approach to handling database migrations, as many frameworks released in recent years tend to either overlook this or provide a mechanism that won't scale to larger projects.

Software Development, Databases, Rust, Scalability

68 points by YorickPeterse 86 days ago | 36 comments

Join-Accumulate Machine: A Semi-Coherent Scalable Trustless VM (graypaper.com)
We present a comprehensive and formal definition of Jam, a protocol combining elements of both Polkadot and Ethereum.

Blockchain, Scalability, Trustless Computing, Cryptography, Distributed Systems

16 points by danicuki 99 days ago | 1 comments

Were RNNs all we needed? (arxiv.org)
The scalability limitations of Transformers regarding sequence length have renewed interest in recurrent sequence models that are parallelizable during training.

Deep Learning, Sequence Modeling, Scalability

520 points by beefman 106 days ago | 260 comments

Stop Designing Your Web Application for Millions of Users When You Dont Have 100 (darrenhorrocks.co.uk)
It’s easy to get carried away when you’re building a new web app. You’ve got big ideas, you picture millions of users flocking to your platform, and you start imagining the kind of infrastructure needed to handle all that traffic. So, you build for scale from day one—optimising databases, setting up powerful servers, and ensuring everything is robust enough for massive growth.

Web Development, Optimization, Scalability

120 points by riz_ 120 days ago | 98 comments

SwiftUI Apps at Scale (jacobbartlett.substack.com)

SwiftUI, Mobile Development, User Interfaces, Scalability

17 points by jakey_bakey 136 days ago | 0 comments

How to Build a Scalable Ingestion Pipeline for Enterprise GenAI Applications (enterprisebot.ai)

Generative AI, Enterprise Applications, Data Engineering, Scalability

7 points by ritzaco 137 days ago | 0 comments

Behind AWS S3's Scale (highscalability.com)

Cloud Computing, Scalability

22 points by zerojames 141 days ago | 11 comments

Unified Grid: How we re-architected Slack for our largest customers (slack.engineering)