Hacker News with Generative AI: Cloud Storage

Bauplan – Git-for-data pipelines on object storage (bauplanlabs.com)
Bauplan is a Pythonic data platform that provides functions as a service for large-scale data pipelines and git-for-data over S3 data lakes.

Data Pipelines, Python, Object Storage, Git, Cloud Storage

83 points by barabbababoon 10 days ago | 45 comments

Cloudflare R2 Data Catalog: Managed Apache Iceberg tables with zero egress fees (cloudflare.com)
Apache Iceberg is quickly becoming the standard table format for querying large analytic datasets in object storage. We’re seeing this trend firsthand as more and more developers and data teams adopt Iceberg on Cloudflare R2. But until now, using Iceberg with R2 meant managing additional infrastructure or relying on external data catalogs.

Cloud Storage, Data Analytics, Apache Iceberg, Cloudflare

47 points by kalendos 16 days ago | 8 comments

OxiCloud – lightweight alternative to Nextcloud file sharing (Rust) (github.com/DioCrafts)
I built OxiCloud because I wanted a simpler, faster file storage solution than existing options. After struggling with NextCloud's performance on my home server, I decided to create something that prioritizes speed and simplicity while still being robust enough for daily use.

File Sharing, Rust, Cloud Storage, Open Source, Alternatives

33 points by sandreas 26 days ago | 4 comments

Escaping USA Tech, Bye Bye Dropbox, Hello Jottacloud (mjanssen.nl)
I recently got more concerned about what is happening in the US since a certain president He-Who-Must-Not-Be-Named and what this does in the world and my personal life.

Data Privacy, Cloud Storage, Technology, USA Politics

18 points by marc0janssen 32 days ago | 0 comments

Chunking Attacks on Tarsnap (and Others) (daemonology.net)
Ten years ago I wrote that it would require someone smarter than me to extract information from the way that Tarsnap splits data into chunks.

Security, Encryption, Cloud Storage, Data Security

10 points by todsacerdoti 35 days ago | 1 comments

Filen – Encrypted cloud storage hosted in Germany (filen.io)

Cloud Storage, Privacy, Germany

4 points by Mossy9 39 days ago | 0 comments

Ask HN: in 2025 Which project does not have any good open-source alternatives? (ycombinator.com)
Dropbox server implementation. Has a great open source MacOS client, but the server implementation is needed. I would contribute financially towards such a project, as Dropbox will never be satisfied to remain reliable file sync (and I am currently a paying customer).<p>https://www.maestral.app/

Open Source, File Syncing, Software, Programming, Cloud Storage

31 points by Mugiballa7 48 days ago | 21 comments

Show HN: One year of bewCloud (a simpler Nextcloud alternative) (bewcloud.com)
Are you tired of the complexities and limitations of existing cloud platforms like Nextcloud and ownCloud?

Cloud Storage, Open Source, Alternatives, Self Hosting

50 points by BrunoBernardino 64 days ago | 49 comments

AWS S3 SDK breaks its compatible services (xuanwo.io)
The Apache Iceberg community has raised a PR S3: Disable strong integrity checksums to disable the newly introduced integrity checksums in AWS S3 SDKs.

AWS, Software, Cloud Storage, Open Source, Programming

190 points by ulrischa 64 days ago | 86 comments

Show HN: WinCse – Integrating AWS S3 with Windows Explorer (github.com/cbh34680)
WinCse is an application that integrates AWS S3 buckets with Windows Explorer, allowing you to treat S3 buckets as if they were part of your local file system.

Windows, AWS, Cloud Storage, File Management, Open Source

65 points by cbh34680 64 days ago | 20 comments

Backblaze Drive Stats for 2024 (backblaze.com)
As of December 31, 2024, we had 305,180 drives under management. Of that number, there were 4,060 boot drives and 301,120 data drives. This report will focus on those data drives as we review the Q4 2024 annualized failure rates (AFR), the 2024 failure rates, and the lifetime failure rates for the drive models in service as of the end of 2024.

Hardware, Data Storage, Cloud Storage, Statistics

646 points by TangerineDream 74 days ago | 205 comments

Backblaze seemingly does not support files greater than 1 TB (wadetregaskis.com)
For nearly a month now, Backblaze has been fixated on a particular file of mine, that happens to be over 1 TB in size. Backblaze seemingly uploads it completely, but then on the next backup it uploads it again, even though it has not changed (in eight years!). Ad infinitum.

Cloud Storage, Backblaze

91 points by amwolff 81 days ago | 80 comments

Husky: Efficient Compaction at Datadog Scale (datadoghq.com)
In a previous blog post, we introduced our Husky event store system. Husky is a distributed storage system that is layered over object storage (e.g., Amazon S3, Google Cloud Storage, Azure Blob Storage, etc.), with the query system acting as a cache over this storage. We also did a deep dive into Husky’s ingestion pipelines that we built to handle the scale of our customer data. In this post, we’ll cover how we designed Husky’s underlying data storage layer.

Data Storage, Distributed Systems, Cloud Storage, Databases, Event Streaming

30 points by gttalbot 86 days ago | 2 comments

Ask HN: What's the best way to de-Google photos? (ycombinator.com)
After a little over a year of having a kid, I've run out of google storage (photos/drive/mail). I have a pixel phone and like the automatic backup to photos... however, for years I've considered using that as a short-term backup, since I don't trust google not to nuke my account or lock me out.

Cloud Storage, Google, Privacy

7 points by strix_varius 90 days ago | 16 comments

Microsoft to Archive Unlicensed OneDrive Accounts This Month (petri.com)
Microsoft will start archiving unlicensed OneDrive for Business accounts on January 27.

Microsoft, OneDrive, Cloud Storage, Business, Accounts

3 points by taubek 108 days ago | 1 comments

We Have Google Drive at Home: Musings on Merkle-Tree Based File Sharing (dolthub.com)
Suppose you have a directory of files that you want to sync with your friends. When the files change, you want your friends to be able to download just the changes without needing to re-download the entire directory again. And you want this to scale, no matter how many or how large the files are. What's the best way to do this?

File Sharing, Data Synchronization, Distributed Systems, Cloud Storage

15 points by PaulHoule 111 days ago | 0 comments

Show HN: Open Rewind – POC for audio and screen and video streaming to S3 (github.com/janwilmake)
Today I was inspired by this tweet after which I fell into this rabbit hole to create an efficient recorder for your Screen, System audio, and Mic.

Open Source, Streaming, Audio/Video, Cloud Storage

78 points by wwoessi 111 days ago | 33 comments

Ask HN: Why Google Photos still so expensive when storage is so much cheaper? (ycombinator.com)
Ask HN: Why Google Photos still so expensive when storage is so much cheaper?

Google Photos, Cloud Storage, Pricing, Cost, Tech

9 points by sh_tomer 114 days ago | 4 comments

Show HN: I built open source file sharing solution using AWS S3 (kinsta.app)
Configure S3 Settings

Open Source, File Sharing, AWS, Cloud Storage

63 points by rohitghumare 117 days ago | 36 comments

Hetzner: S3-compatible Object Storage is here (hetzner.com)
Object Storage is the S3-compatible storage solution that grows with your data requirements - highly available, secure and flexible.

Cloud Storage, S3, Hetzner, Object Storage

5 points by coffeeri 128 days ago | 0 comments

Mount S3 like local file system for macOS (mountos.app)
mountOS (mount Object Storage) allows seamless integration of S3-compatible cloud storage, making it accessible and usable as a file system

macOS, Cloud Storage, File Systems, S3

8 points by princejwesley 129 days ago | 3 comments

5x faster satellite data retrieval (terrafloww.com)
The rapid growth of Earth observation data in cloud storage, which will continue to grow exponentially, powered by falling rocket launch prices by companies like SpaceX, has pushed us to think of how we access and analyze satellite imagery.

Satellite Data, Earth Observation, Cloud Storage, SpaceX, Data Analysis

7 points by sid_tf 138 days ago | 1 comments

S3 Tables (meltware.com)
AWS announced S3 Tables yesterday, which brings native support for Apache Iceberg to S3. It’s hard to overstate how exciting this is for the data analytics ecosystem.

AWS, Data Analytics, Apache Iceberg, Cloud Storage

43 points by pongogogo 143 days ago | 2 comments

New Amazon S3 Tables: Storage optimized for analytics workloads (amazon.com)
Amazon S3 Tables give you storage that is optimized for tabular data such as daily purchase transactions, streaming sensor data, and ad impressions in Apache Iceberg format, for easy queries using popular query engines like Amazon Athena, Amazon EMR, and Apache Spark.

Cloud Storage, Data Analytics, Big Data

19 points by craigkerstiens 143 days ago | 3 comments

Hetzner Object Storage (hetzner.com)
Object Storage is the S3 compatible storage solution that grows with your data requirements - highly available, secure and flexible.

Cloud Storage, Data Storage, Hetzner, Object Storage

11 points by matteocontrini 144 days ago | 7 comments

Wasabi (S3 Storage) is down (wasabi.com)
We are currently investigating reported network errors across all regions. Access to both Console and S3 services may return errors. We will update this page as we have more information.

Cloud Storage, Downtime, Wasabi

13 points by _Bruno42 147 days ago | 4 comments

Amazon S3 now supports the ability to append data to an object (amazon.com)
Amazon S3 Express One Zone now supports the ability to append data to an object.

Cloud Storage, Data Storage

199 points by notgiorgi 155 days ago | 67 comments

Launch HN: Regatta Storage (YC F24) – Turn S3 into a local-like, POSIX cloud FS (ycombinator.com)
Hey HN, I’m Hunter the founder of Regatta Storage (https://regattastorage.com). Regatta Storage is a new cloud file system that provides unlimited pay-as-you-go capacity, local-like performance, and automatic synchronization to S3-compatible storage. For example, you can use Regatta to instantly access massive data sets in S3 with Spark, Pytorch, or pandas without paying for large, local disks or waiting for the data to download.

Cloud Storage, S3, YC, Software, File Systems

587 points by huntaub 158 days ago | 319 comments

Firefiles: Open-Source Alternative to Dropbox, Google Drive and Tresorit (openalternative.co)
Firefiles revolutionizes cloud storage management by offering a unified platform for all your storage needs.

Cloud Storage, Open Source, File Storage, Dropbox Alternatives

10 points by sardonyx001 160 days ago | 3 comments

Delivering 15TB of 4K video with Cloudflare R2 for $2.18 (screencasting.com)
We serve a lot of video. In fact, just last month, our viewers watched over 15 terabytes of it—and with several new courses planned, our monthly bandwidth will more than double next year.

Video Streaming, Cloud Storage, Cost Optimization

70 points by peter_d_sherman 160 days ago | 14 comments