Hacker News with Generative AI: Data Analysis

How Russian Data on Sanctioned Companies is disappearing and how we put it back (opensanctions.org)
Since Russia's full-scale invasion of Ukraine in 2022, the dictatorship has increasingly hidden and falsified data in its trade registry. We’ve churned through 100GB of compressed XML files, so that we can now present both: the current, memory-impaired version as well as the pre-war data.
Data Analysis, Russia, Ukraine, Sanctions
7 points by Major_Grooves 30 days ago | 0 comments
Bluesky user activity has declined by 23% over the past three months (bluefacts.app)
This page shows how activity and usage of Bluesky has been growing over time. You can see how many users are actively posting on Bluesky and how many posts are added to Bluesky on a daily basis.
Social Media, Data Analysis, Bluesky, Twitter Alternative
21 points by uwemaurer 30 days ago | 21 comments
Python Pandas Ditches NumPy for Speedier PyArrow (thenewstack.io)
Python, Pandas, PyArrow, Performance Optimization, Data Analysis
17 points by puttycat 31 days ago | 5 comments
Frequentism and Bayesianism: A Practical Introduction (2014) (jakevdp.github.io)
One of the first things a scientist hears about statistics is that there is are two different approaches: frequentism and Bayesianism. Despite their importance, many scientific researchers never have opportunity to learn the distinctions between them and the different practical approaches that result. The purpose of this post is to synthesize the philosophical and pragmatic aspects of the frequentist and Bayesian approaches, so that scientists like myself might be better prepared to understand the types of data analysis people do.
Statistics, Science, Bayesian Statistics, Frequentist Statistics, Data Analysis
7 points by Tomte 34 days ago | 0 comments
Calculating Oil Storage Tank Occupancy with Help of Satellite Imagery (2017) (medium.com)
At TankerTrackers.com, our mission statement is to present a bird’s eye view of the physical oil market with help of tanker-tracking, storage changes and official government statistics.
Satellite Imagery, Data Analysis, Supply Chain
5 points by admp 35 days ago | 0 comments
New DSL "MassQL" lets scientists query mass spectrometry data (news.ucr.edu)
Biologists and chemists have a new programming language to uncover previously unknown environmental pollutants at breakneck speed – without requiring them to code.
Science, Programming Languages, Data Analysis, Environmental Science
23 points by jacklondon 36 days ago | 10 comments
Show HN: High-resolution surface analysis with Lidar data (github.com/r-follador)
High-resolution surface analysis with LiDAR data
Data Analysis, LiDAR, GitHub, 3D Modeling
66 points by folli 37 days ago | 10 comments
Discord Unveiled: A Comprehensive Dataset of Public Communication (2015-2024) (arxiv.org)
Discord has evolved from a gaming-focused communication tool into a versatile platform supporting diverse online communities.
Social Media, Data Analysis, Communication, Research, Community
152 points by leotravis10 37 days ago | 179 comments
Capalyze – Natural language data analysis (capalyze.ai)
上传表格，提问智答，生成洞见
Data Analysis, Business Intelligence, Software, Tools
7 points by alexliu518 40 days ago | 9 comments
What Is This Thing Called Swing? (ds.mpg.de)
Jazz must swing - jazz musicians agree on that. However, even a century after the beginnings of jazz, there is still no general agreement what exactly constitutes the swing feel. With a dedicated experiment and data analyses on more than 450 well-known jazz solos, we have tried to unravel some secrets of swing.
Music, Jazz, Research, Data Analysis
8 points by Tomte 40 days ago | 1 comments
Show HN: Buckaroo – Data table UI for Notebooks (github.com/paddymul)
Buckaroo is a modern data table for Jupyter that expedites the most common exploratory data analysis tasks.
Data Visualization, Jupyter Notebooks, Data Analysis, Python, Open Source
105 points by paddy_m 40 days ago | 9 comments
Show HN: Fahmatrix – A Lightweight, Pandas-Like DataFrame Library for Java (github.com/moustafa-nasr)
Fahmatrix is a lightweight, modern Java library for working with tabular data, inspired by Python's Pandas and rooted in the idea of making data understanding (fahm) easy on the JVM.
Java, Data Analysis, Libraries, Python, Pandas
46 points by mousomashakel 42 days ago | 13 comments
Show HN: CSV GB+ by Data.olllo – Open and Process CSVs Locally (microsoft.com)
Data Analysis, Software, CSV, Open Source
51 points by olllo 44 days ago | 22 comments
Backblaze Drive Stats for Q1 2025 (backblaze.com)
Welcome to the first Drive Stats of 2025. In case you missed it, the 2024 Drive Stats report was the last for long-time Drive Stats guru, Andy Klein, who is happily retired—off putting the “green” in greener pastures by working on his golf game. We–being Backblaze staff writer Stephanie Doyle and Chief Technical Evangelist Pat Patterson–are picking up where Andy left off, bringing you the metrics and analysis you know and love. Now, on to the numbers! 
Data Analysis, Hard Drives, Cloud Storage
21 points by HieronymusBosch 45 days ago | 0 comments
Gmail to SQLite (github.com/marcboeker)
This is a script to download emails from Gmail and store them in a SQLite database for further analysis.
Email, Database, Python, Software, Data Analysis
338 points by tehlike 49 days ago | 114 comments
Launch HN: Nao Labs (YC X25) – Cursor for Data (ycombinator.com)
Hey HN, we’re Claire and Christophe from nao Labs (https://getnao.io/). We just launched nao, an AI code editor to work with data: a local editor, directly connected with your data warehouse, and powered by an AI copilot with built-in context of your data schema and data-specific tools.
Data Analysis, AI, Programming, Tools, Software
158 points by ClaireGz 49 days ago | 64 comments
Show HN: Using eBPF to see through encryption without a proxy (github.com/qpoint-io)
Qtap: An eBPF agent that captures pre-encrypted network traffic, providing rich context about egress connections and their originating processes.
Security, Networking, eBPF, Open Source, Data Analysis
259 points by tylerflint 50 days ago | 78 comments
Show HN: YouTube Time Machine – browser extension to find forgotten videos (frankmeeuwsen.com)
Did you know that the average YouTube video is viewed about 41 times? For a platform that seemingly features professional content creators and is the second most visited site after Google, this view count might seem disappointing. Or is something else going on? I’ll explain how this fact inspired me to create a browser extension that makes visible the videos that the YouTube algorithm keeps hidden.
Browser Extensions, YouTube, Data Analysis, Video Content
10 points by thatinternetguy 52 days ago | 0 comments
Energy efficiency of heat pumps in residential buildings using operation data (nature.com)
As heat pumps become more prevalent in residential buildings, effective performance monitoring is essential.
Energy Efficiency, Heat Pumps, Residential Buildings, Data Analysis
10 points by gnabgib 52 days ago | 2 comments
Show HN: TextQuery – Query CSV, JSON, XLSX Files with SQL (textquery.app)
TextQuery is an all-in-one desktop app to import, query, modify, and visualize your raw data with SQL.
Data Analysis, Software, Databases, SQL, Desktop Applications
163 points by shubhamjain 53 days ago | 57 comments
Internet usage pattern during power outage in Spain and Portugal (akamai-mpulse.com)
On Monday this week, the Iberian Peninsula suffered a major power outage that disabled many services across these countries. In this post I'll look at the patterns we saw in mPulse data during this time.
Internet Usage, Power Outages, Spain, Portugal, Data Analysis
108 points by ghoshbinayak 53 days ago | 110 comments
DuckDB is probably the most important geospatial software of the last decade (dbreunig.com)
What happens when you embed geospatial capabilities in generalist data tools? More people engaging with geo data.
Geospatial Data, Software, Databases, Data Analysis
491 points by dbreunig 55 days ago | 165 comments
Determining favorite t-shirt color using science (ostwilkens.se)
I'm looking to simplify my wardrobe, and the t-shirt is a staple. I like solid color t-shirts, and so the main differentiating factor is the color. But what color? There is only one way to find out. That is: create images of myself with different colored t-shirts, and evaluate them in an ELO-based arena.
Science, Fashion, Data Analysis, Personal Projects
16 points by KomoD 55 days ago | 7 comments
Backstory to the Survivorship Bias Plane (yuxi-liu-wired.github.io)
I discover the exact backstory to that picture of an airplane with red dots on top of it.
Statistics, Visualizations, Data Analysis, History
4 points by YuxiLiuWired 55 days ago | 0 comments
The PCAP (weberblog.net)
For the last couple of years, I captured many different network and upper-layer protocols and published the pcaps along with some information and Wireshark screenshots on this blog. However, it always takes me some time to find the correct pcap when I am searching for a concrete protocol example. There are way too many pcaps out there.
Networking, Capture Analysis, Wireshark, Blog Posts, Data Analysis
3 points by Tomte 56 days ago | 0 comments
Zipf's Law (wikipedia.org)
Zipf's law (/zɪf/; German pronunciation: [tsɪpf]) is an empirical law stating that when a list of measured values is sorted in decreasing order, the value of the n-th entry is often approximately inversely proportional to n.
Statistics, Mathematics, Linguistics, Language, Data Analysis
6 points by baxtr 57 days ago | 0 comments
Normalizing Ratings (blogspot.com)
Data Analysis, Statistics, Normalization
51 points by Symmetry 57 days ago | 52 comments
You Wouldn't Download a Hacker News (jasonthorsness.com)
And now I can analyze it with DuckDB. Behold the fraction of total comments and stories referencing key topics over time!
Hacker News, Data Analysis, DuckDB, Trends
458 points by jasonthorsness 59 days ago | 227 comments
US Tariff Flow Analyzer (tradeflows.us)
Trade, Economics, Data Analysis, United States
6 points by forthwall 60 days ago | 0 comments
We Found Insurance Fraud in Our Crash Data (levs.fyi)
When we set out to build geospatial risk scores for vehicle crashes at Matrisk AI, we never expected that a side by side look at Vehicle Identification Numbers and crash timelines would hint at possible insurance fraud. But data sometimes surprises you. Below, I’ll walk through how we stumbled upon this discovery, what we found, and why it might matter for anyone insuring vehicles.
Insurance Fraud, Data Analysis, Machine Learning, Risk Assessment, Transportation
74 points by Ostatnigrosh 60 days ago | 73 comments