Hacker News with Generative AI: Data Science

USDS Engineering Director Resigns: 'This Is Not the Mission I Came to Serve' (wired.com)
The director of data science and engineering for the United States Digital Service—which Elon Musk rebranded as the US DOGE Service—has resigned from her position.

Resignation, Elon Musk, Data Science

9 points by jay_kyburz 14 hours ago | 7 comments

Show HN: Bag of words – Build and share smart data apps using AI (github.com/bagofwords1)
Bag of words enables users to create comprehensive dashboards with a single prompt and refine them iteratively.

AI, Data Visualization, Data Science, Software, Show HN

12 points by bow14 3 days ago | 8 comments

Physics Informed Neural Networks (pages.dev)
I mentioned in a previous post that as part of my position as junior DS for IKEA, I also get the opportunity to take part in their training program (called AI accelerator program). It’s a six months long program in which we are trained on all aspects of data science and AI by both professionals but also through various courses.

Machine Learning, Artificial Intelligence, Data Science

96 points by nchagnet 4 days ago | 20 comments

Basketball has evolved into a game of calculated decision-making (nabraj.com)
Basketball has evolved from a game of unpredictability into a game of calculated decision-making with the use of data and analytics.

Basketball, Sports, Analytics, Data Science

125 points by nabaraz 5 days ago | 261 comments

Classic Data science pipelines built with LLMs (github.com/Pravko-Solutions)
Below you’ll find an overview of each file, what it showcases, and links to the actual code. You can run these examples (assuming you’ve installed FlashLearn and set your “OPENAI_API_KEY”) simply by using:

Data Science, Generative AI, Programming, GitHub

196 points by galgia 11 days ago | 86 comments

Stop using zip codes for geospatial analysis (2019) (carto.com)
Uncover deeper insights beyond ZIP codes with geospatial analysis. Explore the limitations of ZIP codes and discover alternatives for spatial understanding.

Geospatial Analysis, Data Visualization, Data Science, Cartography

184 points by voxadam 13 days ago | 129 comments

AI by Hand Exercises in Excel (github.com/ImagineAILab)
AI by Hand ✍️ Exercises in Excel

Artificial Intelligence, Machine Learning, Excel, Data Science

122 points by todsacerdoti 14 days ago | 8 comments

Clinical Trials Are Drowning in Data but Starving for Patients (seangeiger.substack.com)
The first clinical trial happened on a damp naval ship in 1747. James Lind, a Scottish naval surgeon, divided twelve scurvy-stricken sailors into pairs to test different treatments: oranges and lemons, cider, vinegar, sulfuric acid, garlic paste, and seawater.

Clinical Trials, Health, Data Science

7 points by benjaminmaccini 14 days ago | 0 comments

Ask HN: Learning PySpark and Related Tools (ycombinator.com)
Hey HN,<p>I have been working in the data-science and machine-learning domain for the past 8 years or so. I have not been exposed to tools such as PySpark etc. which are being asked frequently in job descriptions. What resource or certification can I use to get upto par on PySpark?<p>Thanks!

Data Science, Machine Learning, Python, Programming, Big Data

9 points by rookie123 15 days ago | 6 comments

Ask HN: Is There Any Pattern Matching Bible or Set of Essential Readings? (ycombinator.com)
Publication analyzing/unifying topics like Dynamic Time Warping, Spell checking, Knuth-Moris-Pratt andother string search, Hidden Markov Chains, and many other text/image/signal processing algorithms/metrics in coherent form?

Pattern Matching, Algorithm, Data Science, Text Processing, Computer Science

6 points by ZevsVultAveHera 19 days ago | 8 comments

Ask HN: How to passively prepare for a job interview? (ycombinator.com)
Hey there fellow HNers! I am currently at a stage where I am working within a Tech Consulting company for 5-6 years. My plan is to prepare myself well in the next 5-6 months and then possibly start actively interviewing for jobs. How can I do passive job search and interviewing in the meanwhile so that I can setup myself best after few months? I am in a data-science and machine-learning domain.

Job Searching, Data Science, Machine Learning, Career Advice

34 points by rookie123 21 days ago | 13 comments

Two Bites of Data Science in K (zdsmith.com)
“no bowler with as many wickets has a better average”.

Data Science, Programming, Statistics

71 points by crux 25 days ago | 10 comments

Modern Polars – A side-by-side comparison of the Polars and Pandas libraries (kevinheavey.github.io)
This is a side-by-side comparison of the Polars and Pandas dataframe libraries, based on Modern Pandas by Tom Augsburger.

Data Analysis, Data Science, Libraries, Python

31 points by sebg 28 days ago | 4 comments

Cosine Similarity: Not the Silver Bullet We Thought It Was (shaped.ai)
In the world of machine learning and data science, cosine similarity has long been a go-to metric for measuring the semantic similarity between high-dimensional objects. However, a new study by researchers at Netflix and Cornell University challenges our understanding of this popular technique, exposing the underlying issues that could lead to arbitrary and meaningless results.

Machine Learning, Data Science, Research, Cosine Similarity

10 points by elashri 33 days ago | 0 comments

Hallucination is a problem we'll have to live with for a long time (theregister.com)
A notable flaw of AI is its habit of "hallucinating," making up plausible answers that have no basis in real-world data. AWS is trying to tackle this by introducing Amazon Bedrock Automated Reasoning checks.

Generative AI, Artificial Intelligence, Data Science

9 points by rustastra 33 days ago | 11 comments

PyViz – Overview of the Python visualization landscape (pyviz.org)
The Python visualization landscape can seem daunting at first. These overviews attempt to shine light on common patterns and use cases, comparing or discussing multiple plotting libraries. Note that some of the projects discussed in the overviews are no longer maintained, so be sure to check the list of dormant projects before choosing that library.

Python, Data Visualization, Libraries, Tools, Data Science

47 points by leonry 35 days ago | 3 comments

Directory of job boards by category (remote only, data science, etc.) (jobsearchdb.com)
Find an active job board in your career niche

Job Searching, Remote Work, Data Science

26 points by dougjudytobin 35 days ago | 2 comments

Don't use cosine similarity carelessly (migdal.pl)
Midas turned everything he touched into gold. Data scientists turn everything into vectors. We do it for a reason — as gold is the language of merchants, vectors are the language of AI1.

Cosine Similarity, Data Science, AI, Machine Learning

439 points by stared 37 days ago | 96 comments

Dbt Labs acquires SDF Labs (getdbt.com)
The TL;DR: today, I have the pleasure of announcing that dbt Labs has acquired SDF Labs. The two teams are already working side-by-side to bring SDF’s SQL comprehension technology into the hands of dbt users everywhere. SDF will be a massive upgrade to the very heart of the dbt user experience moving forward.

Software, Data Science, Acquisitions, Data Pipelines

109 points by karakanb 37 days ago | 48 comments

Show HN: SemHash – Fast Semantic Text Deduplication for Cleaner Datasets (github.com/MinishLab)

Data Science, Machine Learning, Software, GitHub

19 points by Pringled 39 days ago | 6 comments

How to become a Data Scientist? My journey, overview of skill set, practice tips (mljar.com)
In recent years, many people have been drawn to the field of data science, often believing it to be a fast track to wealth.

Data Science, Career Advice, Machine Learning

12 points by pplonski86 42 days ago | 4 comments

Small Data [video] (youtube.com)

Data, Videos, Data Science

86 points by spacechild1 45 days ago | 11 comments

Time-Series Anomaly Detection: A Decade Review (arxiv.org)
Recent advances in data collection technology, accompanied by the ever-rising volume and velocity of streaming data, underscore the vital need for time series analytics.

Time-Series Analysis, Data Science, Machine Learning, Anomaly Detection, Research

446 points by belter 45 days ago | 80 comments

JPL Horizons on-line solar system data and ephemeris computation service (nasa.gov)
The JPL Horizons on-line solar system data and ephemeris computation service provides access to key solar system data and flexible production of highly accurate ephemerides for solar system objects ( loading… ). ( unable to load body counts… please try later ).

Space Exploration, Astronomy, Data Science, Solar System

125 points by perihelions 52 days ago | 25 comments

Name Collision of the Year: Vector (crunchydata.com)
I can’t get through a zoom call, a conference talk, or an afternoon scroll through LinkedIn without hearing about vectors. Do you feel like the term vector is everywhere this year? It is.

Data Science, Technology, Trends, AI, Machine Learning

12 points by winslett 56 days ago | 0 comments

Optimality of Frequency Moment Estimation (weizmann.ac.il)

Data Science, Algorithms, Statistics

8 points by cryptohell 57 days ago | 1 comments

Model Evaluation with RandomForest and AdaBoost (deepnote.com)
This notebook demonstrates how to train and evaluate machine learning models, specifically focusing on the Random Forest Classifier and AdaBoost Classifier. We will use synthetic data and visualize key metrics, including accuracy, precision, recall, and F1-score.

Machine Learning, Data Science, Classification, Random Forest, AdaBoost

18 points by QuantumCoder111 66 days ago | 2 comments

Maximum likelihood estimation and loss functions (rish-01.github.io)
When I started learning about loss functions, I could always understand the intuition behind them. For example, the mean squared error (MSE) for regression seemed logical—penalizing large deviations from the ground-truth makes sense. But one thing always bothered me: I could never come up with those loss functions on my own. Where did they come from? Why do we use these specific formulas and not something else?

Machine Learning, Statistics, Mathematics, Data Science

119 points by snprajwal 67 days ago | 31 comments

Ask HN: Why is Ilya saying data is limited when the whole world is data? (ycombinator.com)
In his recent talk Ilya S. said that the data running out is a fundamental constraint on the scaling laws.

Generative AI, Data Science

12 points by georgestrakhov 67 days ago | 16 comments

Array Languages for Clojurians (2020) (appliedscience.studio)
A discussion in the Clojure data science Zulip led me to Slobodan Blazeski’s enlightening article Array languages for Lisp programmers in the journal of the British APL Association. He quotes Alan Perlis:

Clojure, Data Science, Programming Languages, Functional Programming

106 points by simonpure 68 days ago | 13 comments