Hacker News with Generative AI: Accuracy

30% drop in O1-preview accuracy when Putnam problems are slightly variated (openreview.net)
As large language models (LLMs) continue to advance, many existing benchmarks designed to evaluate their reasoning capabilities are becoming saturated.
Artificial Intelligence, Benchmarking, Research, Accuracy
560 points by optimalsolver 17 days ago | 526 comments
Elektročas HH3 – the most accurate pendulum clock on the planet (cern.ch)
Can a pendulum clock be good to one second in 158 million years? A piece of cake...
Timekeeping, Physics, Clocks, Accuracy
109 points by tverbeure 18 days ago | 42 comments
Elektročas HH3: the most accurate pendulum clock on the planet (cern.ch)
Can a pendulum clock be good to one second in 158 million years? A piece of cake...
Timekeeping, Physics, Clocks, Accuracy
7 points by fanf2 19 days ago | 1 comments
We fine-tuned Llama and got 4.2x Sonnet 3.5 accuracy for code generation (finecodex.com)
Generative AI, AI Models, Code Generation, Accuracy
136 points by banddk 20 days ago | 77 comments
How can this 6 axis robot have a static accuracy of 0.05 mm? (2021) [video] (youtube.com)
Robotics, Engineering, Accuracy, Manufacturing
275 points by unclefuzzy 84 days ago | 80 comments
Can the New Mathstral LLM Accurately Compare 9.11 and 9.9? (secondstate.io)
Mathematics, AI, Accuracy
23 points by 3Sophons 182 days ago | 32 comments
“Imprecise” language models are smaller, speedier, and nearly as accurate (ieee.org)
Generative AI, Computer Science, Accuracy
302 points by jnord 232 days ago | 169 comments
Improving GPT 4 Function Calling Accuracy (composio.dev)
Generative AI, AI, Programming, Accuracy
40 points by hudixt 254 days ago | 1 comments