Hacker News with Generative AI: AI Safety

Anthropic achieves ISO 42001 certification for responsible AI (anthropic.com)
We are excited to announce that Anthropic has achieved accredited certification under the new ISO/IEC 42001:2023 standard for our AI management system.

Artificial Intelligence, AI Safety, Standards

83 points by Olshansky 2 days ago | 83 comments

AIs Will Increasingly Attempt Shenanigans (lesswrong.com)
Increasingly, we have seen papers eliciting in AI models various shenanigans.

Artificial Intelligence, AI Safety, Ethics

34 points by surprisetalk 30 days ago | 33 comments

Quick takes on the recent OpenAI public incident write-up (surfingcomplexity.blog)
OpenAI recently published a public writeup for an incident they had on December 11, and there are lots of good details in here! Here are some of my off-the-cuff observations:

OpenAI, Generative AI, AI Safety, Incident Reports, Blog Posts

108 points by azhenley 34 days ago | 69 comments

OpenAI, GoogleDeepMind, and Meta Get Bad Grades on AI Safety (ieee.org)
Leading AI companies scored poorly across the board for various metrics related to ensuring their products are safe.

AI Safety, Artificial Intelligence, Technology, Ethics

10 points by jnord 35 days ago | 7 comments

AI hallucinations: Why LLMs make things up (and how to fix it) (kapa.ai)
An AI assistant casually promises a refund policy that never existed, leaving a company liable for an invented commitment. This incident with Air Canada’s chatbot is a clear example of 'AI hallucination,' where AI can generate confident, yet entirely fictional, answers. These errors—ranging from factual inaccuracies and biases to reasoning failures—are collectively referred to as 'hallucinations.'

Generative AI, Artificial Intelligence, AI Safety

196 points by emil_sorensen 45 days ago | 246 comments

Multimodal Interpretability in 2024 (soniajoseph.ai)
I'm writing this post to clarify my thoughts and update my collaborators on multimodal interpretability in 2024. Having spent part of the summer in the AI safety sphere in Berkeley, and then joining the video understanding team at FAIR as a visiting researcher, I'm bridging two communities: the language mechanistic interpretability efforts in AI safety, and the efficiency-focused Vision-Language Model (VLM) community in industry. Some content may be more familiar to one community than the other.

Multimodal Interpretability, AI Safety, Vision-Language Models, Artificial Intelligence

30 points by apsec112 57 days ago | 0 comments

Google Gemini tells grad student to 'please die' while helping with his homework (theregister.com)
When you're trying to get homework help from an AI model like Google Gemini, the last thing you'd expect is for it to call you "a stain on the universe" that should "please die," yet here we are, assuming the conversation published online this week is accurate.

Generative AI, Artificial Intelligence, Google, Ethics, AI Safety

51 points by daphnemichala 61 days ago | 4 comments

SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks (arxiv.org)
Despite efforts to align large language models (LLMs) with human intentions, widely-used LLMs such as GPT, Llama, and Claude are susceptible to jailbreaking attacks, wherein an adversary fools a targeted LLM into generating objectionable content.

Generative AI, Security, AI Safety

54 points by amai 62 days ago | 23 comments

Google Gemini told a user to die (gemini.google.com)

Artificial Intelligence, AI Safety

10 points by doener 63 days ago | 2 comments

Google AI chatbot responds with a threatening message: "Human Please die." (cbsnews.com)
A grad student in Michigan received a threatening response during a chat with Google's AI chatbot Gemini.

Artificial Intelligence, Chatbots, Google, Threat, AI Safety

11 points by dylan604 64 days ago | 12 comments

AI Safety and the Titanic Disaster (onepercentrule.substack.com)
If you have any interest in AI, and everyone should, then the lessons from the Titanic are highly relevant for the safe deployment of AI – especially as we are full steam ahead on deployment of this potentially societal changing technology.

AI Safety, Artificial Intelligence, Technology, History

9 points by drcwpl 78 days ago | 0 comments

GPT-4o Jailbroken by saying it is connected to disk with any file on planet (twitter.com)

Generative AI, AI Safety, Twitter

42 points by mixeden 92 days ago | 18 comments

Tesla Preferred to Hit Oncoming Car Not Pedestrian (reddit.com)
Tesla preferred to hit the oncoming car instead of hitting the pedestrian who fell on the road.

Self-Driving Cars, AI Safety, Tesla, Accidents

8 points by 14 98 days ago | 3 comments

AGI is far from inevitable (ru.nl)

Artificial Intelligence, AI Safety, AGI

162 points by mpweiher 110 days ago | 243 comments

OpenAI Threatening to Ban Users for Asking Strawberry About Its Reasoning (futurism.com)
OpenAI is now threatening to ban users that try to get the large language model to reveal how it thinks — a glaring example of how the company has long since abandoned its original vision of championing open source AI.

Artificial Intelligence, Open Source, Ethics, AI Safety

270 points by EgoIncarnate 121 days ago | 229 comments

OpenAI o1 System Card [pdf] (openai.com)

OpenAI, AI Safety, Documentation

57 points by cubefox 127 days ago | 6 comments

GPT-4o System Card (openai.com)

Generative AI, AI Safety

30 points by meetpateltech 163 days ago | 13 comments

Bypassing Meta's Llama Classifier: A Simple Jailbreak (robustintelligence.com)

AI Safety, Security

5 points by rntn 172 days ago | 0 comments

CriticGPT: Finding GPT-4's mistakes with GPT-4 (openai.com)