Hacker News with Generative AI: Vision-Language Models

RT-2: Vision-Language-Action Models (2023) (robotics-transformer2.github.io)
We study how vision-language models trained on Internet-scale data can be incorporated directly into end-to-end robotic control to boost generalization and enable emergent semantic reasoning.
Robotics, Artificial Intelligence, Vision-Language Models, Machine Learning
76 points by elsewhen 17 days ago | 13 comments
Multimodal Interpretability in 2024 (soniajoseph.ai)
I'm writing this post to clarify my thoughts and update my collaborators on multimodal interpretability in 2024. Having spent part of the summer in the AI safety sphere in Berkeley, and then joining the video understanding team at FAIR as a visiting researcher, I'm bridging two communities: the language mechanistic interpretability efforts in AI safety, and the efficiency-focused Vision-Language Model (VLM) community in industry. Some content may be more familiar to one community than the other.
Multimodal Interpretability, AI Safety, Vision-Language Models, Artificial Intelligence
30 points by apsec112 56 days ago | 0 comments