Hacker News with Generative AI: Pre-training

Exploring Impact of Code in Pre-Training (arxiv.org)
Generative AI, Pre-training, Language Models
5 points by ijk 92 days ago | 2 comments
Electra: Pre-Training Text Encoders as Discriminators Rather Than Generators (2020) (arxiv.org)
Machine Learning, Language Models, Pre-training, Deep Learning
65 points by luu 129 days ago | 9 comments
CatLIP: Clip Vision Accuracy with 2.7x Faster Pre-Training on Web-Scale Data (arxiv.org)
Computer Vision, Machine Learning, Image Recognition, Pre-training
48 points by panabee 210 days ago | 4 comments