Bernice: A Multilingual Pre-trained Encoder for Twitter

Explore a groundbreaking multilingual RoBERTa language model called Bernice, designed specifically for Twitter data analysis. Learn about the development of this pre-trained encoder, which was trained from scratch on 2.5 billion tweets across multiple languages. Discover how Bernice outperforms other models adapted to social media data and strong multilingual baselines in various monolingual and multilingual Twitter benchmarks. Gain insights into the unique challenges of processing Twitter's multilingual content and how Bernice addresses the significant differences between Twitter language and other domains commonly used in large language model training.