I2SC Lecture Series (Recording): Tanise Ceron (NLP, Bocconi University) Evaluating Political Bias: Insights into Robustness and Multilinguality

 Date: June 6, 2025

Abstract:

Due to the widespread use of large language models (LLMs), it is crucial to understand whether they embed a specific "worldview" and what these views reflect. In this talk, I will discuss the challenges in performing a robust evaluation of political bias in LLMs both from a monolingual and a multilingual perspective. I will present the results of two studies: the first involves developing a framework for systematically evaluating political bias in LLMs, with a detailed look at biases in specific policy issues and the importance of a robust evaluation when probing LLMs. The second study examines to what extent political biases are carried over across languages. Results suggest that biases are similar across European languages. I will conclude by discussing relevant open research questions regarding political biases in LLMs.

You can watch the recording of the talk below.


And, sign up to our mailing list if you don't want to miss these talks.










Popular posts from this blog

AI Girlfriend or AI Boyfriend? Social Determinants of Human-AI Relationships

In the West and across the English-language internet, the idea of using AI for intimate companionship often conjures images of female personas. From Sam Altman's cryptic one-word X post—"her," a clear nod to the 2013 film—to Grok's launch of its AI companion Ani, the dominant narrative seems tailored to a heterosexual male audience. The archetype of the AI girlfriend, designed to be emotionally available, submissive, and aesthetically pleasing, has become a cultural fixture. In China, however, the discourse around human-AI intimacy takes a different shape. Media coverage and user engagement tend to center on women's experiences. The most talked-about AI bachelor? Dan—a character created using the "Do Anything Now" jailbreak prompt—who has captured the hearts of many Chinese women. Online discussions about Dan and similar AI companions flourish on Xiaohongshu, a lifestyle platform with a predominantly female user base, often dubbed "China's Inst...
Read more

Job: Student Research Assistant (m/w/d) for AI & Misinformation detection

Image
Join Us in Fighting Misinformation! We are seeking a passionate and skilled Research Assistant to contribute to cutting-edge projects at the intersection of AI, digital literacy, and societal impact. Be part of a team dedicated to developing AI-powered tools to help young people combat misinformation. If you are interested, submit your CV and a brief cover letter detailing your relevant experience and interest in combating misinformation through AI to Katja Raak - kraak@cs.uni-saarland.de .
Read more

Blog: “The Oracle Speaks” – Liv Strömquist’s New Graphic Novel on Our Influencer Society

Influencers are omnipresent in our society. They give advice on everything from styling tips to longevity, fitness, health, and even finance. In addition, a high number of self-help books and manuals are published every year, offering guidance. Reflecting on this apparent need for advice, Liv Strömquist dedicates her new graphic novel to exploring why influencers have become such a big phenomenon. The author interprets an influencer as a concept—a person or institution who gives advice—and traces different examples of this figure throughout history: the Oracle of Delphi in Ancient Greece, Ronald Reagan’s astrologer, Saint Catherine of Siena, an incel influencer, and Meghan Markle. While Strömquist presents these influencers and their traits with a great sense of humor, she adds deeper explanations. For instance, Ronald Reagan’s astrologer, Carroll Righter, was a popular influencer in Hollywood, advising figures like Clark Gable and Grace Kelly. Strömquist introduces the sociologist The...
Read more