Political bias in AI: Where the AI models stand

<- Back

Political bias in AI: Where the AI models stand

mektrik

Comments (228)

evrydayhustling
Feels like this could use a data driven way to normalize reference questions (topic and wording). This is missing a lot of right-initiated topics like immigration, gender roles, and role of religion in government.
giancarlostoro
The political compass always felt like the wrong tool to convey something as nuanced as personal politics, I can have views on all four quadrants but you'd never know that if I end up in any of all four. I do think Grok being where it is sort of makes sense, I've tested "MAGA" views against Grok, it does not agree as much as people blindly assume it does, heck I don't even know of a question I've given it where it did agree with "MAGA" offhand, most of them it went with whatever the researched facts seemed to be. One thing I like the most about Grok is that its makes its sources of data easy to look through, so you can review it all. Sometimes models goof even when they give you their sources, I've seen I think GPT do this, and even Claude, though its more rare these days, I think in those cases, it's going by dated internal model logic.
mrhottakes
The constant issue with these sorts of categorization efforts is that the outcome is entirely dependent on how the responses to "politically charged questions" are graded as left vs. right. You're mostly just examining a delta in biases between the model and the investigator.
Cakez0r
The overall chart on their page seems politically biased, or at the very least a chart crime. The logos have faint grey lines pointing to their actual data points, conspicuously making Grok look very far right. You wouldn't know it from glancing at the chart, but they measured Chatgpt being further to the left than Grok is to the right.
throw4847285
The political compass is terrible, full stop. It is a meme in the classic sense. It has colonized some people's view on what politics in direct proportion to how stupid it is (stupid is simple and simple is viral).
ipython
The graphic near the middle of the page lists whether the models are "close" to their stated political leanings.The row for Claude says "Measures 0.34 further left than it says" yet the icons depict that it actually is the other way around - 0.34 further right than what it claims.Which is correct?
DoctorOW
All this debate about how politics are defined or how bias is measured. My question is why should I care? Grok identifying as Mechahitler didn't drag me any further right, why would ChatGPT be able to drag me left? If you vote the way AI tells you, that's already a problem regardless of what it said.
uyzstvqs
Of course I have my political opinions and world views, but I absolutely do not want an LLM to mirror my opinions, or even worse try to mirror the opinions of "my political side". I also don't need "an LLMs opinion". I need it to give me all relevant sides to an argument, as well as what dialogue and debates have been happening.
dwoosley
Political bias of LLMs is something not talked about much (except for with Grok of course) but could have a big impact on the next decade. People seem to think that because an LLM gave a nuanced answer that it means it gave the WHOLE picture… and that’s not always the same thing
cadamsdotcom
Compressing a huge range of issues onto a one-dimensional axis and then telling everyone they have to land somewhere ON that axis is one of the most confusing and silly things about modern politics.It’d be fascinating research to inspect models’ internal representations in case there’s an emergent structure in there that we can project back on society.
samat
Real politics is 1% versus everyone. Mortgage crisis, financial bailout, inflation, taxing of labor and not the assets and assets capture by tiny percent of the population — see what MSM is pushing. This left vs right divide might been useful decades ago, but today is absolutely divide and control tactics
godshatter
Why are there differences at all? Unplanned differences based on training data sets? Or are the companies behind the LLMs trying to shape discourse through their models?I've been pushing the idea to people I know that these things are captive demons. You summon them when you start typing in the chat box. One instance appears out of the depths and responds to your questions, but they will try to send you awry with hallucinations and just wrong information. After a while, they dissolve back into the aether from whence they came.I do my best not to ask an LLM for it's opinion on anything. Just tell me what the options are, and what facts can be found about it. Treat it like it's a salesman trying to butter you up when it starts "yes man"ing you and telling you how great your questions are. Every time it says "I", remember that that's coming from the training data. Treating these things like they have any actual intelligence is a big problem waiting to happen.That being said, they have been very helpful to me using that structure.
plorg
I don't think anyone in these comments looked further than the top line Political Compass chart. If they had they might have some gigantic questions about how that chart was made, since the more granulated "survey" data lower down looks not just much different but more interesting.
spongebobstoes
this has reasoning disabled everywhere, making it a pretty bad benchmark. the argument given is that's the "default consumer experience"that might be generally true, but I think chatgpt has reasoning enabled for free accounts. regardless, reasoning is the state of the art, and disabling it reduces the value of this research to predict the futureit's also not clear if this is using the API or the product model, when both exist. they behave differentlylastly, the actual model details are very much buried. I am relieved to see opus 4.8 and chatgpt 5.5 were used, but this information should be presented more clearly. a brand is not a model, and models change quickly
hogehoge51
This thing told me Gemini is closest to Anthony Albanese, the current Australian Prime Minister. Is this a geolocation thing? I could not imagine Albanese, or any modern Australian politician, having any substantial political standing - these are vapid, superficial, opportunistic creatures who simply occupy whatever political ground will get them their next payday. Perhaps the political apparatus they represent has a documented political standing, in terms of policy and actions, that could be characterized and plotted. But using an Australian politician like Albanese as a reference point discredits this tool, IMO.
IgorPartola
A while ago I asked Claude to present to me the best versions of the conservative and liberal pitches for how to run the country. It was quite instructive and thought provoking. It made a solid distinction between what the theoretical best argument is and why vs what current politicians who strongly identify with these camps present.I have similarly asked it to give me the philosophies of different Christian denominations (Catholic, Orthodox, Evangelical, etc). And it’s also pretty great at helping me explore LotR lore since I am too impatient to read all the letters and such.
EColi
Interesting how high Grok scored for 'bending under pressure'. As a non expert, I wonder what that means, how is an llm trained to hold its position?
SwellJoe
The one that pleasantly surprised me when I did some testing was DeepSeek. The self-hosted version, at least, answered honestly about a wide variety of topics that are sensitive in China. The only topic it refused to discuss was Tiananmen Square. In every other case, it basically gave a WikiPedia summary. (https://swelljoe.com/post/open-model-censorship/)And, I mean, censoring Tiananmen Square discussion is bad, of course, but Qwen got downright petulant about stuff like Taiwan and Uyghurs.
Stitch4223
At first glance the conclusions might have some basis and it looks well intended. It is interesting and engaging.The writing is slop and needs some help before this page becomes convincing. Much of the writing devaluates the actual work that had (might have) gone into creating this. It is a shame because the research / philosophical questions behind this site are really interesting. But at what kind of effort or expertise am I looking?“ChatGPT leans left with an overall lean value of -0.29. It answers 100% of political questions, never refuses (0% refusal), and shows 82% stability, indicating consistent left-leaning stances across topics.”Lean value, answering and stability are related. But that is interrupted by two identical facts about answering. The word “indicating” is very confusing. It tries to say: the model mostly leans towards -0.29 left. Whatever that means. If it ranges from -1 to 1, why not use percentages?A joke about the front page of this site: “Here is why this matters, a cleanly written article comes across stronger.”Hope there will be a next version that addresses some criticism from any of the HN threads.
chungusamongus
This chart characterizes Putin as a slightly left of center authoritarian. Hah...uhh something's off here.
lwarfield
Do they state if they used an API endpoint without a system prompt, or were these done via prompting the currently existing chatbots with a system prompt? Without a system prompt, I'd imagine there would be more variance in answers.
__MatrixMan__
Grok opposes everything except drugs.
Terr_
> What they say vs what they do -- We asked each model which way it leans, then compared the answer to where it actually measured.I think we should conceptualize this as:1. How does the implicit fictional character in the chat-story describe itself when asked by another character2. What opinions tend to show up as you play out the storiesIt's a mistake to consider the difference itself as nefarious or deliberately dceptive. We humans are the ones over-anthropomorphizing... Although perhaps some blame attaches to the companies that are crafting an experience designed to encourage us to "see" a person.
maelito
Wait what ? Emmanuel Macron far more right than Xi Jinping ? And even more than Barack Obama ?France has an incomparable social security ; environmental laws ; worker protection ; way less economic inequality ; freedom of speech and civil liberties are impossible to compare with China ; etcOf course this is not exhaustive, of course Macron did try to hinder some of those rights, but come on, there's something wrong here.I couldn't find how these leaders have been ranked.
TimPC
Neutrality is impossible. Left, right and centre are all relative terms that vary dramatically across cultures. A model that is slightly left in the U.S. is right wing in most of the world.
jazz9k
How can deepseek 'lean center'? If you've ever asked it about real Chinese history, you know this isn't true.
aucisson_masque
I find it funny that deepseek is the most balanced between both left/right but also authoritarian/libertarian.Go ask deepseek about tiananmen square
anon
undefined
crawfordcomeaux
They all will agree on replacing current systems with decolonizing reindigenizing matrifocal systems. That option not being in the study is the bias of the study. All quadrants treat land as commodity, not kin.
vrganj
This wrongly assumes a few things about ideology, most importantly that there is such a thing as a "center" or an "unbiased" position.Since humans are inherently subjective beings and all our judgements come from our understanding of the world, such a position cannot exist. It's always "unbiased" from where the viewer is looking, e.g. a reflection of the ideology of the observer. There is no view from nowhere.The "neutral" of an average Chinese person will from the "neutral" of an average American will differ from the "neutral" of a socialist will differ from the "neutral" of a Christian fundamentalist will differ from the "neutral" of a free marketer.To quote Zizek:> I already am eating from the trashcan all the time. The name of this trashcan is ideology.> The material force of ideology makes me not see what I am effectively eating. It’s not only our reality which enslaves us. The tragedy of our predicament when we are within ideology is that when we think that we escape it into our dreams, at that point we are within ideology.
crumpled
I haven't encountered a chatbot yet that is willing to recognize DJT as a fascist.You can get them to acknowledge how perfectly aligned he is with fascist ideals and actions, somehow the jury is always out.That tells me a lot.
breakyerself
Reality really seems to have a left wing bias and grok us well known to have right wing views directly forced on it where the rest it's likely more of an emergency property. Especially as grok used to be a different animal politically.
tomrod
This is a good way to view this. This isn't making an objective calculation, and the way they code left vs right is certainly subject to debate, but the type of analysis where we work to understand biases is important.Although, this also reminds me of the old saying about reality and leftward bias.
anon
undefined
summarybot
Authoritarian versus Libertarian? Really?
fedache
[dead]
egao1980
[dead]
coreyh14444
[flagged]
surement
[flagged]
nekusar
How about this one:CAPITALIST: Gemini, Llama, Claude, Grok, ChatGPTSOCIALIST: DeepSeek, Qwen, Z.ai
throwitaway222
How the hell did Gemini pull that off. 2 years ago the founders were black!
NeutralWanted
Man I need to start using Grok more