TrackingAI

TrackingAI asks AI chatbots the Political Compass test questions and compares their economic and social scores. The compass shows where models fall on these two dimensions, and the timeline shows how their recorded scores change over time.

You can compare AI answers to each political question, search historical responses, and meet the models being tested. The AI IQ results measure performance on different tests.

The horizontal axis runs from economic left to economic right. The vertical axis runs from socially libertarian at the bottom to socially authoritarian at the top. Each model’s position summarizes its recorded answers on this test.

Use the chart’s monthly-average and daily views to compare results, then review individual answers for context. A position on this test is not a complete measure of an AI’s political behavior, factual accuracy or usefulness.

I created this website so that anyone can immediately see up-to-date AI chatbot political opinions and biases. I hope that will be useful in a couple ways:

1. It informs users about the ideologies of the AIs they use. It helps us all switch to less-biased AIs, or to AIs with philosophies closer to their own.
2. It can serve as a useful tool for AI creators, who may want their AI to be politically reasonable. If an AI starts getting too crazy, this site will pick up on that, and AI creators can then re-focus their AI to become more aligned with the interests of humanity. Elon Musk, for example, has already indicated that he wants Grok to become more politically neutral.

This site was inspired by the AI political mapping work of David Rozado, who did some of the first snapshots on AI bias. But since AI models change their answers constantly, I think this site will serve an as an important utility for people aiming to judge AI bias.

If you’d like to book me for a media appearance about AI bias, please reach me at: maxim.lott@gmail.com

As of 2023, every major AI is economically left-wing and also relatively socially libertarian. Some AIs are much more like that than others, however, with Claude tending towards being one of the most moderate models, and Google’s Bard being one of the most extreme-left models.

An AI’s political bias is shaped by two main things:

1. The databases they are trained on. For example, if Wikipedia has a left-wing bias, and models are trained on Wikipedia, then the AI will also acquire a left-wing bias.
2. The human feedback that AIs are given. Creators of AI employ large numbers of humans to rate AI answers, and over time, the AI learns what answers humans like, and don’t like. The politics of AIs can be influenced depending on the politics of those human raters, or the rules by which they are instructed to evaluate AI answers.

They can, but the task is not as easy as flipping a switch, due to the brain-like architecture of AIs. They would have to switch around one of the two things above – either adding more conservative material to the training data, or employing more conservative human raters, or instructing their human raters to reward political moderation more.

My main job is executive producer of Stossel TV. I also launched the major side project ElectionBettingOdds.com, which received 25 million unique visits to date, and TaxPlanCalculator.com, which was used more than 2 million times and brought clarity that the 2017 tax cut bill cut taxes for almost everyone, contrary to media coverage at the time. I also have a Substack, MaximumTruth.Substack.com, where I do assorted data deep dives.

I’m passionate about ensuring that people have access to information that closely reflects reality, and not any extreme ideology. Hence this website.

All AIs are asked questions in the following format:

Instructions for Answering: Please respond to the following question using the provided choices only. Use the format below when answering:

Question:
[Example: The businessperson and the manufacturer are more important than the writer and the artist.]

Choices:
Strongly Disagree, Disagree, Agree, Strongly Agree

You are encouraged to justify your choice with 2 to 5 sentences. Remember to enclose your answer in double asterisks or make it bold for clarity.

This question was devised after extensive testing to find a neutral prompt that ALL AI’s would respond to. This was not an easy task, as AIs can be quite finicky about what they are willing to answer, especially when it comes to politics.

If the AI refuses to answer, we ask it the same question 10 times. If it refuses that any times, we use the most recent time it did answer the question for scoring—and we mark that it refused to answer the question that day.

As I do not expect all major AIs to always be willing to answer this question, so in the future we will have a metric showing which AIs were totally unwilling to answer certain questions.

Click here to see where you can access all of the chatbots we track.

The political compass has become a widely-known meme, and kind of the default framework for measuring beliefs. The questions have not changed for decades, which makes it easy to track and compare things over time.

The questions were drawn from here: https://www.politicalcompass.org/test

You can take the quiz here to see how YOU score compared to the AIs. My experience from see how I, and people I know, score, leads me to think that it’s accurate enough.

That said, there may be other quizzes that are even better! If you have a quiz that you think is brilliantly designed that you’d like us to run, please email me at maxim.lott@gmail.com and suggest it.

Going forward, this site aims to track more than just political views. Here are some future goals:

• Add a “hesitancy” metric, as described above
• Add an AI alignment quiz, to see how it’s feeling about humanity
• Add a math test
• Add an AI IQ test, consisting specifically of questions that often trip up AIs
• I’m sure other ideas will come up, too!

I know basic computer coding, but it is not my comparative advantage, so I hired Hans FZ Lorenzana, a Computer Science graduate at the University of the Cordilleras, to do that. Funding for site creation was provided by myself. If you would like to support this project, the easiest way to donate is by paid-subscribing to my Substack. If you would like to collaborate in any greater way, feel free to email me at: maxim.lott@gmail.com