Skip to Main Content

AI tools and academic study

Bias and Hallucinations

There are two major issues with AI generators - bias and hallucinations. These significantly challenge the reliability and fairness of any AI content.
 

Bias in AI Generators
Bias in AI can be described as systematic errors or distortions of the results returned through AI systems. This mostly manifests as unfair/prejudiced content, mirroring human biases found within training data.

Sources of AI Bias

  • Training Data: AI models are trained on vast data that can hold biases and prejudices of society.
  • Algorithmic Design: how algorithms are designed can inadvertently introduce or compound bias.
  • Human influence: developers' biases can be embedded into how models are designed and trained.

Examples of AI bias

  • Facial recognition systems performing poorly for one racial group or another.
  • Job application algorithms favoring one demographic group.
  • Language models holding stereotypical or discriminating content.

Impact of Bias

This can enable the bias in AI to perpetuate and increase already existing social inequalities, hence ensuring that the individuals from minority groups are treated unfairly and that the discriminatory practices are maintained.

 

Hallucinations in AI Generators
AI hallucinations refer to instances when AI generators fabricate false information or misleading facts and provide them as true.

Characteristics of AI hallucinations include:

  • Plausible but false information
  • Fabricated details, quotes, or sources
  • Confident presentation of inaccurate content

Causes of hallucinations:

  • Imperfect generative models trained to maximize likelihood.
  • Redistribution errors between text and internal representations
  • Relying too much on memorized knowledge during pre-training

 

Types of Hallucinations:

  • Factual errors: Falsified information stated as true
  • Fictional response: Fully invented responses
  • Response misalignment: Responses irrelevant to the input question

 

Addressing Bias and Hallucinations
A number of the following methods can be applied to such effects:
 

Bias Reduction:

  • Diversify the sources of the training data
  • Run bias detection and correction algorithms on your data
  • Audit AI systems regularly for fairness

Hallucination Reduction:

  • Fact-checking AI outputs against reliable sources
  • Comparing with the results using multiple AI platforms
  • Incorporating mechanisms that involve fact-checking
  • Improving the quality of training data and model architectures

 

Conclusion
Bias and hallucinations are two of the most important challenges when developing and deploying AI generators. The first step in developing responsible AI is to acknowledge these problems. Progress is being made on these issues, but continued research and further watchfulness are necessary to ensure that AI systems produce fair, accurate, and reliable outputs.

 

Sources:

Farquhar, S., Kossen, J., Kuhn, L. and Gal, Y. (2024) ‘Detecting hallucinations in large language models using semantic entropy’, Nature, 630, pp. 625–630. Available at: https://doi.org/10.1038/s41586-024-07421-0

MIT Sloan Teaching and Learning Technologies (2024) ‘When AI Gets It Wrong: Addressing AI Hallucinations and Bias’. Available at: https://mitsloanedtech.mit.edu/ai/basics/addressing-ai-hallucinations-and-bias/ (Accessed: 13 August 2024).    

Perplexity AI (2024) Perplexity AI response to Kathy Neville, 13 August.       

University of Oxford (2024) Major research into ‘hallucinating’ generative models advances reliability of artificial intelligence. Available at: https://www.ox.ac.uk/news/2024-06-20-major-research-hallucinating-generative-models-advances-reliability-artificial?ref=image (Accessed: 13 August 2024).

 

Further Reading:

Gender shades: Intersectional phenotypic and demographic evaluation of face datasets and gender classifiers. 

ChatGPT and AI text generators: Should academia adapt or resist? 

How can we counteract generative AI’s hallucinations?

They’re all so dirty and smelly:’ study unlocks ChatGPT’s inner racist.

AI language models are rife with different political biases.

Educause quickpoll results: Adopting and adapting to generative AI in higher ed tech. 

Humans are biased. Generative AI is even worse. 

Time for class 2023: Bridging student and faculty perspectives on digital learning. 

Tackling bias in artificial intelligence (and in humans). 

Ageism, sexism, classism, and more: 7 examples of bias in AI-generated images. 

When A.I. chatbots hallucinate. The New York Times.