​

When Models Make Things Up

​

What Are Hallucinations?

​

Why Models Hallucinate

• Phone autocomplete: “The weather is…” → “sunny”
• AI model: “The function to reverse a list is…” → makes up something plausible

​

Common Hallucination Types

​

In Code Generation

# Model suggests:
df.remove_outliers(threshold=3)

# Reality: pandas DataFrames don't have this method
# You need to implement outlier removal yourself

​

In Documentation

// Model suggests:
import { debounce } from "react";

// Reality: React doesn't export debounce
// You need lodash or write your own

​

In Configuration

// Model suggests for package.json:
{
  "type": "module",
  "exports": "./index.js",
  "browser": true  // This isn't a valid option!
}

​

In Facts and Numbers

• “BERT has exactly 340 million parameters” (it’s actually 110M or 340M depending on version)
• “PyTorch was released in 2014” (actually 2016)
• “The optimal learning rate is always 2e-5” (depends on many factors)

​

Knowledge Cutoff Issues

​

What This Means

• Libraries released after the cutoff → Wrong or no information
• Recent best practices → Outdated advice
• Current versions → Old version numbers
• New features → Complete fabrication

​

Other Model Limitations

​

Mathematical Weakness

# Don't trust models for:
- Complex arithmetic
- Counting characters in strings
- Generating truly random numbers
- Precise floating-point operations

​

Logical Reasoning

• “All birds can fly. Penguins are birds. Can penguins fly?” → May say yes
• Counting problems: “How many ‘r’s in ‘strawberry’?” → Often wrong
• Spatial reasoning: “If A is north of B, and B is north of C…” → Gets confused

​

Consistency Issues

• Give different answers to the same question
• Change their “opinion” based on how you phrase things
• Agree with you even when you’re wrong

​

Context Limitations

• Forget earlier information
• Mix up details
• Generate based on partial understanding

​

How to Spot Hallucinations

​

Verification Strategies

​

For Code

1. IDE/Compiler Feedback
   • Red underlines indicate problems
   • Import errors show fake modules
   • Type errors reveal incorrect APIs
2. Documentation Check
   • Always verify in official docs
   • Check version compatibility
   • Look up method signatures
3. Run and Test
   • Execute the code
   • Write unit tests
   • Use debugging tools

​

For Information

1. Cross-Reference
   • Check multiple sources
   • Verify dates and numbers
   • Confirm with official documentation
2. Ask for Sources
   • Request specific documentation links
   • Ask which version supports a feature
   • Get example code from real projects
3. Test Claims
   • Verify mathematical calculations
   • Check logical conclusions
   • Run benchmarks yourself

​

Working Around Limitations

​

Prompt Engineering

# Instead of:
"Write a function to process data"

# Try:
"Write a Python function using pandas 2.0 to filter rows where column 'age' > 18"

​

Provide Context

# Include relevant details:
"Using React 18.2 with TypeScript 5.0, create a component..."

# Provide examples:
"Following this pattern from our codebase: [example]"

​

Iterative Refinement

1. Get initial suggestion
2. Test and identify issues
3. Feed errors back to model
4. Refine until correct

​

Use Model Strengths

• Pattern recognition
• Code structure
• Explaining concepts
• Generating boilerplate
• Suggesting approaches

• Latest information
• Precise calculations
• Guaranteeing correctness
• Domain-specific details
• Real-time data

​

Hallucinations in Training

​

Training Data Issues

• Incorrect labels: Model learns wrong patterns
• Outdated information: Old documentation in training set
• Contradictions: Conflicting examples confuse the model
• Sparse data: Model guesses for rare cases

​

Overfitting

• Generates training examples verbatim
• Can’t generalize to new inputs
• Mixes memorized fragments incorrectly

​

Underfitting

• Makes random guesses
• Generates generic responses
• Misses important patterns

​

Reducing Hallucinations in Your Models

​

Data Quality

​

Training Strategies

1. Validation Sets
   • Hold out test data
   • Check for hallucinations
   • Measure accuracy carefully
2. Regularization
   • Prevent overfitting
   • Use dropout
   • Early stopping
   • Weight decay
3. Temperature Control
   • Lower temperature = more conservative
   • Higher temperature = more creative (more hallucinations)
   • Find the right balance

​

Post-Training

• Human evaluation: Have experts check outputs
• Automated testing: Build test suites
• Confidence scores: Add uncertainty estimates
• Fallback options: “I don’t know” responses

​

The Verification Mindset

• Question confident assertions
• Test generated code
• Verify claimed facts
• Cross-reference sources
• Understand before using

​

Living with Limitations

• Speed: Generate solutions in seconds
• Breadth: Know about many topics
• Creativity: Suggest approaches you hadn’t considered
• Tirelessness: Available 24/7

​

Learn More from Our Research

​

Next Steps