GitHub Issue #3154 -- agents simulate tool usage instead of actually calling tools. Here's why it happens and how to stop it.

The Problem

Your CrewAI agent has a search tool. You watch the logs: it thinks, decides to search, writes 'Action: search_tool', then writes a convincing Observation -- but the tool was never actually called. The agent fabricated the observation from its training data.

This is called tool hallucination. It's documented in GitHub Issue #3154 and is consistently one of the top support questions in the CrewAI community. It's particularly dangerous because the agent produces confident-looking output based on invented information -- and nothing in the output signals that the tool wasn't called.

Why It Happens

CrewAI's default agent uses the ReAct pattern: the LLM generates Thought → Action → Observation cycles. The LLM writes both the Action (what to call) and the Observation (what came back). If the underlying model is unclear about the boundary between generating an action and actually executing it, it will generate the Observation itself instead of waiting for the real tool response.

The root causes are:

  • Using a model that doesn't reliably follow tool-calling instructions (GPT-3.5, some open-source models)
  • Unclear or overlapping tool descriptions that confuse the agent
  • Too many tools -- the agent becomes uncertain which to call and skips calling
  • System prompt that doesn't sufficiently emphasise that tools must be called, not imagined

Fix 1: Upgrade the Model

Tool hallucination is dramatically less common with stronger models. If you're using GPT-3.5, Claude Haiku, or an open-source model below ~7B parameters, switch to GPT-4o, Claude Sonnet, or Claude Opus. The improvement is usually immediate and significant.

from crewai import Agent
from langchain_openai import ChatOpenAI
from langchain_anthropic import ChatAnthropic
 
# Instead of the default gpt-3.5-turbo
agent = Agent(
    role="Researcher",
    goal="Find accurate information using real tool calls",
    llm=ChatOpenAI(model="gpt-4o"),          # Much more reliable
    # or
    llm=ChatAnthropic(model="claude-sonnet-4-6"),  # Also highly reliable
    tools=[search_tool],
)

Fix 2: Enforce Tool Use in the System Prompt

Add explicit instructions to your agent's backstory that make the expectation clear: tools must be called, observations must come from real tool responses, and fabricating observations is forbidden.

agent = Agent(
    role='Research Analyst',
    goal='Research topics thoroughly using your tools',
    backstory=(
        'You are a meticulous research analyst.\n\n'
        'CRITICAL RULES:\n'
        '1. When you decide to use a tool, you MUST actually invoke it\n'
        '   and wait for the real response. Never write a fake Observation.\n'
        '2. If a tool returns an error, report the error -- do not invent a result.\n'
        '3. If you cannot find information with your tools, say so honestly.\n'
        '4. Never use training knowledge as a substitute for actual tool calls.'
    ),
    tools=[search_tool],
    verbose=True,
)

Fix 3: Reduce and Clarify Tool Descriptions

Agents with 5+ tools are significantly more likely to hallucinate than agents with 2–3 tools. Each additional tool increases ambiguity about which one to call. Two principles: give each agent the minimum tools it needs, and write tool descriptions that make the exact use case unambiguous.

from crewai.tools import BaseTool
from pydantic import BaseModel, Field
 
class SearchInput(BaseModel):
    query: str = Field(description="The specific search query to look up")
 
class WebSearchTool(BaseTool):
    name: str = "web_search"
    description: str = (
        "Search the web for CURRENT, REAL-TIME information. "
        "Use this when you need up-to-date facts, news, prices, or "
        "information that may have changed recently. "
        "Do NOT use for historical facts you already know."
    )
    args_schema: type[BaseModel] = SearchInput
 
    def _run(self, query: str) -> str:
        # Actual implementation
        return real_search_api(query)
The description should answer three questions: what does this tool do, when should it be used, and when should it NOT be used. The 'when not to use' constraint is especially important -- it prevents the agent from defaulting to the tool for everything.

Fix 4: Verify Tool Calls with Verbose Logging

Enable verbose mode and look for the actual tool execution in the logs. A real tool call looks different from a hallucinated one.

# REAL tool call -- you'll see this in verbose logs:
Action: web_search
Action Input: {"query": "latest AI regulation news 2026"}
[Tool web_search called with input: latest AI regulation news 2026]
Observation: [actual API response here]
 
# HALLUCINATED tool call -- no tool execution line:
Action: web_search
Action Input: {"query": "latest AI regulation news 2026"}
Observation: [LLM-generated fake response here]

If you don't see the [Tool ... called] line, the tool was not executed. This confirms hallucination and tells you the fix is needed.

Fix 5: Use the tools Validation Callback

For production systems where tool fidelity is critical, add a step_callback that validates each agent step. If a step contains an Observation but no corresponding real tool output, raise an error before the agent continues.

def validate_tool_call(step_output):
    """Callback to detect hallucinated tool calls."""
    # Check if this step involved a tool call
    if hasattr(step_output, 'tool') and step_output.tool:
        # Log every real tool invocation
        print(f"VERIFIED TOOL CALL: {step_output.tool}")
        print(f"Input: {step_output.tool_input}")
        print(f"Output length: {len(str(step_output.result))} chars")
    return step_output
 
agent = Agent(
    role="Researcher",
    step_callback=validate_tool_call,
    tools=[search_tool],
    ...
)

Quick Reference

  • Tool hallucination = agent writes a fake Observation instead of calling the tool
  • Primary fix: use a stronger model (GPT-4o, Claude Sonnet+)
  • Add explicit 'never fabricate tool results' instructions to agent backstory
  • Give each agent the minimum tools it needs -- fewer tools = less confusion
  • Write tool descriptions that specify when to use AND when NOT to use the tool
  • Enable verbose=True and look for [Tool ... called] lines to confirm real execution
  • Add a step_callback to log and validate every tool invocation in production