Tool Use & Function Calling: How LLMs Interact with External Systems

Tool use and **function calling** represent one of the most significant advances in LLM capabilities. Instead of just generating text, models can now invoke external functions, query APIs, and interact with systems—transforming from conversational assistants into capable agents. This guide explains how **tool use** works and how to design systems that LLMs can effectively interact with.

Understanding Tool Use

What Is Tool Use?

Tool use (also called function calling) allows LLMs to:

1. Recognize when a task requires external capabilities
2. Select the appropriate tool from available options
3. Generate properly formatted arguments
4. Interpret results and continue the conversation

Instead of hallucinating an answer, the model can actually retrieve real data or perform real actions.

The Tool Use Flow

User: "What's the weather in Berlin?"
           ↓
LLM: Recognizes need for external data
           ↓
LLM: Selects weather_lookup tool
           ↓
LLM: Generates arguments: {"city": "Berlin"}
           ↓
System: Executes function, returns {"temp": 8, "conditions": "cloudy"}
           ↓
LLM: "The weather in Berlin is 8°C and cloudy."

How It Differs from RAG

Both can be combined: an agent might use RAG for background knowledge and tools for real-time actions.

Tool Use Across Providers

OpenAI Function Calling

OpenAI introduced function calling in June 2023. Tools are defined with JSON Schema:

{
  "type": "function",
  "function": {
    "name": "get_weather",
    "description": "Get current weather for a location",
    "parameters": {
      "type": "object",
      "properties": {
        "location": {
          "type": "string",
          "description": "City name, e.g., 'Berlin, Germany'"
        },
        "units": {
          "type": "string",
          "enum": ["celsius", "fahrenheit"],
          "description": "Temperature units"
        }
      },
      "required": ["location"]
    }
  }
}

{
  "choices": [{
    "message": {
      "tool_calls": [{
        "id": "call_abc123",
        "type": "function",
        "function": {
          "name": "get_weather",
          "arguments": "{\"location\": \"Berlin, Germany\", \"units\": \"celsius\"}"
        }
      }]
    }
  }]
}

Anthropic Tool Use

Claude uses a similar but distinct format:

{
  "name": "get_weather",
  "description": "Get current weather for a location. Use this when users ask about weather conditions.",
  "input_schema": {
    "type": "object",
    "properties": {
      "location": {
        "type": "string",
        "description": "City and country, e.g., 'Berlin, Germany'"
      },
      "units": {
        "type": "string",
        "enum": ["celsius", "fahrenheit"],
        "default": "celsius"
      }
    },
    "required": ["location"]
  }
}

{
  "content": [
    {
      "type": "tool_use",
      "id": "toolu_01ABC",
      "name": "get_weather",
      "input": {
        "location": "Berlin, Germany",
        "units": "celsius"
      }
    }
  ]
}

Google Gemini Function Calling

Gemini uses function declarations:

{
  "functionDeclarations": [{
    "name": "get_weather",
    "description": "Get weather for a location",
    "parameters": {
      "type": "OBJECT",
      "properties": {
        "location": {
          "type": "STRING",
          "description": "City name"
        }
      },
      "required": ["location"]
    }
  }]
}

Key Differences

Designing Effective Tools

Tool Definition Best Practices

1. Clear, Specific Names

// Good
"name": "search_product_catalog"

// Bad
"name": "search"  // Too vague
"name": "searchProductCatalogByQueryAndFilterAndSort"  // Too complex

2. Detailed Descriptions

The description is critical—it's how the LLM decides when to use the tool:

{
  "name": "create_calendar_event",
  "description": "Creates a new event on the user's calendar. Use this when the user wants to schedule a meeting, appointment, reminder, or any time-based event. Requires at least a title and start time. The event will be created in the user's default calendar unless specified otherwise."
}

3. Well-Documented Parameters

{
  "parameters": {
    "type": "object",
    "properties": {
      "query": {
        "type": "string",
        "description": "Search query. Supports boolean operators (AND, OR, NOT) and phrase matching with quotes. Example: '\"wireless mouse\" AND ergonomic'"
      },
      "category": {
        "type": "string",
        "enum": ["electronics", "clothing", "home", "sports"],
        "description": "Product category to filter results. If omitted, searches all categories."
      },
      "max_results": {
        "type": "integer",
        "minimum": 1,
        "maximum": 50,
        "default": 10,
        "description": "Maximum number of results to return"
      }
    },
    "required": ["query"]
  }
}

4. Use Enums for Constrained Values

{
  "status": {
    "type": "string",
    "enum": ["pending", "confirmed", "cancelled"],
    "description": "Filter by booking status"
  }
}

Enums prevent invalid values and help the LLM understand valid options.

Tool Categories

{
  "name": "lookup_order",
  "description": "Retrieves order details by order ID or customer email",
  "parameters": {
    "type": "object",
    "properties": {
      "order_id": {"type": "string"},
      "email": {"type": "string", "format": "email"}
    }
  }
}

{
  "name": "update_shipping_address",
  "description": "Updates the shipping address for an order. Only works for orders not yet shipped.",
  "parameters": {
    "type": "object",
    "properties": {
      "order_id": {"type": "string"},
      "new_address": {
        "type": "object",
        "properties": {
          "street": {"type": "string"},
          "city": {"type": "string"},
          "postal_code": {"type": "string"},
          "country": {"type": "string"}
        },
        "required": ["street", "city", "postal_code", "country"]
      }
    },
    "required": ["order_id", "new_address"]
  }
}

{
  "name": "send_confirmation_email",
  "description": "Sends an order confirmation email to the customer. Use after successful order placement.",
  "parameters": {
    "type": "object",
    "properties": {
      "order_id": {"type": "string"},
      "include_tracking": {"type": "boolean", "default": true}
    },
    "required": ["order_id"]
  }
}

Handling Complex Objects

For nested structures, define clear schemas:

{
  "name": "create_invoice",
  "parameters": {
    "type": "object",
    "properties": {
      "customer": {
        "type": "object",
        "properties": {
          "name": {"type": "string"},
          "email": {"type": "string"},
          "address": {
            "type": "object",
            "properties": {
              "street": {"type": "string"},
              "city": {"type": "string"},
              "country": {"type": "string"}
            }
          }
        },
        "required": ["name", "email"]
      },
      "items": {
        "type": "array",
        "items": {
          "type": "object",
          "properties": {
            "description": {"type": "string"},
            "quantity": {"type": "integer", "minimum": 1},
            "unit_price": {"type": "number", "minimum": 0}
          },
          "required": ["description", "quantity", "unit_price"]
        },
        "minItems": 1
      },
      "due_date": {
        "type": "string",
        "format": "date",
        "description": "Payment due date in YYYY-MM-DD format"
      }
    },
    "required": ["customer", "items"]
  }
}

Implementation Patterns

Basic Tool Execution Loop

def run_agent(user_message, tools, max_iterations=10):
    messages = [{"role": "user", "content": user_message}]

    for _ in range(max_iterations):
        response = llm.chat(messages=messages, tools=tools)

        # Check if model wants to use tools
        if response.tool_calls:
            for tool_call in response.tool_calls:
                # Execute the tool
                result = execute_tool(
                    tool_call.name,
                    tool_call.arguments
                )

                # Add tool result to conversation
                messages.append({
                    "role": "tool",
                    "tool_call_id": tool_call.id,
                    "content": json.dumps(result)
                })
        else:
            # No more tool calls, return final response
            return response.content

    return "Max iterations reached"

Parallel Tool Calls

Modern LLMs can request multiple tools simultaneously:

{
  "tool_calls": [
    {
      "id": "call_1",
      "function": {"name": "get_weather", "arguments": "{\"location\": \"Berlin\"}"}
    },
    {
      "id": "call_2",
      "function": {"name": "get_weather", "arguments": "{\"location\": \"Munich\"}"}
    }
  ]
}

Execute in parallel for efficiency:

import asyncio

async def execute_tools_parallel(tool_calls):
    tasks = [
        execute_tool_async(tc.name, tc.arguments)
        for tc in tool_calls
    ]
    return await asyncio.gather(*tasks)

Error Handling

Tools should return structured errors:

def execute_tool(name, arguments):
    try:
        result = tools[name](**arguments)
        return {"success": True, "data": result}
    except ValidationError as e:
        return {"success": False, "error": "invalid_input", "message": str(e)}
    except NotFoundError as e:
        return {"success": False, "error": "not_found", "message": str(e)}
    except Exception as e:
        return {"success": False, "error": "internal_error", "message": "An error occurred"}

The LLM can then respond appropriately:

Tool result: {"success": false, "error": "not_found", "message": "Order #12345 not found"}
LLM: "I couldn't find an order with that number. Could you double-check the order ID?"

Confirmation for Destructive Actions

For state-changing operations, implement confirmation:

{
  "name": "delete_account",
  "description": "Permanently deletes a user account. IMPORTANT: Always confirm with user before calling.",
  "parameters": {
    "type": "object",
    "properties": {
      "user_id": {"type": "string"},
      "confirmation_code": {
        "type": "string",
        "description": "Confirmation code provided by user"
      }
    },
    "required": ["user_id", "confirmation_code"]
  }
}

Tool Use for Web Integration

Exposing Your API as Tools

Transform your REST API into tool definitions:

POST /api/products/search
Body: { "query": string, "category": string, "limit": number }
Response: { "products": [...], "total": number }

{
  "name": "search_products",
  "description": "Search the product catalog",
  "parameters": {
    "type": "object",
    "properties": {
      "query": {"type": "string"},
      "category": {"type": "string"},
      "limit": {"type": "integer", "default": 10}
    },
    "required": ["query"]
  }
}

OpenAPI to Tools Conversion

Many tools can auto-generate from OpenAPI specs:

# openapi.yaml
paths:
  /products/{id}:
    get:
      operationId: getProduct
      summary: Get product by ID
      parameters:
        - name: id
          in: path
          required: true
          schema:
            type: string

Becomes:

{
  "name": "getProduct",
  "description": "Get product by ID",
  "parameters": {
    "type": "object",
    "properties": {
      "id": {"type": "string"}
    },
    "required": ["id"]
  }
}

Rate Limiting and Quotas

Protect your systems from agent overuse:

def execute_tool_with_limits(name, arguments, user_id):
    # Check rate limits
    if rate_limiter.is_exceeded(user_id, name):
        return {
            "success": False,
            "error": "rate_limited",
            "message": "Too many requests. Please try again in 60 seconds.",
            "retry_after": 60
        }

    # Execute and track
    result = execute_tool(name, arguments)
    rate_limiter.record(user_id, name)
    return result

Security Considerations

Input Validation

Never trust LLM-generated arguments blindly:

def search_database(query: str, table: str):
    # Validate table name against whitelist
    allowed_tables = ["products", "orders", "customers"]
    if table not in allowed_tables:
        raise ValidationError(f"Invalid table: {table}")

    # Parameterized query (never string concatenation)
    return db.execute(
        "SELECT * FROM ? WHERE name LIKE ?",
        [table, f"%{query}%"]
    )

Permission Scoping

Tools should respect user permissions:

def update_order(order_id: str, updates: dict, user_context: dict):
    order = get_order(order_id)

    # Check ownership
    if order.user_id != user_context["user_id"]:
        if not user_context.get("is_admin"):
            raise PermissionError("Cannot modify another user's order")

    # Check what can be modified
    allowed_fields = get_allowed_fields(user_context["role"])
    for field in updates.keys():
        if field not in allowed_fields:
            raise PermissionError(f"Cannot modify field: {field}")

    return apply_updates(order, updates)

Audit Logging

Log all tool invocations:

def execute_tool_with_logging(name, arguments, context):
    log_entry = {
        "timestamp": datetime.utcnow(),
        "tool": name,
        "arguments": sanitize_for_logging(arguments),
        "user_id": context.get("user_id"),
        "session_id": context.get("session_id"),
        "request_id": generate_request_id()
    }

    try:
        result = execute_tool(name, arguments)
        log_entry["success"] = True
        log_entry["result_summary"] = summarize_result(result)
    except Exception as e:
        log_entry["success"] = False
        log_entry["error"] = str(e)
        raise
    finally:
        audit_logger.log(log_entry)

    return result

Advanced Patterns

Tool Chaining

Some tasks require multiple tools in sequence:

User: "Book me a flight to London next Tuesday and add it to my calendar"

Agent:
1. search_flights(destination="London", date="next Tuesday")
2. book_flight(flight_id="BA123", passenger=user)
3. create_calendar_event(title="Flight to London", datetime=...)

Design tools to return IDs that can be passed to subsequent tools.

Dynamic Tool Loading

Load tools based on context:

def get_available_tools(user_context):
    tools = [base_tools]  # Always available

    if user_context.get("is_premium"):
        tools.extend(premium_tools)

    if user_context.get("department") == "sales":
        tools.extend(sales_tools)

    if user_context.get("permissions", {}).get("can_delete"):
        tools.extend(destructive_tools)

    return tools

Fallback Strategies

Handle tool failures gracefully:

def search_with_fallback(query):
    # Try primary search
    try:
        result = primary_search.execute(query)
        if result["total"] > 0:
            return result
    except Exception:
        pass

    # Fallback to secondary
    try:
        return secondary_search.execute(query)
    except Exception:
        return {"products": [], "total": 0, "fallback_used": True}

Testing Tool Definitions

Schema Validation

Test that your schemas are valid:

import jsonschema

def test_tool_schema(tool_definition):
    # Validate against JSON Schema draft
    jsonschema.Draft7Validator.check_schema(
        tool_definition["parameters"]
    )

LLM Selection Testing

Test that the LLM selects the right tool:

test_cases = [
    {
        "input": "What's the weather in Paris?",
        "expected_tool": "get_weather",
        "expected_args": {"location": "Paris"}
    },
    {
        "input": "Find me a blue shirt under $50",
        "expected_tool": "search_products",
        "expected_args": {"query": "blue shirt", "max_price": 50}
    }
]

def test_tool_selection(test_cases):
    for case in test_cases:
        response = llm.chat(
            messages=[{"role": "user", "content": case["input"]}],
            tools=all_tools
        )
        assert response.tool_calls[0].name == case["expected_tool"]

Integration Testing

Test the full loop:

def test_end_to_end():
    result = run_agent(
        "What's my order status for order #12345?",
        tools=[lookup_order_tool]
    )
    assert "shipped" in result.lower() or "delivered" in result.lower()

Conclusion

Tool use transforms LLMs from conversational interfaces into capable agents that can interact with real systems. The key to effective tool use lies in:

1. Clear tool definitions with detailed descriptions
2. Well-structured schemas that guide the LLM
3. Robust error handling that enables graceful recovery
4. Security measures that protect your systems
5. Comprehensive testing that validates behavior

As you design APIs and services, consider how they might be exposed as tools. The same principles that make APIs developer-friendly—clear documentation, consistent patterns, helpful error messages—also make them LLM-friendly.

The organizations that master tool use will build the most capable AI agents. Start by exposing your existing APIs as tools, then iterate based on how agents actually use them.