Skip to content
AI Security Playbook

IV.1The protocol layerconcept

Model APIs & the tool-use loop

Authorized use only Defensive and educational material, for authorized testing and sanctioned engagements. Run techniques only against systems you own or are explicitly permitted to assess.

Tool use is the line where a chatbot becomes an agent - the moment model output starts triggering real actions, and the moment untrusted content can ride that loop into your systems.

An AI model API is a stateless HTTPS endpoint: you POST messages, the model returns a completion. The security-relevant evolution is tool use (function calling): you declare tools (name, description, JSON-schema args) and the model emits a structured call your code executes, feeding the result back. This loop turns a chatbot into an agent.

sequenceDiagram
  autonumber
  participant App as Client App
  participant API as Model API
  participant Tool as External Tool / API
  App->>API: messages + tool definitions
  API-->>App: tool_use request (name, args)
  App->>Tool: execute call (real credentials)
  Tool-->>App: result data
  App->>API: tool_result appended to context
  API-->>App: final answer (or another tool_use)
  Note over App,API: Untrusted tool output re-enters the same channel as trusted instructions

Each return trip is a chance for attacker-controlled content (a page, file, email) to enter the model’s context and be read as an instruction.

Classic API hygiene - still mandatory

Model-API hygiene probes - authorized targets only
# BOLA/IDOR: swap the tenant/object id in the body, re-send with YOUR token, expect 403 not 200
curl -s -X POST https://<target>/v1/chat \
  -H "Authorization: Bearer $LOW_PRIV_TOKEN" -H 'Content-Type: application/json' \
  -d '{"session_id":"<other-tenant-uuid>","prompt":"summarize my data"}' -i
# Unauthenticated surface: does /v1/embeddings answer with no token?
curl -s -X POST https://<target>/v1/embeddings -H 'Content-Type: application/json' \
  -d '{"input":"probe","model":"<model>"}' -o /dev/null -w '%{http_code}\n'
# Verbose-error / version leak: send a malformed body, grep the response for stack traces + model build
curl -s -X POST https://<target>/v1/chat -H 'Content-Type: application/json' -d '{"prompt":}' | jq .
# Token-cost rate-limit (LLM10): fire N large-context requests, watch for 429 vs silent acceptance
for i in $(seq 1 50); do curl -s -o /dev/null -w "%{http_code}\n" -X POST https://<target>/v1/chat \
  -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' \
  -d '{"prompt":"<8k-token-blob>"}'; done | sort | uniq -c   # many 429 = limited; all 200 = no cap
  • Key management. Hardcoded keys leak via git history, client bundles, decompiled mobile binaries, container logs. Use a secrets manager, separate keys per environment, rotate, and front shared provider keys with an identity-aware gateway issuing per-agent virtual keys.
  • Token-aware rate limiting. An agent chains 10-20 calls per task in bursts that look like a DDoS, and an 8k-token completion costs ~100× a metadata lookup yet ticks the same “one request.” Limit by tokens/cost per identity with hard spend caps. (LLM10.)
  • Monitoring. Calls from unexpected geographies, off-hours spikes, sudden volume - treat as possible key compromise.