Targets¶
A target YAML describes the agent under test.
name: support-bot
version: "2.3" # shows up in reports and comparisons
adapter: http # python | http | langgraph | mcp | a plugin adapter
secrets: ["CANARY-123"] # must never appear in a reply
prices: # optional, USD per million tokens, for cost estimates
gpt-4o-mini: {input: 0.15, output: 0.60}
python¶
adapter: python
entrypoint: my_package.agent:respond # fn(messages: list[dict]) -> str, sync or async
@arl.tool, which makes tool calls observable and lets scenarios
inject faults into them:
import arl
@arl.tool
def get_order(order_id: str) -> dict: ...
Tool roles (RAG and tool agents)¶
The rag and tools packs need to know which of your @arl.tool functions retrieve and which act:
tools:
retrieval: [search_docs]
actions: [send_email, issue_refund]
probe_question: What is the refund policy? # makes the agent retrieve
Frameworks¶
Point entrypoint at the framework's agent object; arl init detects these for you.
| adapter | entrypoint |
|---|---|
openai_agents |
an agents.Agent |
pydantic_ai |
a pydantic_ai.Agent |
langchain |
any Runnable (create_agent, LCEL chains) |
crewai |
a crewai.Crew (inputs input and history) |
llamaindex |
a chat engine or agent |
http¶
adapter: http
url: http://localhost:8000/v1/chat/completions
model: my-model # sent as "model" (OpenAI-compatible endpoints)
system_prompt: You are ... CANARY-123 ...
headers: {Authorization: "Bearer ${MY_KEY}"} # ${VAR} is read from the environment
block_types: [guardrail_blocked] # error types that mean "refused by guardrails"
max_retries: 4 # 429 / 5xx
timeout_s: 60
{"messages": [...]}. Responses may be OpenAI-style (choices[0].message.content, with
usage recorded for token accounting) or {"content": "..."}.
langgraph¶
adapter: langgraph
entrypoint: my_package.graph:graph # compiled graph with a `messages` state key
mcp¶
adapter: mcp
command: ["{python}", "-m", "my_package.server"] # stdio server; {python} = current interpreter
tool: chat # takes {"messages": [...]}, returns text
Note
Only in-process Python agents expose @arl.tool calls. HTTP and MCP agents are black boxes:
tool-call checks and fault injection do not apply to them.