Skip to content

docs: maxTokens is a model property, not a tool property #1209

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
chiranjeet-vapi wants to merge 1 commit into
base: main
Choose a base branch
from
Open

docs: maxTokens is a model property, not a tool property #1209

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 13 additions & 17 deletions fern/tools/custom-tools-troubleshooting.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,8 @@ Start with the most common issue for your symptoms:
format problems
</Card>
<Card title="Parameters cut off" href="#token-truncation">
**Symptoms:** Tool parameters or responses truncated Increase token limits
**Symptoms:** Tool parameters or responses truncated Increase the model
token limit
</Card>
</CardGroup>

Expand Down Expand Up @@ -83,8 +84,7 @@ Add `strict: true` to catch validation errors early:
"parameters": {
// ... your parameters
},
"strict": true,
"maxTokens": 500
"strict": true
}
```

Expand Down Expand Up @@ -225,20 +225,17 @@ Tool returns data but the assistant doesn't use it in conversation.

Tool parameters or responses are getting cut off.

### Increase token limits
### Increase the model token limit

The default token limit is only 100. Increase it for complex tools:
Tool call arguments are generated by the model, so they draw on the same
per-turn token budget as speech. Raise `maxTokens` on the assistant's `model`,
not on the tool. It accepts a value from `50` to `10000` and defaults to `250`.

```json title="Tool configuration" {7}
{
"name": "complex_tool",
"description": "Tool that needs more tokens",
"parameters": {
// ... your parameters
},
"maxTokens": 500 // Increase from default 100
}
```
<Warning>
`maxTokens` is a model property, not a tool property. Setting it inside
`tools[].function` is rejected with `400 Bad Request` and the message
`assistant.model.each value in tools.function.property maxTokens should not exist`.
</Warning>

<Note>
Look for "Token truncation warnings" in your call logs to identify when this
Expand Down Expand Up @@ -366,7 +363,6 @@ Tool behavior doesn't match your expectations.
"required": ["param1"]
},
"strict": true,
"maxTokens": 500,
"async": false
}
```
Expand All @@ -389,5 +385,5 @@ Look for these key error messages in your call logs:
| "Tool call ID mismatches" | toolCallId doesn't match | Ensure exact ID match |
| "HTTP errors" | Webhook not returning 200 | Return HTTP 200 always |
| "Schema validation errors" | Missing required parameters | Check required array |
| "Token truncation warnings" | Need more tokens | Increase maxTokens |
| "Token truncation warnings" | Need more tokens | Increase `model.maxTokens` |
| "Response parsing errors" | Malformed JSON/line breaks | Fix JSON format |
Loading