stop sending the max_tokens param to api, when set to default
P
Public Fish
I just realized this issue today, but I was hitting the max_tokens limit on sonnet 4, and the message was misleading saying it comes from the api reaching it's limit... but there was just no way that could be true with that small amount of text.
Changing the chat max_tokens to 64000 does not return that error message, but setting it to default cuts it off...
I would expect the default to not be sent to the api at all, which would allow up to the maximum permitted by the api. Right now, apparently using default, is sending some other default values that do not match the model capabilities.
At least if I normally set max tokens per model, it's sending it correctly and even switching it on each chat when changing models, so that's great.