Skip to main content

Async chat completions

SOTA models support up to 128k output tokens. It is impossible to leverage that through the Venice chat completions API endpoint, which times out after 15 minutes. Please create a new async chat completions endpoint, that has no time limit, and ideally, follows the same paradigm used by the batch endpoint of Anthropic and OpenAI: half the price, but may take up to 24 hours to complete.

Log in to comment and vote

No comments yet

Be the first to share your thoughts.