Async chat completions
SOTA models support up to 128k output tokens. It is impossible to leverage that through the Venice chat completions API endpoint, which times out after 15 minutes. Please create a new async chat completions endpoint, that has no time limit, and ideally, follows the same paradigm used by the batch endpoint of Anthropic and OpenAI: half the price, but may take up to 24 hours to complete.
Log in to comment and vote
No comments yet
Be the first to share your thoughts.