rare-sapphire•2y ago
API POST calls timing out
When I make calls to an external API via the API block (POST), if the call runs for more than 60 seconds, an error is generated which says, "Unable to fetch response".
I need the API block (POST) to wait longer, since the responses sometimes take more than 60 seconds.
Is there someplace that this timeout value can be set? I have no way to shorten the time the calls take as calculations are being performed that are out of my control. I know the server is working at the other end because I can see the response being sent by the server after the timeout occurs.
I have searched the documentation, this server, and Google and I cannot find this answer - but maybe I'm just not looking in the right place...
5 Replies
Are you calling the assistant api?
rare-sapphireOP•2y ago
I am, but not directly. Voiceflow calls my server, which calls the Assistant API. The Assistants API is very slow - it can take up to 3 minutes to answer. Typically it is between 30 and 90 seconds. When the calls are completed in less than 60 seconds, it works fine.
correct-apricot•2y ago
max runtime of vf api request is one min
no option to change that
Ideally you want to use a custom action to handle that part (with a custom integration or a modified version of our react chat).
We are testing extensions for the chat widget and this should help for those kind of usage.
rare-sapphireOP•2y ago
So it is "busy waiting" for a reply from the API?
It's a good thing, really, because I began to look around for a "local" solution to this problem: that of using my own documents in a vector database to by analyzed with an LLM, and I found a core-i9 running a single 4090 GPU with the LLM running locally is way, way faster than GPT in terms of response time, and the responses I get are actually far better, as the local LLM is not weighed down by being trained on so much stuff like OpenAI's GPT. So I can run the whole thing locally instead, right next to my other servers, and skip the cloud altogether! I do think VF is a nice product though.... I hope they solve the timeout thing.
It's a good thing, really, because I began to look around for a "local" solution to this problem: that of using my own documents in a vector database to by analyzed with an LLM, and I found a core-i9 running a single 4090 GPU with the LLM running locally is way, way faster than GPT in terms of response time, and the responses I get are actually far better, as the local LLM is not weighed down by being trained on so much stuff like OpenAI's GPT. So I can run the whole thing locally instead, right next to my other servers, and skip the cloud altogether! I do think VF is a nice product though.... I hope they solve the timeout thing.