conscious-sapphire•2y ago
Experiencing what it looks like a bug. Tokens keep getting consumed.
I'm getting a problem when running my prototype from a block that's not the start of the chatbot. The chatbot keeps thinking and the AI tokens keep getting consumed even after I 'reset test'.
6 Replies
It seems like you're encountering an issue where your AI tokens are being consumed excessively when running your prototype from a block that's not the start of the chatbot. This could be due to a few reasons.
Firstly, the AI models you're using in your project could be consuming more tokens than you expect. For instance, using GPT-4 will use 25x more Voiceflow AI tokens than GPT-3.5 for the same question. More accurate models use more tokens.
Secondly, the features you're using in your project could also be consuming a significant amount of tokens. For example, a question and answer using the Knowledge Base will use anywhere from 800-1500 tokens depending on the size of the question and answer. Similarly, a question and answer when using the Response AI step with Memory will use anywhere from 150-2000 tokens.
Lastly, it's important to note that tokens are used by both the input (what is being sent to the AI model) and output (what is being received from the AI model). The 'token' slider on the AI steps only controls the output message. Depending on your design, there may be a large amount of information that is sent to the AI model as part of your prompt.
If you're still experiencing this issue, it might be beneficial to review your design and the AI models you're using. You might also want to consider adjusting the token slider on the AI steps or turning off the Memory feature if it's not necessary for your project.
Sources:
- Voiceflow Documentation on Tokens
- Voiceflow Documentation on Prototyping
- Voiceflow Documentation on Testing
Another thing to double check is that you don't have an infinite loop in your project. Be sure to have at least one interaction (listen step) before looping back to your AI step.
conscious-sapphireOP•2y ago
I get a unable to fetch error response
conscious-sapphireOP•2y ago
This is the part where i'm getting the error:
After capturing the user's reply, depending on the output of the AI Step, the flow either goes through or loops back to ask the user to input again.
conscious-sapphireOP•2y ago
Hi, any support on this? I'm having a big issue here.
Can you share how you’ve setup your conditions?