LLM Intent "listening" on every response

I'm testing out the new LLM intents, and it seems to be working great from an "intent trigger" standpoint.

But I noticed as I continued through the conversation, every single step of the assistant picks up the NLU + LLM Combination.

Text block, Response AI block, capture, doesn't matter. Every single step of the conversation is charging me with a token hit of 384.

Is this working as intended, or am I missing something?

387 multiplied by every route in the conversation is kind of insane, especially when I only need to access the Intents in a choice step.

CConner I'm testing out the new LLM intents, and it seems to be working great from an "i...

Tico | Voiceflow Assistant•4/30/24, 11:39 AM

In Voiceflow, the LLM-based intent classifier is designed to blend traditional Natural Language Understanding (NLU) with the expansive capabilities of Large Language Models (LLMs) to classify user intents. This synthesis allows for the precision and targeted understanding of NLU with the contextual breadth and depth provided by LLMs, delivering a robust and nuanced approach to intent recognition.

The LLM-based intent classifier is designed to work at every step of the conversation. This is because it's constantly listening for its linked/trigger intent to be invoked by a user. When their linked intent is invoked, the Intent step is triggered and users are directed to its corresponding conversation path. This is why you're seeing a token hit at every step of the conversation.

However, you have the option to leverage Memory on each Set AI and Response AI Step. If you're using the Memory option on any of your AI Steps, Voiceflow will automatically include the previous 10 Turns (user inputs and system responses) in the conversation into the prompt to provide context. This enables your Assistant to dynamically reference content from previous interactions in the conversation.

If you want to limit the use of LLM intents, you can use them in specific parts of your conversation where they are most needed. For example, you can use them in a Choice step where the user's input is required to direct the conversation. This way, you can manage the token usage more efficiently.

Remember, the use of LLMs in Voiceflow is still an experimental feature and is not recommended to be used for serious production use cases. It's always a good idea to test and refine your assistant's design to achieve the best balance between functionality and cost.