KB providing false answers
Hey, so my clients KB answers with false information that is not provided in the reference information. It has the information to answer correctly, but answers with information that is not in the original information. How can I fix this?
17 Replies
The Knowledge Base (KB) in Voiceflow works by leveraging your documents and website data to power answers. Here's a high-level overview of how it functions:
1. Upload a Document: When you upload a document, it is turned into 'chunks' or pieces of text.
2. Question Processing: When a question is sent to the KB, it determines which 'chunks' are most similar to the question.
3. Wrapper Prompt Creation: The system combines the chunks, the question, and the custom instructions you provided into a structured wrapper prompt. This entire package is sent to the AI model.
4. Answer Generation: The AI model returns an answer based on the information provided.
If you're implementing the Knowledge Base, you can add new data sources by navigating to the Knowledge Base tab within the Content section of your dashboard. You can select the type of data source that aligns with your requirements, such as URL(s), Plain text, Sitemap, PDF, and Docx. You also have the option to set a refresh rate for individual data sources to ensure the information remains up-to-date.
To further refine the performance of your Knowledge Base, customization options are available through the KB Settings modal. Here, you can adjust settings such as Model, Temperature, Max Tokens, Instructions, System, and Chunk Limit to tailor the generation of responses to your specific needs.
For more detailed guidance, you can refer to the following resources:
- How the Knowledge Base Works – Voiceflow
- Step 2: Create Assistant
Has this just started happening? Is the agent live?
It has happened before, but not like this. And yes it’s live
compatible-crimson•2y ago
Can you give an example of false information? @Radiant raindrop
Off course, so the subject was CFT “cognitive functional therapy” and I asked the question “how long is the training period for CFT?”. The answer was between 8-10 months, but it responded with 12 days wich is completely false.
I knew that it had the information for the question so I was surprised when it answered with false info. Anything I can do about it?
compatible-crimson•2y ago
Hm, that likely didn't hit the KB but instead hit the AI model I'm guessing
Maybe, but I can’t change the model when I’m using the KB response can I?
compatible-crimson•2y ago
I mean the answer is not coming from the KB, but the model
does it say "AI" or have a brain icon next to it?
The brain/kb icon
compatible-crimson•2y ago
Hmm weird. Does it show the chunk / source it’s coming from?
Yeah, and in the chunks is the correct information
It’s super weird
eastern-cyan•2y ago
If you're using the Response AI step you can change the model @Radiant raindrop - navigate to prompt settings, likely you will have to adjust/limit the prompt, depending on the model + prompt you may get hallucinations. If querying from the KB you can also choose which model. If this is a chunking error, we can look deeper into it, please email support@voiceflow.com with subject line 'DISCORD - KB Issue'
Please include the KB sources + vf file and any step(s) to reproduce. A video/screenshots will be helpful too!
cc: @Callum
Thanks! I’ll first try to change the AI model in the KB. Where do I do that btw?
optimistic-gold•2y ago
try using the query api too
along with what Braden and radian said
and you should modify your bots response instructions in the prompt
/system settings
So call the VF query API?
Will try to better it
optimistic-gold•2y ago
Yes
Thanks, I’ll try that