fair-rose•3mo ago
Broken knowledge base
My project's knowledge base is broken. Whatever I enter in the search, even if it's just a set of letters, it shows a chunk score of 80+%. Not only does it give such high percentages, it also searches by one link and does not take into account my text files that are in the knowledge base.
When I delete the only URL, it does not find any chunks at all when searching by the rest of the data.
For clarity, I created another project with the same knowledge base and there the same query, consisting of a set of letters, gives 15-30% relevance, not 80+.
Because of this error, my knowledge base search cannot find normal chunks to answer the question, everything broke for me because of this.
I had the same problem 4 months ago in another project. Then the knowledge base only had URLs, and each search also gave 80+ relevance. Changing the model, resetting the settings and changing the model prompt did not help me. The only thing that helped me was resynchronizing all the data with the YURL.
But in this project I can't do that, because I mostly have files, not links. And it's very strange that this is happening, I would like you to pay attention, since you apparently don't know about it.
KB search behaves the same way.
I would not want to go and delete my entire knowledge base and add it again now, I'm sure that this problem will also happen in the future, it would be better if you fixed it, or told me what the problem is and how I can solve it in the best way.
17 Replies
It sounds like the KB is behaving oddly—returning very high chunk scores (80%+) when your knowledge base has a URL and then finding nothing when that URL is removed. Here are a few ideas you might try:
• Verify that your file uploads are being properly processed. If your text files have very little content or aren’t formatted in a way that encourages proper chunking, the KB might not be breaking them up into distinct, searchable chunks.
• Check your KB Search step settings, particularly the “Minimum chunk score.” If the threshold is too high or too low for file-based content, it might be skewing the results. Adjusting this slider can sometimes help the search better distinguish between less relevant (low-scoring) and more relevant chunks.
• If your project mixes URL data with file data, note that extraction methods can vary. URL content might be processed differently and end up with high scores even when the query is pretty generic, whereas files may need more detailed content to yield meaningful scores. If possible, try to separate (or filter) the sources so that you can target just the file-based chunks.
• Although you’ve already tried model changes and prompt resets, a partial re-sync (updating or re-uploading a few files) might trigger a fresh chunking process. This might help without having to delete your entire KB.
You can read more about how the KB processes data and set up your KB Search step in our documentation here: Voiceflow Docs.
I hope these suggestions help narrow down the issue. Feel free to share more details if you continue to see unexpected behavior!
fair-roseOP•3mo ago
fair-roseOP•3mo ago
@Braden (Voiceflow CEO) Please pay attention. This is not the first time this problem has happened
fair-roseOP•3mo ago
Important point, I just duplicated my "killed project" and inserted all the knowledge base files that were there.
Also created a clean project
I made 3 identical queries to the knowledge base in each project. The query itself is a random set of letters and numbers.
Here is how these projects behaved:
1st Screen - Project with broken Knowledge Base - Showed 0 chunks
2nd Screen - Duplicate of a broken project - Shows 80% relevance
3rd Screen - New project from scratch - Works as expected, showed only 19-33% relevance
The same default settings were used everywhere, I didn't change anything. LLM - GPT 4.1.
@Moderator @Braden (Voiceflow CEO)
You are looking at this all wrong. The answers are all the same. No answer found. The chunks being returned aren't that big a deal.
garbage in, then you will get garbage out.
As far as the chunk score, that doesn't matter either.
@kanzeitai
fair-roseOP•3mo ago
Sorry, but this answer does not address the actual technical problem.
I have tested with the exact same KB files in three projects:
In the broken project, no chunks are ever returned for any query, not even for pure garbage input. Chunk count is always zero, no matter the input.
In a duplicate project where I manually uploaded the same files, I always get very high chunk scores (80%+) for any query, even for random letters/numbers — which is clearly not normal. These are the same files.
In a new project, using the exact same files, chunk scores and search results behave correctly (19-33% for random input, normal for real queries).
This is not about “garbage in, garbage out”. The problem is that the search/index system itself is broken in some projects, regardless of the KB content.
If this was about “bad input,” then the new project with the same files would show the same symptoms, but it doesn’t.
The chunk score being always high is a clear symptom of a corrupted or broken project state.
The fact that the original project can’t return any chunks at all, ever, is a system error, not a KB data problem.
Please escalate this to technical support or an engineer who can look at how the search index or KB state is handled per project. The problem is not with my files, but with the way Voiceflow handles the KB/index inside the project after certain operations or errors.
If necessary, I will provide more information.
@W. Williams (SFT)
I personally have never had an issue with any KB within our account. Maybe someone over at VF can look at your projects @Braden (Voiceflow CEO) @NiKo | Voiceflow @Support
@kanzeitai Can you share a couple of your KB files, so I can try it in my account?
The KB feature evolves regularly, that might be why you have different results with older projects and new ones (for new ones, your KB docs have been handled with new parsing and chunking logic) with more recent uploads. Also, as @W. Williams (SFT) said, it will also depend on what kind of sources you upload. In your screenshot, I can see chunks with only URLs for example. Lastly, unless you use table format upload, one-word or keyword-type searches will not yield great results; use a proper query instead.
fair-roseOP•3mo ago
Sure, here are these 3 files. Let's check then, ask a question in the knowledge base and send me what your chunk score is
fair-roseOP•3mo ago
Show me what question you asked, I will ask the same question in my project in the knowledge base and I will show you that for some reason I will have a very high percentage of chunk score, several times higher than yours with the same settings
@Braden (Voiceflow CEO) @NiKo | Voiceflow @Support Definatley something weird going on here. The KB returned VF docs at a very hi chunk score.
Here are the translated KB docs from above and the questions I used for testing.
fair-roseOP•3mo ago
Ok, look how it works for me.
1 tab - broken project
2 tab - copy of this broken project
3 new project with the same knowledge base
fair-roseOP•3mo ago
In the second tab, the thing is that no matter what I enter in the request, even if it is complete nonsense, my chunk score will always be 70+
But in the 3rd tab, where the new project is given 20-30 when I type nonsense
I want to say that because of this my agent can't find the right answers, because each chunk has a relevance of 70+%, it stops seeing the right ones
Thanks for the context @kanzeitai I've pinged the team, they might join this thread with some more questions for you.
fair-roseOP•3mo ago
HI Niko, I just wanted to know if there is any news on this issue?
I can of course easily create a new project and re-upload the knowledge base there and everything will work fine, but if suddenly such a problem happens in the future, then it will look unprofessional to my clients