fair-rose•17mo ago
Chunk limit setting in knowledge Base
Hi everyone,
i´ve just wanted to ask what the chunk limit setting in the knowledge base mean and what it is used for?
Thanks and best regards,
Marcel
1 Reply
The
Chunk Limit setting in the Knowledge Base (KB) of Voiceflow is a parameter that controls the number of 'chunks' or pieces of text that are retrieved from the vector database to synthesize the response to a user's question.
When a question is asked, the KB's retriever service turns the question into a vector and searches the vector database for the most similar chunks based on a similarity score. The Chunk Limit defines how many of these similar chunks are returned in descending order by similarity score.
The similarity score is determined by semantic search, which goes beyond keyword matching and refers to contextual similarity in meaning between words and phrases. This allows the question to be semantically compared to the KB document chunks that exist. The 'closest' vectors to the question are those with the highest similarity. The retriever will return a number of chunks based on this vector proximity.
The Chunk Limit setting aims to provide flexibility to increase the accuracy of responses in line with certain use cases. In theory, the more chunks retrieved, the more accurate the response, and the more tokens consumed. However, the 'accuracy' tied to chunks is strongly associated with how the KB data sources are curated.
If the KB data sources are curated so that topics are grouped together, this should be more than enough to accurately answer the question. However, if information is scattered throughout many different KB data sources, then likely more chunks of smaller size will increase the accuracy of the response.
You can control the max chunk size of your data sources with the Upload/Replace KB doc APIs, using the query parameter: maxChunkSize.