AI Response's "Not Found Path" not working

So, I am using an AI response step with the knowledge base as the source. I have toggled the "Not Found Path" button on but it's not triggering consistently. Sometimes, when the KB AI step can't find any relevant answer, it takes the "Not Found Path," and other times (mostly), it returns a "Sorry, I don't know the answer to that question."

Does anyone know how to fix this? Is there any workaround for this?