conversation flow + memory without using large amount of tokens

hi guys, im really frustrated as i cant seem to find a good solution for what i want to do. i want to build a chatbot that can answer questions based on the urls + documents i provide in the knowledge base ... the chatbot should be intelligent though, it should be able to know the context in order to provide a natural conversation flow and to understand follow up questions ... so if the user wants to know "hey what is the requirement for studying this course x" the chatbot answers ... "and until when can i sign up" ... the chatbot should still know that the user is talking about course x ... "and whats the due date for applying for course y" the chatbot replys "ah and whats the requirements" the chatbot now should know the user is talking about course y ... guys idk is this too complicated??? is there no way to have a smart conversation without consuming a crazy amount of tokens?

ordinary-sapphire•2/13/24, 9:34 AM

When using memory, you'll always end up using lots of tokens since the whole conversation up until now is added to the request to the AI.

ordinary-sapphire•2/13/24, 9:34 AM

You could build something simpler by yourself and store last question and response in a variable and add those to the prompt yourself.

ordinary-sapphire•2/13/24, 9:35 AM

This way, you don't have the full conversation sent over and use less tokens.

ordinary-sapphire•2/13/24, 9:35 AM

See https://discord.com/channels/1079548823610871889/1206612019818790982

recent-tealOP•2/13/24, 9:45 AM

thank you, i will take a look at that, also: is it maybe possible to store two key informations like which course the user is asking about in a variable and to update it everytime the user is asking about a different course? so that the chatbot only provides every info to every follow up information only in regards to that specific course that the user last mentioned ... ? that way it doesnt need the whole context/memory but only one variable ...

ordinary-sapphire•2/13/24, 9:52 AM

Yes, of course that's possible. You could use the Set AI step to fill the variable.

recent-tealOP•2/13/24, 9:58 AM

does it use a lot of tokens? how would i do that??? sometimes i think i understand voiceflow, then i feel like i dont

ordinary-sapphire•2/13/24, 10:10 AM

It depends. You could also extract the current course out of the text with JavaScript (e.g. search with a regex); this wouldn't use any tokens.

recent-tealOP•2/13/24, 10:18 AM

any example on how that would work?

ordinary-sapphire•2/13/24, 11:01 AM

You mean for extracting the course name via JavaScript?

recent-tealOP•2/13/24, 11:02 AM

yes

ordinary-sapphire•2/13/24, 11:03 AM

Depends on how your responses look like, I can't give you a code snippet that's working out of the box.

ill-bronze•2/13/24, 7:47 PM

@mansen why don't you take the course name in a variable and add it into the prompt saying.
You are having a conversation with {name} regarding course {x} .
Answer the users query accordingly.

Rrecent-teal thank you, i will take a look at that, also: is it maybe possible to store two k...

ill-bronze•2/13/24, 7:53 PM

I suggest you make the glow in a way where. You user predefine the course name/category they're looking for. And then give them the option to change the course name.

ill-bronze•2/13/24, 7:55 PM

I have a few ideas on how you might further change the flow to improve the experience. I'll DM you @mansen