Scraping limitations
Is there a way to limit the URL scraping to 30 pages per minute? And also how much is one credit worth? đ
1 Reply
URL scraping speed
Voiceflowâs built-in crawler doesnât currently expose any knobs for throttling the request rate (e.g. â30 pages per minuteâ). The import runs at the default speed managed by Voiceflow. The only timing control you can set today is the refresh rate (never / daily / weekly / monthly) that determines how often a source is re-synced â but the per-minute crawl rate itself isnât user-configurable.
If you need tighter control you can: ⢠Pre-crawl the site yourself and upload the content as files or via the Knowledge Base API. ⢠Split the sitemap/URL list into smaller batches and import them sequentially. Credit value On self-serve Business (monthly) plans one Voiceflow credit = $0.005 USD (½ cent). Credits are the common currency for all usage: ⢠1 credit per chat message sent by the user ⢠10 credits per minute of voice call time ⢠LLM / TTS / STT usage is billed in fractional credits based on the vendorâs rate card You can find the full pricing table and calculator here: https://docs.voiceflow.com/docs/credits-pricing-table#/
If you need tighter control you can: ⢠Pre-crawl the site yourself and upload the content as files or via the Knowledge Base API. ⢠Split the sitemap/URL list into smaller batches and import them sequentially. Credit value On self-serve Business (monthly) plans one Voiceflow credit = $0.005 USD (½ cent). Credits are the common currency for all usage: ⢠1 credit per chat message sent by the user ⢠10 credits per minute of voice call time ⢠LLM / TTS / STT usage is billed in fractional credits based on the vendorâs rate card You can find the full pricing table and calculator here: https://docs.voiceflow.com/docs/credits-pricing-table#/