[AI Assist] Changes to Website Ingestion crawl limitations

Hello Community,

Hope all is well.

We’re rolling out an update to our Website Ingestion Crawler:

Previously, every website you added had an ingestion limit of 500 pages. Now, this limit applies uniformly across all the websites you add, based on your subscription plan:

  • Free: 100 pages
  • Growth: 500 pages
  • Business: 2,000 pages
  • Enterprise: 10,000 pages

Frequently Asked Questions:

  • Q: Will this affect my current AI setup with websites I’ve already ingested?
    A: No, ingestions completed before today’s update will remain unaffected.
  • Q: I have numerous unique product pages, but my visitor traffic doesn’t justify an upgraded subscription. How can I ensure the AI recognizes my products?
    A: While Website Ingestion helps in swiftly gathering company data, we understand many businesses have numerous product pages. A practical approach is to upload a table as CSV detailing your products, with columns such as Product Name, Description, Price, URL, etc. You can find a quick best practices guide for this here.
  • Q: Can I change my subscription midway through the billing cycle if I need more pages ingested?
    A: You can upgrade your subscription plan to a higher tier at any point from the AI Assist settings by clicking the ‘‘Upgrade’’ button. Once the new plan is set, your monthly message allocation and web crawler ingestion limitations will be increased.
    Q: I tried Website Ingestion Data Source some time ago and AI kept providing the wrong info, should I give this another shot?
    A: Last week we shipped an update to web crawlers’ ability to filter out unneeded information such as selection menus, meta tags, and navigation buttons. This overall improves the consistency and precision of Apollo AI answers when searching for context within the website ingestions. All newly ingested websites will benefit from the improved filter.

We appreciate your understanding as we implement these changes. Should you have any more questions or concerns, please don’t hesitate to reach out.