Links to crawled websites

When Apollo uses a website as source for its response I have tried asking for a link to the webpage. Apollo then sometimes responds with a URL that doesn’t exist. To prevent this behavior I have added “- Provide URLs only from the given knowledge. Do not modify URLs.” to the base prompt but it doesn’t seem to work.

It would be neat if Apollo could understand which website it got the answer from and be able to direct users there for more information when needed. And it would be great if Apollo would not make up its own URLs to pages that doesn’t exist :slight_smile:

I have seen some posts here about creating separate documents with links, but would prefer to just crawl our website for all info and that the bot just keeps track of it all. Prefer to have one source of truth…

Hi Tom,

Thank you for your feedback.

If there are no URLs defined in given knoweldge, AI will sometimes try to reverse engineer them using the Page Titles, resulting in odd looking brackets or 404 links.
If URLs are defined in given knowledge, you can make sure they are used correctly with this Base Prompt addition:

If given knowledge includes a URL, include it in your reply with empty space ‘’ ‘’ before ‘‘https:’’ for example ‘’ https://www.example.com ‘’

We have improvements planned for Website Ingestion to help with directing users to URLs, but a workaround for now would be simply uploading your sitemap as CSV table that you AI can use in combination with Website Ingest to direct users where needed.
You can find an article I posted about this here:

If want me to look at your individual setup and provide insights for optimization or solve any existing problems you can reach me at ralfs@tawk.to

1 Like

Thanks for your response. Looking forward to the planned improvements to website ingestion :slight_smile: Any idea on an approx. timeline for that? Before end of year?

I’ll try the CSV solution for now. Thanks.