Hello, we noticed while running our SEO audit a lot of 403 errors from this page
https://embed.tawk.to/69ef498bd0fd481c759f8f34/1jn7beia0
While the same returns a 200 from regular browser.
Question do you block traffic for this types of crawlers? Per my investigation that seems to be the case.
I think this is their list of IPS
Hi Felipe,
Thanks for reaching out.
You are correct. The embed.tawk.to endpoint that serves the chat widget script blocks bots and crawlers on purpose, including the Ahrefs Site Audit bot. This keeps automated traffic out of your visitor monitoring and reports, so the visitors you see in your dashboard are real people. Regular visitors are not affected, which is why the same URL returns a 200 in your browser.
The widget script only loads the chat widget. It does not add indexable content to your pages. So the 403 in the audit should not affect your rankings. In Ahrefs you can mute this issue in Site Audit issue settings so it doesn’t count against your health score.
Let us know if you have any questions.