About WebHostPuneBot

WebHostPuneBot is the page checker behind WEB HOST PUNE Tools, the backlink analysis tool run by WEB HOST PUNE.

If you see it in your logs, one of our users asked us to check a specific page on your site, usually to confirm that a link they earned is still there and still counts for SEO.

User agent

WebHostPuneBot/1.0 (+https://tools.webhostpune.com/bot)

What it does

• Fetches only the exact page URLs our users submit. It does not follow links or crawl your site.
• Makes one request per page per check, following redirects. A failed request is retried at most 2 more times.
• Sends at most 2 requests to your site at the same time.
• Backs off when you answer 429 Too Many Requests, and honours your Retry-After header.
• Stops reading any page larger than 5 MB.
• Reads the HTML and HTTP headers only: links, canonical tag, robots meta tag and page title.

What it does not do

• It does not log in, submit forms, run JavaScript or download images, scripts and other assets.
• It does not copy or republish your content.

Opting out

Support for robots.txt rules is in development. Until it is released, email us with your domain and we will stop checking it: info@webhostpune.com.

Contact

Questions about the crawler: info@webhostpune.com