Who can crawl your site?
See what robots.txt, robots meta, and X-Robots-Tag say about one URL.
Enter a domain or an exact page URL
- Free
- No account
- No public lookup history
Checking
The core tool stays free. There is no premium plan.
Accounts, cards, onboarding tours, or upgrade nags.
A check never becomes a public listing.
The indie shelf
More small tools, ready now.
This tool stays focused on one job. Other tools by turushan are ready for the next small task.
Plain talk
A tiny tool should be easy to trust.
NO ASTERISKS HIDING DOWN HEREWhat does free mean here?
No account, card, premium plan, or upgrade nag. The core tool stays free.
Where does IndieCrawl's answer come from?
The answer follows The checked website and RFC 9309. The source stays named and linked beside the result it produced.
What happens to what I check?
Nothing you check becomes public inventory. The page uses cookieless analytics and stores no lookup history.