Feature docs · Website security · Module: robots
robots.txt is a Guardbee scan module under website security. Fetches and analyzes robots.txt.
What does it check?
Fetches and analyzes robots.txt. Results appear in the dashboard and scan detail as score, findings, and recommended actions.
Business impact
A bad robots.txt can disclose sensitive paths or hurt SEO.
Guardbee presents risk, impact, and priority — so product, legal, and engineering share one screen.
How to run it
- Create a Guardbee account (25 credits for new accounts)
- Add a brand / URL
- Select robots.txt under Website security
- Run the scan; optionally explain findings with BeeAI
Related docs
Frequently asked questions
Is robots.txt required?
No. Pick modules as needed; you only spend credits on what you run.
Is scanning destructive?
No. Guardbee uses controlled, read-only, limited requests.
Does it work with BeeAI?
Yes. After a scan, BeeAI can summarize findings in business language.