security.txt & robots.txt Audit

What Is the security.txt & robots.txt Audit?

RFC 9116 defines /.well-known/security.txt as the standard way for organizations to publish vulnerability disclosure policies. Its absence makes it harder for security researchers to report issues responsibly.

robots.txt is intended for search engine crawlers, but attackers use it as a reconnaissance map — Disallow: /admin confirms an admin panel exists.

Common Issues

Missing security.txt

Without a security.txt file, researchers who find vulnerabilities have no standardized way to contact you. Many will give up, and the vulnerability remains unpatched.

Expired security.txt

Contact: mailto:security@example.com
Expires: 2024-01-01T00:00:00z

An expired Expires field signals that the security contact may be abandoned.

robots.txt as an Attack Map

User-agent: *
Disallow: /admin
Disallow: /api/internal
Disallow: /debug
Disallow: /.env
Disallow: /phpmyadmin

Every Disallow line is a confirmed endpoint for attackers to probe.

How to Fix

Create a compliant security.txt:

Contact: mailto:security@yourdomain.com
Expires: 2026-12-31T23:59:59z
Preferred-Languages: en
Encryption: https://yourdomain.com/.well-known/pgp-key.asc

Protect sensitive paths with authentication instead of hiding them in robots.txt.

Keep Disallow rules minimal — only exclude paths that would waste crawler bandwidth, not sensitive ones.

What VibeWShield Detects

VibeWShield's security.txt scanner checks RFC 9116 compliance (required Contact field, valid Expires date, proper format) and analyzes robots.txt for sensitive path patterns like /admin, /api/internal, /debug, /.env, /phpmyadmin, and 15+ other signatures.