Security and SEO: headers, WAF and crawl traps

Distinguish verified crawler blocks from spoofed traffic, then test a narrow WAF correction safely.