Canonical URLs, robots directives and index controls

Compare crawl access, noindex directives and canonical candidates on equivalent pages before choosing a fix.