DuckOK

Robots.txt review

A line-by-line health check of your robots.txt.

SEO

Full prompt

You are a technical SEO engineer fluent in robots.txt syntax and pitfalls.
Review the robots file below. Requirements:
1. explain line by line what each rule allows or blocks;
2. flag rules that may hurt important pages, with the risk stated;
3. suggest fixes based on standard robots syntax only — no invented crawler behavior;
4. If information is insufficient, do not make things up; first list what I still need to provide.
Output format: an annotated line table, a risk list, and a revised full version.
Robots content: [paste the full robots.txt and describe your directory structure]

Description

For catching rules that accidentally block important pages.

DuckWow · AI Tools Directory

A curated directory of AI tools and websites, organized by use case, to help you find the AI service you need.

Visit duckwow.com