# Allow all search engines to crawl the site except /cgi-bin/ User-agent: * Disallow: /cgi-bin/ # Block specific bad bots User-agent: BadBot Disallow: / # Block other known spam bots (Optional) User-agent: MJ12bot Disallow: / User-agent: AhrefsBot Disallow: / User-agent: SemrushBot Disallow: / # Allow Googlebot to fully crawl everything User-agent: Googlebot Allow: / # Specify your sitemap location Sitemap: https://www.sattaadda.com/sitemap.xml