You found us in your logs.
AwesomeTechStackBot fetches public pages to work out what a website is built with and how current it is. Here is exactly what it does, and how to stop it.
How to block it
Name the token in your robots.txt. We read it before every crawl and we follow it - there is no appeal process and no exception list.
User-agent: AwesomeTechStackBot Disallow: /
To allow it again, remove that block. A rule for * that is not addressed to us does not silence us, which is deliberate and worth stating plainly: it is how a crawler that emulates a real visit behaves. If you want us out, name us - the line above is all it takes, and it works within a day.
What it requests
- Public pages, the way a browser asks for themUsually one page, occasionally a handful. We render it to see the same thing a visitor would - scripts included - because half of what we grade only exists after they run.
- The files you publish for machines
robots.txt, your sitemap,llms.txtand the.well-knownentries - the ones that exist to be read. - Your TLS certificateRead from the handshake, to see when it expires.
What it never does
- No forms, no logins, no purchasesIt reads. It never submits anything, never signs in, and never follows a link that would change something on your site.
- No content harvestingWe record which technologies and versions a page reveals, not your text. Nothing from your pages is republished or resold.
- No hammeringAn audit is a small number of requests. Monitored sites are re-checked on a schedule - daily at most, and only where somebody asked for it.
Why it looks like a browser
The agent string carries a normal browser identity and our name, the same shape Googlebot uses. The browser half is not a disguise: we grade what a real visitor's browser receives, so we have to ask for pages the way one does. The name is there so you can see it is us, block it in one line, and read this page.
Anything you see claiming to be this crawler while ignoring the rule above is not us. If that happens, tell us - the address is below and a person reads it.
Reach a person
Write to hello@awesometechstack.com with the domain and roughly when you saw the requests. Rate limits, a full stop, a question about a log line - all fine. Contact form