Can an AI actually read your site?
In 2024 a lot of people blocked “the AI bots”. Since then the crawlers have split in two: one set collects content to train models, a different set builds the index behind the answers ChatGPT, Claude and Perplexity give. Most sites still have the setting they chose before that split, which often means they are invisible to AI search while still being trained on.
This reads your robots.txt properly, tells you which of those two things you are actually doing, and checks whether your content is even in the HTML a crawler receives. Then it writes the rules for whichever outcome you want.
Can an AI actually read your site?
Three different questions, usually run together. Are the crawlers that put you in AI answers allowed in - as opposed to the ones that only train on you? Is your content in the HTML a crawler receives, or does it appear after JavaScript runs? And is there enough structure for a model to say what your business is without guessing?