AI crawler scanning an AI-built website and finding less readable content than a human sees in the browser.

I Built Websites With AI. Then I Discovered AI Couldn’t Read Them

I built websites with AI that looked perfectly fine.

The navigation worked. The pages looked good. I could open them on my phone, click around and see exactly what I expected to see.

Then I built a tool to check whether AI systems could actually reach and read websites.

So naturally, I tested my own.

That is when I found a problem I didn’t even know I had.

Some of the sites I had built with AI builders were not nearly as readable to crawlers as they looked to me in a browser. That is a pretty wild sentence when you think about it: I used AI to build websites that AI could have trouble reading.

I didn’t build the tool to prove AI builders were bad

I want to make that clear because I still like AI builders. I have used them a lot. They can take an idea and turn it into something real ridiculously fast.

I built the Marketur AI Readability Check because I wanted a better way to see what AI crawlers could actually access and understand on a website.

The tool checks things like robots.txt rules, AI crawler access, server-readable content, internal pages, structured data and signals from services sitting in front of the website.

Once it worked, testing my own sites was obvious.

I expected a few warnings. I did not expect it to make me look differently at websites I had already considered finished.

What I saw in a browser wasn’t necessarily what a crawler got

This is the part that is easy to miss when you are building with AI.

A modern browser runs JavaScript and turns an application into the website you see. A crawler may fetch the underlying HTML and have a very different experience depending on how that crawler works and how the site was built.

Google has become very capable at rendering JavaScript, and its current documentation specifically says using JavaScript to load content does not inherently make a site harder for Google Search. But that does not mean every crawler on the internet behaves like Googlebot.

Google’s JavaScript SEO documentation is worth reading if you build modern web apps.

AI discovery adds another layer. OpenAI says that sites wanting content discoverable in ChatGPT search should allow OAI-SearchBot. That access question is separate from whether the useful content is actually present in what the crawler receives.

OpenAI’s publisher guidance explains the crawler-access side.

The page problem caught my attention

One thing I had already noticed with AI builders is that asking for a website with multiple pages does not always mean the builder creates the architecture I had in my head.

I might ask for Home, About, Pricing and Contact and get something that looks like a normal multi-page website. Underneath, though, it can be a JavaScript application using client-side routing, shared application shells or content that only becomes meaningful after the browser runs the app.

That does not automatically make the site bad. A properly built single-page application can absolutely have crawlable routes and strong SEO.

The problem is assuming the AI handled all of that because the navigation looks right.

I wrote more about that specific issue in Your AI Website Builder May Be Creating Pages Crawlers Can’t Read.

This is another reason I went back to WordPress

I have already written about why I went back to WordPress after spending heavily on AI builders.

This discovery added another reason.

With WordPress, when I create a normal page I understand what I am getting. It has its own URL. The server returns content for it. I can control the title, metadata, internal links, schema and the rest of the SEO setup.

That doesn’t make WordPress magically perfect. You can build a terrible WordPress site too.

What I like is that I am starting from an architecture I understand, then adding AI on top of it. That is a big part of what I am doing with Marketur.

I think builders should test this themselves

My own sites are not a scientific sample, and I am not going to pretend they prove every Lovable site or every AI-built site has this problem.

They proved something much more useful to me: I had websites with problems I did not know existed until I tested them.

That is why I think the right response is not to panic about AI builders. It is to test what you built.

Run your site through the free AI Readability Check. Look at what it can actually fetch. Check the internal pages. Check your robots rules. Check whether important content exists in the HTML a crawler receives.

If everything passes, great.

If it doesn’t, at least you found out before assuming AI systems could see the same website you see.

That is exactly how I found out.

Leave a Comment

Your email address will not be published. Required fields are marked *

Scroll to Top

More

☀️ Light Mode
📰 Latest Posts
Loading...
📊 Community Stats
Loading...
🟢 Online Now
Loading...
👋 New Members
Loading...
👥 Popular Groups
Loading...