Our website vanished from Google Search
In early July 2024 our company's website content vanished from Google. It was pretty scary — none of our clients could find us or our products, website's request form submissions cratered.
When I tried to search any product names, no relevant results would appear. I searched "site:takeprofittech.com", and only a few "technical" irrelevant pages with broken content were found (which, to be fair, should have been hidden by our SEO settings, but the pages were so irrelevant, we didn't even remember they existed, and they had never popped up anywhere before). Here's the screenshot of how it looked like:
When searching our brand name, our main page was technically still showing, but Google thought it had no content, so the search result looked awful and broken.
Our Google Search Console started spamming us with errors affecting hundreds of pages like:
- "Page indexed without content"
- "Crawled - currently not indexed"
- "Page with redirect"
(Even writing down these error messages triggers my flashbacks)
I remembered that we had encountered a very similar issue with our website's Google visibility almost exactly one year ago — if I remember correctly, it started in August 2023 and lasted up until October 2023 until it fixed itself. Of course, during those two months we'd tried so many different things to alleviate the problem: re-did all keywords and meta tags, re-wrote texts, wrote more text, added more pictures, added videos (we heard Google liked embedded videos?), optimized sitemaps, improved page and text colors to have higher contrast, installed WordPress caching plugins to improve page load speed — nothing helped.
We were re-submitting each major page for re-indexing by Google, which helped for a couple of days, and then inevitably each page would vanish from index again, with the same status: "Crawled - currently not indexed". We tried consulting all our web dev friends and acquaintances. We hired SEO experts to guide us through these issues — nobody actually knew what was happening.
Googling for symptoms didn't give us any hint at all. "Obviously the search robot says that there's not enough content to index, just add content" — but no amount of text or pics would change search engine's opinion about our pages being without content. We felt helpless.
When the issue vanished as suddenly as it appeared, it was a relief, but I feared the issue would recur, and it did!
But this time, my colleague Kate found a solution on the internet: turning off Cloudflare's caching for our domain (going from orange cloud "Proxied" to gray cloud "DNS only"):
— and it fixed it!
Why? I don't know, maybe Cloudflare was blocking or somehow scaring the Google's web crawler. Why was it not affecting the website in the months prior? No idea, maybe some kind of software update for Cloudflare's proxies or Googlebot's logic triggered different behavior. Would it have helped one year ago? I don't know either. Honestly, I wouldn't even think (and I didn't) that Cloudflare could be the source of such problems — so many websites use it, surely many other people would find themselves in the same situation?
Remembering how helpless we were, I thought it's a good idea to write this post, so that other people could maybe find it and stop their business from dying.