Skip to content
Searchpedia SEO field notes Callum Bennett Callum

Site ops

SEO Programming

I underestimated how much programming choices like JavaScript rendering and server configuration affect crawlability, and now I check every technical layer before blaming content.

Beginner5 min readUpdated 2026-07-27Notes by Callum Bennett

Start here

  • Audit your robots.txt and noindex tags before every launch to prevent accidental blocks.
  • Use semantic HTML with proper heading hierarchy so search engines understand page structure.
  • Verify that critical content is accessible without JavaScript and that Google can render your dynamic pages.
  • Submit an XML sitemap and use internal links to guide crawlers to important pages.
  • Check Core Web Vitals and mobile rendering using real devices, not just desktop Lighthouse.

Plain-English take

I think of SEO programming as building a house with clear signage. You want Googlebot (the delivery person) to find the front door easily, walk through without tripping, and know what each room is. That means clear paths via internal links and [sitemaps](/sitemap/), open doors with no accidental noindex or robots.txt blocks, and proper labels using semantic HTML and structured data. It is not about writing "SEO-friendly code" as a separate task; it is about making sure the technical decisions you already make do not inadvertently hide content.

For example, a site I audited had all product descriptions loaded via JavaScript after the initial HTML. Google could see the product titles but not the descriptions—core content invisible. Fixing that by moving the descriptions into the initial HTML improved organic traffic by 40% within two months.

Another common issue: using <div> for everything instead of <h1>, <h2>, <nav>. That does not break anything, but it leaves ranking signals on the table. Semantic HTML helps search engines parse your content hierarchy and may influence featured snippets.

The plain English takeaway: make sure your code does not get between the user and the search engine.

When it actually matters

This is not about every tweak; it is about specific moments where programming decisions have outsized impact. Launching a new site or redesign is the best time to get it right. The framework you choose—whether static, server-rendered, or client-rendered—affects crawlability, [performance optimisation](/performance-optimization/) potential, and indexation. I once inherited a React single-page app with all content rendered client-side. Google managed to index some pages, but the crawl budget was wasted on blank shells. Switching to server-side rendering for the critical content fixed that.

Another high-impact moment is adding dynamic content, like user-generated reviews or real-time availability. If that content loads via AJAX after interaction, search engines may never see it. Use a sitemap to alert Google to new or updated content, and test rendering with the URL Inspection tool. Also, when you reorganise your [website structure](/website-structure/), ensure internal links update and redirects are set properly. A single broken [canonical tag](/canonical-tag/) during migration can dilute page equity for months. Finally, if you implement [structured data](/schema-markup/), double-check that the markup matches what is visible on the page. Google penalises misleading structured data, even if accidental.

One more example: when I switched an e-commerce site from incremental pagination to infinite scroll with JavaScript, I forgot to implement pushState or update the URL. Google could only see the first page of products. That cost us thousands in monthly organic revenue until I added proper [pagination](/pagination/) signals.

What I got wrong

I made mistakes that cost me traffic and taught me the hard way. First, I accidentally blocked an entire staging site from Google by leaving Disallow: / in [robots.txt](/robots-txt/) after a migration. The traffic drop took three weeks to notice because the site still worked for users. Check that file religiously.

Second, I assumed Googlebot could render JavaScript like a modern browser. I built a single-page app where product descriptions loaded via AJAX after user interaction. Google never saw them. I had to rebuild with server-side rendering for critical content. Now I always test with the URL Inspection tool.

Third, I ignored mobile rendering. I optimised for desktop Core Web Vitals, but my mobile site was a subdomain with different content. Google switched to mobile-first indexing, and my rankings tanked. Merge the mobile site and test everything on a real phone.

Finally, I spent days crafting schema markup for events without verifying that the event details were visible on the page. Google ignored it because the content was not there. Verify structured data against rendered HTML.

Next step

Quick answers

How does JavaScript affect SEO?

JavaScript can hide critical content if search engines cannot render it. Ensure important text and links are in the initial HTML or use server-side rendering. Test with Google's URL Inspection tool to verify rendered content. Avoid blocking JavaScript in robots.txt.

Should I avoid using JavaScript frameworks for SEO?

Not necessarily. Modern search engines can render JavaScript, but it adds complexity. If you use a client-side framework, implement dynamic rendering or pre-rendering for crawlers. Static sites or server-rendered apps are safer for critical pages. Choose based on your team's ability to maintain crawlability.

What is the most common SEO programming mistake?

Accidentally blocking pages via robots.txt or noindex tags. I have seen it happen during staging migrations, site launches, and even in production after updates. Always audit these directives before going live. Use the robots.txt tester in Google Search Console to confirm.

Sources

Primary documentation is linked directly. Anything commercial is marked nofollow.

Notes from Callum Bennett.