SEO Web Development

The first thing I do on any SEO web development project is run a crawl audit. I use Screaming Frog (free up to 500 URLs) to map the site structure, identify orphan pages, broken links, and missing metadata. That gives me a baseline of what's broken before I write a line of code. Then I fix internal linking so that important pages are reachable via <a> elements from the homepage or top-level pages. I ensure that every page has a descriptive title tag and meta description; I've seen sites where half the pages use default 'Untitled' — that's a ranking leak. I also set up a [sitemap](/sitemap/) to help Google discover URLs, especially on larger or more complex sites.

Next, I tackle performance: reducing render-blocking resources, minifying CSS and JavaScript, compressing images, and using a CDN. For a recent client, I cut load time from 4.2 seconds to 2.1 seconds by deferring JavaScript and serving WebP images. That directly improved their [Core Web Vitals](/core-web-vitals/) scores, and keyword rankings followed. I also implement responsive, mobile-first design because mobile usability is a ranking factor — refer to [mobile SEO](/mobile-seo/) for more on that. If the site uses JavaScript heavily, I check that content and links are in the initial HTML or that server-side rendering is in place; see [JavaScript SEO](/javascript-seo/) for the gotchas there. I reserve [robots.txt](/robots-txt/) for crawl budget management, not blocking — I use noindex for pages I want out of the index. Finally, I consolidate duplicate URLs with [canonical tags](/canonical-tag/) so search engines know which version is preferred.

That's the order I follow: crawl, link structure, metadata, sitemap, performance, mobile, JavaScript rendering, robots.txt, canonicals. It's a checklist that never fails me, even on large e-commerce sites with thousands of products.

Blocking important pages with robots.txt. If you disallow a URL, Google cannot see its noindex tag, so the page may still get indexed via other signals. I've seen clients block their blog section and wonder why old posts still appear in search results. Use robots.txt only for crawl budget control, not to hide content. The correct approach is noindex or authentication.

Relying on JavaScript-only rendering. If key content or links appear only after JavaScript executes, Google may not see them. I once audited a React SPA where all product links were injected via JS — Google crawled only the homepage. Use server-side rendering or dynamic rendering for critical content. Edge case: if your site has infinite scroll or load-more buttons, ensure each page has a unique URL and proper [pagination](/pagination/) links.

Overlooking mobile behaviour. Desktop search and mobile behaviour differ; I use Google's Mobile-Friendly Test and check actual device rendering. A site that looks fine on a 27-inch monitor might be unusable on a phone. I once had to redo an entire navigation after ignoring mobile tap targets.

Messing up canonical tags. Pointing all similar pages to one canonical can cause loss of indexation for legitimate variants. Use self-referencing canonicals or proper cross-domain ones if syndicating content. I've also seen rel="canonical" pointing to non-existent URLs — that's a silent error.

Slow server response time. Even with front-end optimisation, a TTFB over 600 milliseconds hurts rankings. I use a CDN, and caching layers, and make sure the backend isn't bogged down by database queries. Performance optimisation is not just about the browser.

Finally, ignoring duplicate content across subdomains or protocol variations. I set up 301 redirects for all www vs non-www, HTTP vs HTTPS, and trailing slash variants. The canonical URL tag alone isn't enough if you have four versions of the same page live.

Early on, I thought robots.txt was the catch-all for keeping pages out of search results. I blocked a whole section of a client's site, only to realise Google couldn't see the noindex tags I'd placed there. The pages stayed indexed for weeks. Now I always use noindex for that purpose and reserve robots.txt for limiting crawl budget on non-essential directories. That was a painful lesson.

Another mistake: I ignored mobile-first indexing for too long. In 2018, I redesigned a site with a desktop-first approach and used responsive design as an afterthought. That site's rankings dropped after Google switched to mobile-first indexing. I lost a client because of it. Now I design mobile-first from the wireframe stage.

I also underestimated the impact of render-blocking resources. I once shipped a site with five CSS files and ten JS files loaded synchronously, causing a 3-second render delay. After minifying and deferring, the page loaded in under 1.5 seconds and keyword rankings recovered within weeks. That experience taught me to optimise the website early in development, not as a post-launch polish.

Finally, I used to think that as long as content was on the page, Google would find it. I didn't ensure semantic HTML or that content was in the DOM — I relied on CSS background images and pseudo-elements. That content was invisible to Google. Now I check the rendered DOM in Search Console's URL Inspection tool before I call anything done. Semantic elements like <article>, <nav>, and <main> also help structure the site clearly — something I cover in website structure planning.