Web Dev SEO

Start with crawlability. I begin every project by verifying that each page intended for indexing has at least one crawlable <a> element pointing to it. For single-page apps, that means moving from hash-based routing to real URLs and using the History API. My go-to test: open the page without JavaScript — if the navigation is missing, Googlebot sees the same shell. Next, I ensure semantic HTML is used consistently. Wrapping content in <div>s is fine for layout, but search engines rely on heading tags, <nav>, and <main> to parse structure. I also set up an XML sitemap immediately, submitted via [Sitemap](/sitemap/) guidance, and reference it in robots.txt. After every deploy, I resubmit the sitemap in Search Console. I see too many dev teams treat the sitemap as a one-off task. I also add a self-referencing [canonical tag](/canonical-tag/) on every page, even when duplicates are not obvious. This prevents accidental duplicate issues from UTM parameters or HTTP/HTTPS variations. For JavaScript-heavy sites, I follow the principles outlined in the [JavaScript SEO](/javascript-seo/) guide: each content state gets its own URL, and I use server-side rendering or prerendering for critical landing pages. I also incorporate structured data during the build phase. Adding Schema markup for articles or products is easier when templates are flexible. I map schema properties to CMS fields so developers do not have to tag each page manually. One example: on an e-commerce site, I had the team embed Product schema in the initial HTML, not as a JavaScript injection. Google consistently picked up that data. I keep an eye on [duplicate content](/duplicate-content/) issues from pagination, session IDs, and printer-friendly views. Each variant gets a canonical or parameter handling in Search Console. I also compress images, lazy-load below-fold content, and monitor Core Web Vitals from the start. This path catches 90% of common indexing problems before launch.

First watch-out: CSS-generated content. I have seen developers add headings or descriptions using CSS pseudo-elements like ::before and ::after. Google does not treat that content as real text. Always put important text in the HTML. Second: confusion between robots.txt and noindex. If you disallow a URL via [robots.txt](/robots-txt/), Googlebot cannot fetch it, so the URL may still appear in search results if other sites link to it. For pages you really want out of the index, always use a noindex meta tag and allow crawling. Third: JavaScript rendering timing. Google's Web Rendering Service can execute JavaScript but has a queue and timeout. I check the rendered HTML in the URL Inspection Tool after each deploy. If critical content is missing, I move it to server-side rendering. Fourth: canonicalisation mistakes. I frequently see rel="canonical" pointing to a different domain or a non-existent page. Also, watch out for self-referencing canonicals that are case-sensitive. Fifth: over-optimising for speed at the cost of content. Lazy-loading images is good, but if you lazy-load the hero image, Largest Contentful Paint can still suffer. I measure real performance with Lighthouse and field data. Sixth: forgetting to update the sitemap after removing pages. Stale sitemaps signal low quality to Google. I regenerate the sitemap on every publish event. Seventh: be careful with pagination. If you use infinite scroll without proper URL updates, Google may only see page one. I implement traditional pagination or load more with pushState. Another watch-out: using noindex on paginated pages to avoid duplicate content is fine for low-value pages, but if you apply it to all paginated product listings, you lose the chance to rank for category-specific queries. I prefer to canonicalise the first page and keep subsequent pages indexable with proper pagination signals.

I made several mistakes early in my career that changed how I approach web dev SEO. First, I underestimated the value of sitemaps. I believed that if internal linking was perfect, Google would find every page naturally. Then I worked on a site with 10,000 product pages. Despite a well-structured link graph, new products took weeks to appear in the index. After submitting a sitemap, indexing dropped to hours. Now I treat the sitemap as a non-negotiable discovery tool. Second, I thought Google could execute JavaScript as reliably as a modern browser. I built a Vue.js app where the entire product page was rendered client-side. Google indexed some pages but missed many, especially those with dynamic filters. I now server-render all content for key pages or use static generation. Third, I ignored canonical tags on small sites. I assumed duplicates only matter at scale. But even a five-page site can have duplicate versions via www vs non-www or trailing slashes. I now set canonicals everywhere, using a simple template variable. Fourth, I once advised a client to disallow their staging subdomain in robots.txt, thinking it would keep it out of the index. But because the staging site was public and linked from external tools, Google indexed pages anyway. Staging should be blocked by authentication, not robots.txt. Fifth, I used to recommend full site-wide lazy-loading, thinking speed was king. But lazy-loading everything above the fold hurt LCP because the browser had to wait to load critical images. Now I prioritise above-fold content to be eagerly loaded. Finally, I used to think SEO was the content team's job. I now realise that development decisions — framework choice, rendering strategy, URL structure — have a bigger impact than any meta description change. I sit in on every architecture review for new projects.