Website SEO Optimization: Complete 2026 Guide — Checklists, robots.txt, GEO

TL;DR. Website SEO in 2026 is no longer just about Title tags and links. The modern approach combines four pillars: technical SEO (Core Web Vitals, robots.txt, schema), on-page (content matching search intent + E-E-A-T), off-page (digital PR instead of link mass) and GEO — Generative Engine Optimization for AI Overviews, ChatGPT Search and Perplexity. This Spilno Agency guide gives you complete checklists for each pillar, a robots.txt template, updated Core Web Vitals thresholds (INP replaced FID on March 12, 2024) and a list of tactics that no longer work.

What website SEO means in 2026

Website SEO (Search Engine Optimization) is the systematic work of getting your site to rank higher in Google for target queries without paying for each click. In 2026 the classic Google search is joined by AI Overviews, ChatGPT Search, Perplexity and Gemini — so the abbreviation SEO is increasingly extended to GEO (Generative Engine Optimization). The principles remain the same — useful content, technical cleanliness, brand authority — but the tools and metrics have evolved.

This guide is updated by Spilno Agency in May 2026. We run SEO for European clients in finance, e-commerce and B2B services as an engineering process with metrics and hypotheses. Below is a complete walkthrough with links to our deep-dive guides on the hardest topics: GSC, semantic core, technical SEO audit, E-E-A-T.

How Google search works: 4 stages

1. Crawling. Googlebot walks the site via internal and external links, reads robots.txt and sitemap.xml.
2. Rendering. Headless Chromium executes JavaScript, builds the DOM, measures Core Web Vitals.
3. Indexing. The page enters the index if it meets requirements: unique content, canonical, allowed in robots/meta.
4. Ranking. The algorithm decides the position based on 200+ signals: relevance, authority, user behaviour, expertise, AI signals for AI Overviews.

A page cannot rank if any of the first three stages is broken. That is why technical SEO is the foundation — not an optional add-on.

The 4 pillars of modern SEO

In 2026 we work on four parallel tracks. None of them is self-sufficient: perfect technical SEO without content produces no traffic, and brilliant content without links does not break through in competitive niches.

Technical SEO — extended checklist

Technical SEO controls whether Google can see, understand and quickly serve your site to users. Gaps here nullify any content effort.

• Site is accessible to Googlebot — verify in GSC URL Inspection
• robots.txt is valid, doesn’t block CSS/JS, contains a Sitemap directive
• sitemap.xml is auto-generated and submitted to GSC (separate maps for pages, news, images, video)
• Canonical URLs set on every page
• hreflang configured reciprocally across all language versions
• HTTPS site-wide, HTTP→HTTPS redirected with 301
• Single URL variant: www or non-www, trailing slash or not — everything else 301
• 404 pages return HTTP 404, not 200
• Soft-404 fixed (empty pages with 200)
• Redirect chains collapsed (A→B→C becomes A→C)
• Duplicate pages consolidated (canonical or 301)
• JSON-LD schema.org: Organization, WebSite, BreadcrumbList everywhere; BlogPosting, Product, FAQPage, HowTo where relevant
• Core Web Vitals in the green range (see below)
• Mobile-friendly: viewport, tap targets, no horizontal scroll
• Responsive images: WebP/AVIF, srcset, lazy-load, explicit width/height
• Server returns proper Last-Modified and ETag
• GZIP/Brotli compression for HTML/CSS/JS
• TTFB (Time To First Byte) ≤ 600 ms, ideally ≤ 200 ms
• Browser cache (Cache-Control) and CDN configured
• Human-readable URLs, no session IDs
• Pagination via rel="next/prev" or a single canonical page
• Internal linking from priority hubs to target pages (≤3 clicks from home)
• Log analysis once a quarter: where Googlebot crawls
• Google Search Console and GA4 configured

What should be in robots.txt

robots.txt is the instruction file for search bots — what to crawl and what to skip. It lives at the domain root: https://site.com/robots.txt. Google checks it on every crawl. A baseline template for a WordPress site:

User-agent: *
Disallow: /wp-admin/
Disallow: /cart/
Disallow: /checkout/
Disallow: /my-account/
Disallow: /?s=
Disallow: /*?utm_*
Disallow: /*?fbclid=
Allow: /wp-admin/admin-ajax.php

# AI bots — restrict separately if needed
User-agent: GPTBot
Allow: /

User-agent: PerplexityBot
Allow: /

Sitemap: https://site.com/sitemap.xml
Sitemap: https://site.com/news-sitemap.xml
Sitemap: https://site.com/image-sitemap.xml

Must-haves:

• User-agent: directive (wildcard or specific bot)
• Disallow: for service and duplicate URLs (admin, cart, search, utm)
• Allow: for critical URLs inside blocked folders (admin-ajax.php)
• Sitemap: — links to all your XML sitemaps (full URL with https://)
• Validate via GSC Settings → robots.txt

What NOT to do:

• Don’t block folders with CSS/JS — Google can’t render the page and will downrank it
• Don’t combine robots.txt Disallow: with on-page noindex — the bot won’t visit the page, won’t see the tag, and will keep it indexed via links
• Don’t use robots.txt to hide private data — disallow doesn’t hide URLs from humans
• Don’t 301-redirect robots.txt — Google may treat it as unavailable
• Don’t exceed 500 KB — Google truncates the file
• Don’t repeat directives for the same bot multiple times — that’s a conflict

On-page SEO — complete checklist

On-page SEO is per-page work: keyword research, tags, structure, content, internal linking. Deep dive: «On-page SEO + audit».

• Semantic core clustered by search intent
• Unique Title per page (50-60 chars, primary keyword + brand)
• Meta description (140-160 chars, action verb + keyword)
• Single H1 per page with the primary keyword naturally placed
• H2/H3 structure covers the cluster subtopics (1 H2 = 1 subtopic)
• Content deeper than top-10 competitors (topical completeness, not just word count)
• First 100 words contain the primary keyword and answer the main question
• FAQ block at the end + FAQPage schema
• Internal links with keyword anchors (3-5 from relevant sections)
• External links to authoritative sources (Google, official data)
• Images: WebP/AVIF, alt with keyword, descriptive filename (seo-checklist-2026.webp)
• Clean URLs: short, with primary keyword, no extra parameters
• Breadcrumbs (BreadcrumbList schema)
• Date published + last updated (datePublished, dateModified)
• Author with bio (Person schema) — especially YMYL
• Page has an obvious user action (form, phone, CTA)
• Glossary / definitions block — boosts AI Overview citations
• Estimated reading time (3-15 min optimal)
• Social buttons (Open Graph + Twitter Card)
• Relevant category/tag, not «everything at once»
• Comments enabled with moderation, or disabled
• Old URL change → 301 redirect from the old URL
• Accessibility WCAG: contrast, focus, readability, aria-label
• Real-query testing via GSC URL Inspection

Off-page SEO — building authority

Off-page SEO in 2026 is no longer «buy 100 links». Google learned to distinguish editorial links from artificial ones. Instead of mass — build brand reputation through digital PR, expert mentions and partnerships.

• Digital PR: expert commentary in top European media
• Guest posts on topical sites with real traffic (not fake PBNs)
• Unlinked brand mentions — Google considers them via NLP
• Local directories (Google Business Profile, OpenStreetMap, EU business directories)
• Partner links from case studies and integrations
• User-generated content (Google Maps reviews, on-site testimonials)
• Editorial-style placements on platforms like Collaborator — topical only
• PR pushes via niche newsletters and Telegram
• Podcasts, interviews, conference talks — generate natural mentions
• Original research, white papers, statistics — get cited with links
• Anchor profile control: ≤30% commercial anchors, the rest branded + URL + generic
• Disavow only for manual penalties or active negative SEO

Content & E-E-A-T

Google evaluates sites by the E-E-A-T framework: Experience, Expertise, Authoritativeness, Trustworthiness. The ‘Experience’ letter was added in 2022 — Google wants to see real, hands-on experience with the topic, especially in YMYL areas (Your Money or Your Life: finance, medicine, law).

• Author pages with bio, photo, LinkedIn / social links
• Person schema on the author page
• Real case studies with numbers, screenshots, charts
• Expert quotes from domain specialists
• Last-updated date on every article
• Regular content refresh of priority articles (every 6-12 months)
• Honest conflict-of-interest disclosure (affiliate markers)
• About-us page with team, history, contacts
• Legal pages: privacy policy, terms, contacts, company registration
• Reviews / testimonials with real names and photos
• Content that explains «why», not just «what»
• Engagement loops: comments, forms, email

Local SEO

• Google Business Profile claimed, verified, 100% filled
• GBP categories chosen precisely (primary + secondary)
• Profile photos, cover, interior, products — refreshed monthly
• Regular Google Posts (news, offers, events) — 2-4 per month
• Reply to 100% of reviews (especially negative ones)
• Q&A pre-populated with common questions
• Contact page with map + identical NAP (Name/Address/Phone) across all directories
• LocalBusiness schema on the contact page
• Local mentions in directories (Foursquare, local marketplaces)
• If multiple physical locations — a dedicated page per city/branch
• Local keywords in Title/H1
• Local rank tracking (GSC by country + rank-tracking tools)

GEO — optimization for AI search

Generative Engine Optimization is the new discipline of 2024-2026. Users increasingly get answers via AI Overviews in Google, ChatGPT Search, Perplexity, Gemini, Microsoft Copilot. Classic SEO traffic for informational queries drops 15-40%, and the winners are sites that AI cites.

• llms.txt at the domain root — a structured summary of your site for LLMs
• Schema.org maxed out — AI reads JSON-LD as «the truth» about the page
• Clear question→answer facts — AI copies exactly this format
• Unique statistics and original research — AI cites such sources
• Expert author with verified bio and Person schema
• Standardized terminology: glossary block, term definitions
• Regularly updated articles with dates — AI prefers fresh
• If you have an API — publish OpenAPI spec (AI agents use it)
• Don’t block AI bots (GPTBot, PerplexityBot, ClaudeBot) in robots.txt
• Content answers the full question rather than dodging — AI takes the first complete paragraph

Core Web Vitals 2026

On March 12, 2024 Google replaced FID (First Input Delay) with INP (Interaction to Next Paint). INP measures site responsiveness across all user interactions over the entire session — a stricter and more realistic metric. Thresholds:

• LCP (Largest Contentful Paint): ≤ 2.5 s good; ≤ 4.0 s needs improvement; > 4.0 s poor
• INP (Interaction to Next Paint): ≤ 200 ms good; ≤ 500 ms needs improvement; > 500 ms poor
• CLS (Cumulative Layout Shift): ≤ 0.1 good; ≤ 0.25 needs improvement; > 0.25 poor

Check scores in GSC Core Web Vitals (CrUX real-user data) and PageSpeed Insights / Lighthouse (lab tests). Fix pages in red or amber — Google applies a downranking signal to them.

Analytics & monitoring

• Google Search Console — primary source for impressions & clicks
• GA4 with conversions (lead form, call, ecommerce purchase)
• GSC ↔ GA4 link (Acquisition → Google Organic)
• Looker Studio dashboard with weekly position, clicks, conversion trends
• Rank tracker (Serpstat, SE Ranking, Ahrefs) — weekly positions
• Server logs or Cloudflare logs — quarterly Googlebot review
• Alerts on >20% w/w traffic drops
• GSC annotations on every release / URL change
• Periodic Screaming Frog / Sitebulb crawl (monthly)
• Core Web Vitals monitoring (PageSpeed Insights API + CrUX)
• Title A/B tests via CMS
• Monthly client report with findings and next steps

What no longer works in SEO 2026

Outdated tactics that still live in old guides. In 2026 they either produce no effect or actively hurt:

• Meta keywords. Google ignored this tag since 2009.
• AMP. No longer required for Top Stories since 2022; project effectively wound down.
• Keyword density 3-5%. Google moved to topical models a decade ago.
• Disavow for most sites. Only for manual penalties.
• Authorship (rel=author). Google removed it in 2014. Use Author schema instead.
• Thousands of paid PBN links. SpamBrain detects them and triggers penalties.
• Duplicate pages targeting different keywords. Cannibalization — merge into hub pages.
• Plain text without structured data. AI Overviews won’t cite such pages.

SEO tools in 2026

• Google Search Console — required, free
• Google Analytics 4 — required, free
• PageSpeed Insights + Lighthouse — Core Web Vitals
• Ahrefs — links, keywords, audit ($129/mo)
• Semrush — competitive analysis, keywords ($139/mo)
• Serpstat — Ukrainian platform, keywords, audit ($59/mo)
• SE Ranking — rank tracking, audit ($65/mo)
• Screaming Frog SEO Spider — desktop crawler (£199/yr)
• Sitebulb — desktop crawler with visualization ($35/mo)
• Looker Studio — dashboards, free
• Bing Webmaster Tools — separate console for Bing/Copilot
• Collaborator.pro / PRposting — editorial placement marketplaces

FAQ

Where do I start SEO in 2026?

Technical audit, semantic core, optimization of priority pages (Title, Meta, H1-H3, content), connecting GSC and GA4. Then content planning and earning quality links.

What should be in robots.txt?

Minimum: User-agent: *, Disallow for service URLs (/wp-admin/, /cart/, /?s=, /*?utm_*), Allow for admin-ajax.php and two Sitemap directives. Don’t block CSS/JS — Google can’t render the page otherwise.

How are Core Web Vitals different in 2026?

On March 12, 2024 Google replaced FID with INP. INP thresholds: ≤200 ms good, ≤500 ms needs improvement, >500 ms poor. LCP and CLS unchanged.

What is GEO in SEO?

GEO (Generative Engine Optimization) — optimization for AI search engines (AI Overviews, ChatGPT Search, Perplexity, Gemini). Key: clear structure, citability, schema.org, factual claims, llms.txt, expert author.

How long does SEO take?

First changes in GSC — 2-4 weeks. Visibility growth — 3-6 months. Stable traffic — 6-12 months. New sites in competitive niches — up to 12-18 months.

Do I need Disavow in 2026?

Mostly no. Google publicly (Sullivan, Mueller, 2024-2025) repeats that Disavow is only useful for manual penalties or active negative SEO.