Crawler Rendering ≠ User Rendering
At Tech SEO Connect 2025, our R&D Director, Giacomo Zecchini, showed how crawler behaviour differs from user rendering.
His findings show that the gap between what users see and what crawlers interpret is widening, especially as AI crawlers grow in influence but lag behind search engines in rendering sophistication.
A major focus of Giacomo’s talk was how rendering quirks, viewport expansion, and layout techniques like 100vh hero sections can cause crawlers to completely misunderstand page structure, hide key content, or misjudge its priority.
Watch the full talk here: https://www.youtube.com/watch?v=kZw6BsIytJU
Main Takeaways
1. Crawlers don’t behave like users
Giacomo emphasised that many AI crawlers still do not execute JavaScript or complete user interactions, meaning:
- Lazy-loaded content may never be added to the page.
- Interactive content never triggers.
- Pseudo-elements and display properties can hide essential information or devaluate the perceived value of it.
Even when JavaScript is executed, crawlers process the page differently from users resulting in mismatches in what gets indexed.
2. Viewport expansion changes layout
One of the standout insights from Giacomo’s talk is Google’s viewport expansion behaviour:
- Google initially renders a page using a fixed viewport (e.g. 1024×1024 or 412×732).
- Then Google expands the viewport height to match the full page.
- This triggers lazy-loaded elements that rely on viewport boundaries.
This can change how the content is shown, exposing or hiding information in ways teams never expect. Such rendering behaviour is unique to crawlers, not humans, so issues often go unnoticed.
3. The 100vh trap
Giacomo highlighted a common layout pitfall: using 100vh for hero banners or sections.
When viewport expansion occurs:
- The crawler recalculates
100vhbased on the expanded viewport. - A hero section intended to fill one screen suddenly becomes much taller, sometimes thousands of pixels.
- Primary content gets pushed dramatically down in the page.
If Google or AI crawlers incorporate layout position or ‘above-the-fold’ relevance into ranking signals, this can unintentionally downgrade the importance of key headings and content.
Even worse:
- Lazy-loaded content beneath a 100vh hero element may never trigger, because the crawler never “reaches” the threshold height for activation.
Giacomo’s recommendation: Use a max-height cap on fullscreen elements to preserve visual design without creating crawler distortions.
4. Lazy loading that works for crawlers
Lazy loading improves performance, but Giacomo reinforced a critical point:
Most crawlers don’t scroll.
Most crawlers don’t trigger user events.
Therefore:
- Scroll-based lazy loading fails silently.
- Touchstart/wheel-based loading doesn’t activate.
- Content that should be indexed simply never appears.
Safer patterns include:
- Intersection Observer API
- Static HTML fallbacks
- Avoiding scroll-triggered content loading entirely
These ensure both humans and crawlers get the same content.
5. Optimise for what crawlers can see
A consistent message in Giacomo’s talk: Content quality matters only if crawlers can see it.
SEO teams should consistently evaluate:
- Rendered DOM, not just raw HTML
- Layout tree and computed styles
- How viewport manipulation affects element position
- Whether key content is pushed too far down
- How AI crawlers differ from search crawlers in visibility
The “technical” in technical SEO is becoming more literal as we now optimise for rendering engines as much as for search algorithms.
Closing
Giacomo’s insights at Tech SEO Connect revealed that many SEO issues today are related to rendering.
From viewport expansion to the 100vh trap, from JS limitations to lazy-loading pitfalls, the modern crawler landscape requires far more precision in how pages are built and served.
As AI-driven discovery accelerates, ensuring that your layouts, behaviours, and rendering patterns align with crawler capabilities is a competitive advantage.
