Shopify has launched a new Core Web Vitals dashboard for online stores. This move shifts focus from traditional speed scores to Real User Monitoring (RUM) data, which reflects actual user experiences more accurately. Previously, Shopify provided a single Speed Score to site owners, but this change means they will now have access to more detailed RUM data.

The dashboard is available only for Liquid-based storefronts and offers insights into Google's Core Web Vitals, a set of metrics crucial for user experience and search engine rankings. The dashboard presents data based on the 75th percentile, aligning with the Core Web Vitals approach, and provides details like metric distribution, trend over time, and user experience categorization (good, moderate, poor).

Shopify (and Fudge :) ) believes that understanding real user interactions is vital for optimizing web performance and ultimately, sales. Filters like device type, date range, and time frame grouping allow for detailed analysis. However, some sites might not yet see this data due to reasons like password protection or lack of traffic.

The shift away from Lighthouse scores, which don't fully represent user experience, is part of a broader industry trend prioritizing real user data over synthetic test scores for SEO and user experience purposes. This update is expected to help Shopify merchants better understand and optimize their sites' performance from a real-world perspective.

Why synthetic test (lab data) is often different

Lab data and field data, often used for assessing website performance, measure different aspects of user experience. Here's a simpler breakdown of why they differ:

Field Data (Real User Data):

• Reflects a range of real-world conditions: different networks, devices, and user behaviors.
• Accounts for variables like browser and platform optimizations (e.g., back/forward cache).
• Represents actual user interactions such as scrolling or tapping.
• Measures real-time user experiences including diverse geographic locations and device capabilities.
• Includes data from different page elements users encounter, depending on various factors like screen size and personalization.

Lab Data (Synthetic Testing):

• Happens in a controlled environment: a single device, network, and location.
• Tests a specific set of conditions, making it ideal for debugging and feature testing.
• Doesn't capture the variety of real-world user behaviors or conditions.
• Typically uses a "clean" browser state, without the influence of caching that real users might have.
• Measures performance based on a predefined set of parameters, which might not represent the majority of real user experiences.

Key Differences in Performance Metrics:

• Largest Contentful Paint (LCP): In lab tests, LCP might be consistent but, in real user data, it varies based on factors like device, personalization, and whether the user follows a direct link or a shared URL with specific parameters.
• Interaction to Next Paint (INP) and Total Blocking Time (TBT): Real user interactions dictate INP, which lab tests can't replicate. Lab data may not consider user wait times or tap delays on mobile devices.
• Cumulative Layout Shift (CLS): Lab tests often only measure shifts during page load and above the fold. Real user data accounts for shifts throughout the page lifespan, including those caused by user interactions.

Lab data is great for identifying potential issues and testing fixes before they go live. It can also help in understanding performance for users on different networks or devices. However, it doesn't fully capture the diversity of real-world user experiences.

Field data, on the other hand, gives insights into how real users interact with a site under a variety of conditions. It helps prioritize improvements based on actual user experiences.

Conclusion

Both lab and field data are valuable for website performance optimization. Field data is crucial for understanding real user experiences, while lab data is beneficial for testing and debugging under specific conditions. A combination of both provides a comprehensive view of website performance.

‍