Tech Report: Lighthouse audit pass rates

[max-ostapenko]

Lighthouse audit pass rates:

• accessibility
• best practices
• performance
• PWA
• SEO

Updated /lighthouse response:

[
    {
        "date": "2024-07-01",
        "technology": "ALL",
        "rank": "ALL"        
        "geo": "ALL",
        "lighthouse": [
            {
                "mobile": {
                    "median_score": 0.835,
                    "pass_rate": 0.00,
                },
                "desktop": {
                    "median_score": 0.84,
                    "pass_rate": 0.00
                },
                "name": "accessibility"
            },
            {
                "mobile": {
                    "median_score": 0.79,
                    "pass_rate": 0.00,
                },
                "desktop": {
                    "median_score": 0.78,
                    "pass_rate": 0.00
                },
                "name": "best_practices"
            },
            {
                "mobile": {
                    "median_score": 0.42,
                    "pass_rate": 0.00
                },
                "desktop": {
                    "median_score": 0.66,
                    "pass_rate": 0.00
                },
                "name": "performance"
            },
            {
                "mobile": {
                    "median_score": null,
                    "pass_rate": 0.00
                },
                "desktop": {
                    "median_score": null,
                    "pass_rate": 0.00
                },
                "name": "pwa"
            },
            {
                "mobile": {
                    "median_score": 0.91,
                    "pass_rate": 0.00
                },
                "desktop": {
                    "median_score": 0.91,
                    "pass_rate": 0.00
                },
                "name": "seo"
            }
        ]
    },
    ...
]

Required schema adjustments

ALTER TABLE core_web_vitals.technologies
  ADD COLUMN lighthouse_score_accessibility_pass_rate FLOAT64,
  ADD COLUMN lighthouse_score_best_practices_pass_rate FLOAT64,
  ADD COLUMN lighthouse_score_performance_pass_rate FLOAT64,
  ADD COLUMN lighthouse_score_pwa_pass_rate FLOAT64,
  ADD COLUMN lighthouse_score_seo_pass_rate FLOAT64;

bq update reports.cwv_tech_lighthouse schema.json

[
  {
    "name": "date",
    "type": "DATE"
  },
  {
    "name": "technology",
    "type": "STRING"
  },
  {
    "name": "rank",
    "type": "STRING"
  },
  {
    "name": "geo",
    "type": "STRING"
  },
  {
    "fields": [
      {
        "name": "name",
        "type": "STRING"
      },
      {
        "fields": [
          {
            "name": "median_score",
            "type": "FLOAT"
          },
          {
            "name": "pass_rate",
            "type": "FLOAT"
          }
        ],
        "name": "desktop",
        "type": "RECORD"
      },
      {
        "fields": [
          {
            "name": "median_score",
            "type": "FLOAT"
          },
          {
            "name": "pass_rate",
            "type": "FLOAT"
          }
        ],
        "name": "mobile",
        "type": "RECORD"
      }
    ],
    "mode": "REPEATED",
    "name": "lighthouse",
    "type": "RECORD"
  }
]

Copilot reviewed 5 out of 6 changed files in this pull request and generated no comments.

Files not reviewed (1)

• definitions/output/reports/cwv_tech_categories.js: Evaluated as low risk

Comments suppressed due to low confidence (1)

definitions/output/reports/cwv_tech_technologies.js:13

• Inconsistent naming of 'technology'. It should be consistently referred to as either 'tech.technology' or 'technology' throughout the file.

tech.technology

[max-ostapenko]

@rviscomi do we need retrospective data? Since when?

[max-ostapenko]

@rviscomi did I get this one correct?
The formula for the pass rate, to be more specific.

[rviscomi]

Sorry I'm fuzzy on the context for this PR. This counts the number of sites whose LH category score >= 90? That's not something we currently have in Looker Studio, so it's not a feature parity thing, and AFAIK it's not something we discussed for the new dashboard.

We DO have a related discussion about highlighting technologies' areas for improvement. For that we could be highlighting the pass rates of individual audits, grouped by category. Or "here are the top ways LH says you can improve this technology's performance score". I believe you're exploring that in #49.

Eventually I would love to expand that to more insights powered by HA custom metrics or other manual analysis, like the % of sites that use fetchpriority=high on the LCP image or the % of sites that have statically discoverable LCP images.

[max-ostapenko]

That's exactly what I thought - I got lost in lighthouse JSON.

This one is the score pass rates, and we discard this - correct?

[rviscomi]

Sorry, yeah I don't think we need category-level pass rates (median score is fine)