feat!: Allow running lighthouse on other pages available in publish folder (#487)

Release-As: 4.0.0

BREAKING-CHANGE: The `path` audit input option no longer affects the served directory for an audit. Use `serveDir` instead. Use `path` to specify the sub directory or `html` file within the served directory that should be audited.

Author: aitchiss

.env.example

@@ -1,5 +1,5 @@
 # Audits
-AUDITS=[{"url":"https://www.example.com","thresholds":{"performance":0.5}},{"path":""},{"path":"route1"},{"path":"route2"}]
+AUDITS=[{"url":"https://www.example.com","thresholds":{"performance":0.5}},{"serveDir":""},{"serveDir":"route1"},{"path":"route2"}, {path: "contact.html"}]
 # Ignored when a url is configured for an audit
 PUBLISH_DIR=FULL_PATH_TO_LOCAL_BUILD_DIRECTORY
 # JSON string of thresholds to enforce

README.md

@@ -4,14 +4,15 @@ A Netlify plugin to generate a Lighthouse report for every deploy
 
 ## Installation options
 
-You can install the plugin for your site using your `netlify.toml` file or the Netlify UI. 
+You can install the plugin for your site using your `netlify.toml` file or the Netlify UI.
 
 For the most customization options, we recommend installing the Lighthouse plugin with a `netlify.toml` file.
 
 `netlify.toml` file-based installation allows you to:
-  - [Run Lighthouse audits for different site paths, such as the contact page and site home page](#run-lighthouse-for-different-site-paths)
-  - [Run Lighthouse audits for a desktop device](#run-lighthouse-for-the-desktop-experience)
-  - [Generate Lighthouse results in a language other than English](#generate-lighthouse-results-in-other-languages)
+
+- [Run Lighthouse audits for different site paths, such as the contact page and site home page](#run-lighthouse-for-different-site-paths)
+- [Run Lighthouse audits for a desktop device](#run-lighthouse-for-the-desktop-experience)
+- [Generate Lighthouse results in a language other than English](#generate-lighthouse-results-in-other-languages)
 
 ### Install plugin through the Netlify UI
 
@@ -46,7 +47,7 @@ Then add the plugin to your `netlify.toml` configuration file:
     output_path = "reports/lighthouse.html"
 ```
 
-By default, the plugin will serve and audit the build directory of the site.
+By default, the plugin will serve and audit the build directory of the site, inspecting the `index.html`.
 You can customize the behavior via the `audits` input:
 
 ```toml
@@ -64,13 +65,27 @@ You can customize the behavior via the `audits` input:
     # you can specify output_path per audit, relative to the path
     output_path = "reports/route1.html"
 
+  # to audit an HTML file other than index.html in the build directory
+  [[plugins.inputs.audits]]
+    path = "contact.html"
+
+  # to audit an HTML file other than index.html in a sub path of the build directory
+  [[plugins.inputs.audits]]
+    path = "pages/contact.html"
+
   # to audit a specific absolute url
   [[plugins.inputs.audits]]
     url = "https://www.example.com"
 
     # you can specify thresholds per audit
     [plugins.inputs.audits.thresholds]
       performance = 0.8
+
+  # to serve only a sub directory of the build directory for an audit
+  # pages/index.html will be audited, and files outside of this directory will not be served
+  [[plugins.inputs.audits]]
+    serveDir = "pages"
+
 ```
 
 The lighthouse scores are automatically printed to the **Deploy log** in the Netlify UI. For example:
@@ -103,7 +118,7 @@ To customize how Lighthouse runs audits, you can make changes to the `netlify.to
 
 By default, Lighthouse takes a mobile-first performance testing approach and runs audits for the mobile device experience. You can optionally run Lighthouse audits for the desktop experience by including `preset = "desktop"` in your `netlify.toml` file:
 
-```
+```toml
 [[plugins]]
   package = "@netlify/plugin-lighthouse"
 
@@ -117,15 +132,15 @@ To return to running Lighthouse audits for the mobile experience, just remove th
 
 ### Generate Lighthouse results in other languages
 
-By default, Lighthouse results are generated in English. To return Lighthouse results in other languages, include the language code from any Lighthouse-supported locale in your `netlify.toml` file. 
+By default, Lighthouse results are generated in English. To return Lighthouse results in other languages, include the language code from any Lighthouse-supported locale in your `netlify.toml` file.
 
 For the latest Lighthouse supported locales or language codes, check out this [official Lighthouse code](https://github.com/GoogleChrome/lighthouse/blob/da3c865d698abc9365fa7bb087a08ce8c89b0a05/types/lhr/settings.d.ts#L9).
 
 Updates to `netlify.toml` will take effect for new builds.
 
 #### Example to generate Lighthouse results in Spanish
 
-```
+```toml
 [[plugins]]
   package = "@netlify/plugin-lighthouse"
 
@@ -146,7 +161,7 @@ yarn local
 
 ## Preview Lighthouse results within the Netlify UI
 
-Netlify offers an experimental feature through Netlify Labs that allows you to view Lighthouse scores for each of your builds on your site's Deploy Details page with a much richer format. 
+Netlify offers an experimental feature through Netlify Labs that allows you to view Lighthouse scores for each of your builds on your site's Deploy Details page with a much richer format.
 
 You'll need to install the [Lighthouse build plugin](https://app.netlify.com/plugins/@netlify/plugin-lighthouse/install) on your site and then enable this experimental feature through Netlify Labs.
 

example/contact.html

@@ -0,0 +1,12 @@
+<!DOCTYPE html>
+<html lang="en">
+  <head>
+    <meta charset="UTF-8" />
+    <meta http-equiv="X-UA-Compatible" content="IE=edge" />
+    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+    <title>Document</title>
+  </head>
+  <body>
+    <h1>Contact</h1>
+  </body>
+</html>

example/index.html

@@ -1,10 +1,12 @@
 <!DOCTYPE html>
-<html>
+<html lang="en">
   <head>
-    <meta charset="utf-8" />
-    <title>Empty Site</title>
+    <meta charset="UTF-8" />
+    <meta http-equiv="X-UA-Compatible" content="IE=edge" />
+    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+    <title>Document</title>
   </head>
   <body>
-    Empty Site
+    <h1>Example</h1>
   </body>
 </html>

example/route1/index.html

@@ -1,10 +1,12 @@
 <!DOCTYPE html>
-<html>
+<html lang="en">
   <head>
-    <meta charset="utf-8" />
-    <title>Route 1</title>
+    <meta charset="UTF-8" />
+    <meta http-equiv="X-UA-Compatible" content="IE=edge" />
+    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+    <title>Document</title>
   </head>
   <body>
-    Route 1
+    <h1>Example Route 1</h1>
   </body>
 </html>

example/route2/index.html

@@ -1,10 +1,12 @@
 <!DOCTYPE html>
-<html>
+<html lang="en">
   <head>
-    <meta charset="utf-8" />
-    <title>Route 2</title>
+    <meta charset="UTF-8" />
+    <meta http-equiv="X-UA-Compatible" content="IE=edge" />
+    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+    <title>Document</title>
   </head>
   <body>
-    Route 2
+    <h1>Example Route 2</h1>
   </body>
 </html>

netlify.toml

@@ -16,11 +16,11 @@ performance = 0.9
 
 [[plugins.inputs.audits]]
 output_path = "reports/route1.html"
-path = "route1"
+serveDir = "route1"
 [[plugins.inputs.audits]]
-path = "route2"
+serveDir = "route2"
 [[plugins.inputs.audits]]
-path = ""
+serveDir = ""
 
 [[plugins]]
 package = "netlify-plugin-cypress"

src/config.js

@@ -1,23 +1,23 @@
 const chalk = require('chalk');
 const { join } = require('path');
 
-const getServePath = (dir, path) => {
-  if (typeof path !== 'string' || typeof dir !== 'string') {
-    return { path: undefined };
+const getServePath = (dir, subDir) => {
+  if (typeof subDir !== 'string' || typeof dir !== 'string') {
+    return { serveDir: undefined };
   }
 
-  const resolvedPath = join(dir, path);
+  const resolvedPath = join(dir, subDir);
   if (!resolvedPath.startsWith(dir)) {
     throw new Error(
       chalk.red(
         `resolved path for ${chalk.red(
-          path,
+          subDir,
         )} is outside publish directory ${chalk.red(dir)}`,
       ),
     );
   }
 
-  return { path: resolvedPath };
+  return { serveDir: resolvedPath };
 };
 
 const getConfiguration = ({ constants, inputs } = {}) => {
@@ -57,14 +57,14 @@ const getConfiguration = ({ constants, inputs } = {}) => {
   }
 
   if (!Array.isArray(audits)) {
-    audits = [{ path: serveDir, url: auditUrl, thresholds, output_path }];
+    audits = [{ serveDir, url: auditUrl, thresholds, output_path }];
   } else {
     audits = audits.map((a) => {
       return {
         ...a,
         thresholds: a.thresholds || thresholds,
         output_path: a.output_path || output_path,
-        ...getServePath(serveDir, a.path),
+        ...getServePath(serveDir, a.serveDir ?? ''),
       };
     });
   }

src/config.test.js

@@ -23,7 +23,7 @@ describe('config', () => {
     expect(config).toEqual({
       audits: [
         {
-          path: undefined,
+          serveDir: undefined,
           url: undefined,
           thresholds: {},
         },
@@ -40,7 +40,7 @@ describe('config', () => {
     expect(config).toEqual({
       audits: [
         {
-          path: 'PUBLISH_DIR',
+          serveDir: 'PUBLISH_DIR',
           url: 'AUDIT_URL',
           thresholds: { performance: 0.9 },
         },
@@ -52,14 +52,18 @@ describe('config', () => {
     process.env.PUBLISH_DIR = 'PUBLISH_DIR';
     process.env.AUDITS = JSON.stringify([
       { url: 'https://www.test.com', thresholds: { performance: 0.9 } },
-      { path: 'route1', thresholds: { seo: 0.9 } },
+      { serveDir: 'route1', thresholds: { seo: 0.9 } },
     ]);
     const config = getConfiguration();
 
     expect(config).toEqual({
       audits: [
-        { url: 'https://www.test.com', thresholds: { performance: 0.9 } },
-        { path: 'PUBLISH_DIR/route1', thresholds: { seo: 0.9 } },
+        {
+          url: 'https://www.test.com',
+          thresholds: { performance: 0.9 },
+          serveDir: 'PUBLISH_DIR',
+        },
+        { serveDir: 'PUBLISH_DIR/route1', thresholds: { seo: 0.9 } },
       ],
     });
   });
@@ -87,7 +91,7 @@ describe('config', () => {
     expect(config).toEqual({
       audits: [
         {
-          path: 'PUBLISH_DIR',
+          serveDir: 'PUBLISH_DIR',
           url: 'url',
           thresholds: { seo: 1 },
           output_path: 'reports/lighthouse.html',
@@ -96,37 +100,40 @@ describe('config', () => {
     });
   });
 
-  it('should append audits path to PUBLISH_DIR', () => {
+  it('should append audits serveDir to PUBLISH_DIR', () => {
     const constants = { PUBLISH_DIR: 'PUBLISH_DIR' };
-    const inputs = { audits: [{ path: 'route1', thresholds: { seo: 1 } }] };
+    const inputs = { audits: [{ serveDir: 'route1', thresholds: { seo: 1 } }] };
     const config = getConfiguration({ constants, inputs });
 
     expect(config).toEqual({
-      audits: [{ path: 'PUBLISH_DIR/route1', thresholds: { seo: 1 } }],
+      audits: [{ serveDir: 'PUBLISH_DIR/route1', thresholds: { seo: 1 } }],
     });
   });
 
   it('should use default thresholds when no audit thresholds is configured', () => {
     const constants = { PUBLISH_DIR: 'PUBLISH_DIR' };
     const inputs = {
       thresholds: { performance: 1 },
-      audits: [{ path: 'route1', thresholds: { seo: 1 } }, { path: 'route2' }],
+      audits: [
+        { serveDir: 'route1', thresholds: { seo: 1 } },
+        { serveDir: 'route2' },
+      ],
     };
     const config = getConfiguration({ constants, inputs });
 
     expect(config).toEqual({
       audits: [
-        { path: 'PUBLISH_DIR/route1', thresholds: { seo: 1 } },
-        { path: 'PUBLISH_DIR/route2', thresholds: { performance: 1 } },
+        { serveDir: 'PUBLISH_DIR/route1', thresholds: { seo: 1 } },
+        { serveDir: 'PUBLISH_DIR/route2', thresholds: { performance: 1 } },
       ],
     });
   });
 
-  it('should throw error on path traversal', () => {
+  it('should throw error on serveDir path traversal', () => {
     const constants = { PUBLISH_DIR: 'PUBLISH_DIR' };
     const inputs = {
       thresholds: { performance: 1 },
-      audits: [{ path: '../' }],
+      audits: [{ serveDir: '../' }],
     };
 
     expect(() => getConfiguration({ constants, inputs })).toThrow(
@@ -136,16 +143,16 @@ describe('config', () => {
     );
   });
 
-  it('should treat audit path as relative path', () => {
+  it('should treat audit serveDir as relative path', () => {
     const constants = { PUBLISH_DIR: 'PUBLISH_DIR' };
     const inputs = {
-      audits: [{ path: '/a/b' }],
+      audits: [{ serveDir: '/a/b' }],
     };
 
     const config = getConfiguration({ constants, inputs });
 
     expect(config).toEqual({
-      audits: [{ path: 'PUBLISH_DIR/a/b', thresholds: {} }],
+      audits: [{ serveDir: 'PUBLISH_DIR/a/b', thresholds: {} }],
     });
   });
 
@@ -182,21 +189,21 @@ describe('config', () => {
     const inputs = {
       output_path: 'reports/lighthouse.html',
       audits: [
-        { path: 'route1' },
-        { path: 'route2', output_path: 'reports/route2.html' },
+        { serveDir: 'route1' },
+        { serveDir: 'route2', output_path: 'reports/route2.html' },
       ],
     };
     const config = getConfiguration({ constants, inputs });
 
     expect(config).toEqual({
       audits: [
         {
-          path: 'PUBLISH_DIR/route1',
+          serveDir: 'PUBLISH_DIR/route1',
           output_path: 'reports/lighthouse.html',
           thresholds: {},
         },
         {
-          path: 'PUBLISH_DIR/route2',
+          serveDir: 'PUBLISH_DIR/route2',
           output_path: 'reports/route2.html',
           thresholds: {},
         },