bug: ssrServer ENOENT firestore.proto

[Bandit]

Describe the Bug

Have previously released/deployed without any problems. Tried to release tonight and when loading a page that relies on a load function with a Firebase call, it fails. Other pages seem to work.

This is the load function:

import { fetchProducts } from "$lib/stripeutils";
export async function load({ page, fetch, session, stuff }) {
    const products = await fetchProducts();
    return {
        props: {
            products,
        },
    };
}

This is the fetchProducts function

import { firestore, functions } from "$lib/firebase/client";
import { getDocs, query, where, orderBy, doc, addDoc, collection, onSnapshot } from 'firebase/firestore';
[...]

export async function fetchProducts() {
    let products = {
        [...]
    };

    const docSnap = await getDocs(query(
        collection(firestore(), "products"),
        where("active", "==", true),
        orderBy("metadata.order")
    ));
    
    await Promise.all(docSnap.docs.map(async (doc) => {
        let product = {
            [...]
        };

        const priceSnap = await getDocs(query(
            collection(doc.ref, "prices"),
            where("active", "==", true)
        ));

        priceSnap.docs.forEach((pdoc) => {
            const price = {
                [...]
            };

            product.prices = [price, ...product.prices];
        });

        const type = product.metadata.type || "addon";
        products[type].items.push(product);
    }));

    return products;
}

The firestore function

import { initializeApp, getApps } from "firebase/app";
import { getFirestore } from 'firebase/firestore';
[...]

const config = {
    [...]
};

function firebase() {
    return getApps().length === 0 ? initializeApp(config) : getApps()[0];
}

function firestore() {
	return getFirestore(firebase());
}

---

The errors:

ssrServer

[2021-10-28T07:36:58.392Z]  @firebase/firestore: Firestore (9.0.0): INTERNAL UNHANDLED ERROR:  Error: ENOENT: no such file or directory, open '/workspace/ssrServer/src/protos/google/firestore/v1/firestore.proto' 

Error: ENOENT: no such file or directory, open '/workspace/ssrServer/src/protos/google/firestore/v1/firestore.proto'
    at Object.openSync (fs.js:498:3)
    at Object.readFileSync (fs.js:394:35)
    at fetch2 (/workspace/ssrServer/index.js:11082:30)
    at Root2.load2 [as load] (/workspace/ssrServer/index.js:11111:11)
    at Root2.loadSync (/workspace/ssrServer/index.js:11121:19)
    at Object.loadProtosWithOptionsSync (/workspace/ssrServer/index.js:14289:31)
    at Object.loadSync (/workspace/ssrServer/index.js:14440:33)
    at loadProtos (/workspace/ssrServer/index.js:106181:43)
    at newConnection (/workspace/ssrServer/index.js:106185:20)
    at OnlineComponentProvider2.createDatastore (/workspace/ssrServer/index.js:109513:26) 

Steps to Reproduce

Have previously deployed without issue, however this is my first release where I've used a Firestore call inside a page's load function. Mainly looking for a place to start my debugging for something like this.

Expected Behaviour

Code should work fine?

svelte-adapter-firebase version

0.9.2

sveltejs/kit version

1.0.0.next.160

[Bandit]

I assume I don't need to manually manage dependencies in functions/package.json. Firestore isn't in there (but also neither is anything else I use in my site)

{
  "name": "functions",
  "description": "Cloud Functions for Firebase",
  "scripts": {
    "lint": "eslint .",
    "serve": "firebase emulators:start --only functions",
    "shell": "firebase functions:shell",
    "start": "npm run shell",
    "deploy": "firebase deploy --only functions",
    "logs": "firebase functions:log"
  },
  "engines": {
    "node": "14"
  },
  "main": "index.js",
  "dependencies": {
    "cors": "^2.8.5",
    "express": "^4.17.1",
    "firebase-admin": "^9.8.0",
    "firebase-functions": "^3.15.5",
    "stripe": "^8.164.0"
  },
  "devDependencies": {
    "eslint": "^7.6.0",
    "eslint-config-google": "^0.14.0",
    "firebase-functions-test": "^0.2.0"
  },
  "private": true
}

[Bandit]

This page is not working either, with the same ENOENT error

import { firestore } from "$lib/firebase/client";
import { collection, query, where, getDocs } from 'firebase/firestore';

export async function load({ page, fetch, session, stuff }) {
    const profilesRef = collection(firestore(), "profiles");
    const profileQuery = query(profilesRef, where("slug", "==", page.params.slug));

    const querySnap = await getDocs(profileQuery);
    
    if (querySnap.empty) return {
        status: 404,
        error: new Error(`Profile not found`),
    };

    return {
        props: {
            profile: {
                uid: querySnap.docs[0].id,
                ...querySnap.docs[0].data(),
            },
        },
    };
}

But if I navigate to this page using the SvelteKit router (e.g. clicking a link in the client) it works, so definitely entirely related to a bug with the ssrServer function.

Also worth noting the SSR-aspect of all these pages work fine locally, so nothing to do (as far as I can tell) with including client-side only dependencies or something like that.

[jthegedus]

I would suggest starting with updates to both SvelteKit and the adapter. There have been quite a few versions released since those versions you list in the issue. Also, there will be some breaking changes from Kit for the adapter API relatively soon which will affect the compilation of the SSR code, so I expect behaviour to change again in another breaking change.

[jthegedus]

I assume I don't need to manually manage dependencies in functions/package.json

Correct, although the SSR code runs on the Cloud Functions, all dependencies for the SSR code are compiled in-line during build phase so you shouldn't need to touch your Cloud Function deps for the Kit APIs or SSR code.

[jthegedus]

first release where I've used a Firestore call inside a page's load function

Many people have had issues with including Firestore directly in load, it can usually be mitigated by performing the Firestore data load in a Kit endpoint and then fetching that kit endpoint like any other REST API

[Bandit]

I updated to v1.0.0-next.192 and 0.13.1 and now get this error on a page that worked prior to upgrading:

TypeError [ERR_INVALID_ARG_VALUE]: The argument 'filename' must be a file URL object, file URL string, or absolute path string. Received undefined
    at Function.createRequire (internal/modules/cjs/loader.js:1173:11)
    at Object.<anonymous> (/workspace/ssrServer/index.js:133837:38)
    at Module._compile (internal/modules/cjs/loader.js:1072:14)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1101:10)
    at Module.load (internal/modules/cjs/loader.js:937:32)
    at Function.Module._load (internal/modules/cjs/loader.js:778:12)
    at Module.require (internal/modules/cjs/loader.js:961:19)
    at require (internal/modules/cjs/helpers.js:92:18)
    at /workspace/index.js:273:31
    at cloudFunction (/workspace/node_modules/firebase-functions/lib/providers/https.js:50:16) 

/EDIT I get this error on every page, and the client gets served with Error: could not handle the request

Ideas?

[jthegedus]

How are you running your app? Just npm run dev in your SvelteKit app directory?

[Bandit]

npm run dev --

Runs fine locally though, just get those errors on the server in the ssrServer function

[jthegedus]

can you share more of your actual code? Hard to debug without it.

[Bandit]

Which part of the code should I share? No page can be served by the ssrServer function, even a static page with no code at all on it.

E.g. a markdown file parsed with mdsvex into svelte that has literally no JS in it triggers this error:

 ssrServer

TypeError [ERR_INVALID_ARG_VALUE]: The argument 'filename' must be a file URL object, file URL string, or absolute path string. Received undefined
    at Function.createRequire (internal/modules/cjs/loader.js:1173:11)
    at Object.<anonymous> (/workspace/ssrServer/index.js:133837:38)
    at Module._compile (internal/modules/cjs/loader.js:1072:14)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1101:10)
    at Module.load (internal/modules/cjs/loader.js:937:32)
    at Function.Module._load (internal/modules/cjs/loader.js:778:12)
    at Module.require (internal/modules/cjs/loader.js:961:19)
    at require (internal/modules/cjs/helpers.js:92:18)
    at /workspace/index.js:273:31
    at cloudFunction (/workspace/node_modules/firebase-functions/lib/providers/https.js:50:16) 

[Bandit]

firebase.json

{
  "functions": {
    "source": "functions"
  },
  "hosting": {
    "public": "public",
    "ignore": [
      "firebase.json",
      "**/.*",
      "**/node_modules/**"
    ],
    "rewrites": [
      {
        "source": "**",
        "function": "ssrServer"
      }
    ],
    "predeploy": ["npm run build"]
  }
}

svelte.config.js

import { mdsvex } from "mdsvex";
import mdsvexConfig from "./mdsvex.config.js";
import firebase from "svelte-adapter-firebase";
import preprocess from "svelte-preprocess";

/** @type {import('@sveltejs/kit').Config} */
const config = {
    "extensions": [".svelte", ...mdsvexConfig.extensions],

    kit: {
        adapter: firebase(),
		// hydrate the <div id="svelte"> element in src/app.html
		target: '#svelte',
        vite: {
	        ssr: {
		        external: ['firebase']
	        }
	},

    preprocess: [
        mdsvex(mdsvexConfig),
        preprocess({
            "postcss": true
        })
    ],

    onwarn: (warning, handler) => {
        const { code, frame } = warning;
        if (code === "a11y-missing-content") return;

        console.log(code);
        handler(warning);
    }
};

export default config;
// Workaround until SvelteKit uses Vite 2.3.8 (and it's confirmed to fix the Tailwind JIT problem)
const mode = process.env.NODE_ENV;
const dev = mode === "development";
process.env.TAILWIND_MODE = dev ? "watch" : "build";

Top of package.json

"scripts": {
		"dev": "svelte-kit dev",
		"build": "npx rimraf public && svelte-kit build --verbose",
		"preview": "svelte-kit preview",
		"lint": "prettier --check --plugin-search-dir=. . && eslint --ignore-path .gitignore .",
		"format": "prettier --write --plugin-search-dir=. ."
	},

Firebase versions: "firebase": "^9.0.0", "firebase-admin": "^9.11.0"

I also get this in my console when running firebase deploy

12:53:27 PM [vite-plugin-svelte] The following packages did not export their `package.json` file so we could not check the "svelte" field. If you had difficulties importing svelte components from a package, then please contact the author and ask them to export the package.json file.
- @firebase/firestore

[Bandit]

This is line 133860 of ssrServer - looks like some Firebase code to me

var require2 = import_module.default.createRequire(import_meta.url);

This is further up in the file, and it looks like import_meta is never written to after initiation here because there's only 2 references of it according to my search (which explains the URL error because import_meta.url is undefined)

// node_modules/@firebase/firestore/dist/index.node.mjs
[...]
var import_meta = {};

/EDIT dug out the code for this part of the index.node.mjs file above (L14241), and it's preceded by these comments:

// This is a hack fix for Node ES modules to use `require`.
// @ts-ignore To avoid using `--module es2020` flag.
const require = module.createRequire(import.meta.url);

Could be something there? Some sort of incompatibility with the way ssrServer is executed or something?

[jthegedus]

12:53:27 PM [vite-plugin-svelte] The following packages did not export their package.json file so we could not check the "svelte" field. If you had difficulties importing svelte components from a package, then please contact the author and ask them to export the package.json file.
-@firebase/firestore

You can ignore this.

---

I would try bumping your Cloud Function Node.js version from 14 to 16 and seeing what happens.

---

If that doesn't work I have this to say: using firestore in load can have issues, as I said before you can move it to a Kit endpoint and mitigate most of the issues.

If the error is coming from firebase/firestore package, my recommendation is to not use the firebase lib for firestore in code that can run on the server. load can run in SSR and on the Client. While the firebase@9.0.0 lib can be used in Node.js environments as well as web, it should only be used when on on the web. Firestore data fetch should be performed by using firebase-admin@10.0.0 on the server side.

So in your load function, you should first check if the execution environment is Server or Client and use firebase-admin to get firestore on the server & firebase to get firestore on the client (web).

This matters because the packages are different with firebase-admin specifically targeting Node.js envs. The authentication for each of the libs is also different:

• server: firebase-admin > get firestore instance > execute query > bypasses security rules because it is run on server using the special admin lib > hits firestore database
• client: firebase > get firestore instance > execute query > pass through security rules with user auth token > hits firestore database

Using the Client lib on the Server raises questions around whether the query bypasses the sec rules on the Firestore server or not, and also, if it does go through the security rules, how can the SSR environment know which user initiated the request.

Given these factors, I recommend NOT using the Client (web) firestore libs on the Server and switching the use of the firebase vs firebase-admin packages depending on the Kit $app/env browser value (I have found I need to dynamically import these libs and that code splitting cannot accurately remove each one from the other bundle, causing errors).

It is possible to have an authenticated user pass their auth token to the initial page call that executes the page SSR, so the web lib can be used in SSR envs and know which users initiated the request and perform security rules appropriately, however this wiring needs to be done manually within the Kit request lifecycle and requires handling Firebase auth tokens yourself, which defeats the point of the Firebase.

Server data fetch on SSR load should be of data that does not require an auth token using the firebase-admin lib.

Client data fetch on client load should use the firebase lib.

Hope this helps you debug.

[Bandit]

@jthegedus thanks for your advice.

What you say about Firestore makes a lot of sense, and keen to do that (though it will obviously take quite some time), but the error I'm now receiving didn't happen before upgrading to Sveltekit v1.0.0-next.192 and svelte-adapter-firebase 0.13.1 so doesn't that imply some sort of bug / issue there?

Especially given my ssrServer function literally can't run due to some sort of compiled code error as per the above (where import_meta.url is undefined)?

---

I tried bumping to Node v16 but the only thing that changes is now I don't get given a reason for the crash in the logs: