gunzipSync failing (0 gzip length) on some npm package tarballs

[andrewbranch]

I’ve found three npm packages that fail to gunzip with fflate (the resulting length is 0), but work with Node.js’s zlib. Here’s a full repro:

// index.mjs
import { gunzipSync as zlibGunzipSync } from "zlib";
import { gunzipSync } from "fflate";

const brokenPackages = [
  "https://registry.npmjs.org/openurl/-/openurl-1.0.2.tgz",
  "https://registry.npmjs.org/is/-/is-0.0.7.tgz",
  "https://registry.npmjs.org/youtube/-/youtube-0.0.5.tgz",
];

for (const url of brokenPackages) {
  const tarballData = new Uint8Array(
    await fetch(url).then((r) => r.arrayBuffer())
  );

  console.log("fflate", gunzipSync(tarballData).length);
  console.log("zlib", zlibGunzipSync(tarballData).length);
}

[101arrowz]

That would be because those particular package tarballs have a bunch of zero padding at the end (for some reason?), and fflate expects a footer to be in the right position. I don't think that's actually spec compliant - GZIP supports concatenated streams (i.e. it would be OK to have a second GZIP header right after the first one and another full stream) but I don't think it's supposed to have padding. In any case you should be able to resolve this with the streaming API, which doesn't look at any footers and supports concatenated GZIP streams (if any of those are on NPM too).

const chunks = [];
new fflate.Gunzip(chunk => chunks.push(chunk)).push(tarballData, true);
console.log("fflate streaming", Buffer.concat(chunks).length);

[andrewbranch]

Thanks for the workaround. That’s sufficient for me, but FWIW neither macOS nor zlib complained about extracting those archives, so perhaps it’s worth supporting somehow. I wish I had more info about how these packages were generated, but I found them as a result of automated analysis on many thousands of npm packages. 🤷