Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

0.33.3 Throws TypeError: A string was expected #4095

Open
Enngage opened this issue May 7, 2024 · 31 comments
Open

0.33.3 Throws TypeError: A string was expected #4095

Enngage opened this issue May 7, 2024 · 31 comments
Milestone

Comments

@Enngage
Copy link

Enngage commented May 7, 2024

Possible bug

Locally everything works fine, but our deployment in Azure (Node 20 LTS, linux) started throwing exceptions such as:

TypeError: A string was expected
2024-05-07T13:07:25.0791187Z 13:07:25 0|abc|     at file:///home/site/wwwroot/dist/abc/server/server.mjs:114:78047
2024-05-07T13:07:25.0812631Z 13:07:25 0|abc|     at new A (file:///home/site/wwwroot/dist/abc/server/polyfills.server.mjs:4:2348)
2024-05-07T13:07:25.0836674Z 13:07:25 0|abc|     at pu.$3e [as _pipeline] (file:///home/site/wwwroot/dist/abc/server/server.mjs:114:78024)
2024-05-07T13:07:25.0857768Z 13:07:25 0|abc|     at pu.v3e [as toBuffer] (file:///home/site/wwwroot/dist/abc/server/server.mjs:114:59649)
2024-05-07T13:07:25.0891412Z 13:07:25 0|abc|     at y3.<anonymous> (file:///home/site/wwwroot/dist/abc/server/server.mjs:160:7917)
2024-05-07T13:07:25.0908622Z 13:07:25 0|abc|     at Generator.next (<anonymous>)
2024-05-07T13:07:25.0924518Z 13:07:25 0|abc|     at file:///home/site/wwwroot/dist/abc/server/chunk-OLFPQWYA.mjs:2:1616
2024-05-07T13:07:25.0946588Z 13:07:25 0|abc|     at new A (file:///home/site/wwwroot/dist/abc/server/polyfills.server.mjs:4:2348)
2024-05-07T13:07:25.0995308Z 13:07:25 0|abc|     at L (file:///home/site/wwwroot/dist/abc/server/chunk-OLFPQWYA.mjs:2:1436)
2024-05-07T13:07:25.1019244Z 13:07:25 0|abc|     at y3.convertImageToWebpAsync (file:///home/site/wwwroot/dist/abc/server/server.mjs:160:7553)

Everything works perfectly fine with 0.33.0. The dependency version change is the only difference in deployments. It works perfectly fine locally even with latest version 0.33.0.

Not sure if I can provide more information since it's a bit unfortunate error and only reproducable within Azure.

@Enngage Enngage added the triage label May 7, 2024
@lovell lovell added question and removed triage labels May 7, 2024
@lovell
Copy link
Owner

lovell commented May 7, 2024

Nothing in the stack trace provided here is from within sharp itself.

If you still require help you'll need to provide some sample code that allows someone else to reproduce or at least better understand how you're trying to use sharp.

@Enngage
Copy link
Author

Enngage commented May 7, 2024

Thanks for a quick response.

Yeah, I know, it's minimized code (sorry). Sadly I can't get full trace from our deployed instance.

All calls to sharp fail, this is an example of one such instance:

import sharp from 'sharp';

// inputImage = Buffer

const webpImage = await sharp(inputImage)
    .webp({
	quality: 100,
	alphaQuality: 100,
	lossless: true,
})
.toBuffer();

The exception happens right here (I've added some logs right before & after the code above to be sure)

Nothing fancy. As I mentioned, it works perfectly fine locally, and in Azure on version 0.33.0. I did a test deploy by only changing 0.33.0 to 0.33.3 and this error started happening again. Changing back fixes it.

Really not sure what's going on and I'm fine with staying on older version, but figured I could post it here just in case others encounter(ed) same issue.

@lovell
Copy link
Owner

lovell commented May 7, 2024

polyfills.server.mjs

My best guess would be that logic in your bundling step is erroneously polyfilling (i.e. monkey-patching) something fundamental, perhaps Object or Buffer.

Please can you try 0.33.1 and 0.33.2 to help narrow this down.

@Enngage
Copy link
Author

Enngage commented May 9, 2024

I don't think the bundling should be a problem here as it works with older versions. It's an Angular 17 SSR app (just FYI). This code is executed strictly on server (Node 20 - LTS on Linux).

I did several new releases one by one:

0.30.0 -> Works
0.30.1 -> Works
0.30.2 -> Works
0.30.3 -> Fails
Downgrade to 0.30.2 -> Works again

The only changes between these releases are sharp versions in package.json.

@lovell
Copy link
Owner

lovell commented May 9, 2024

It's an Angular 17 SSR app... This code is executed strictly on server (Node 20 - LTS on Linux).

My best guess remains that something is being polyfilled in an unexpected or broken way. Can you prevent Angular from generating polyfills for the server side code? If not, what objects and functions are being polyfilled by dist/abc/server/polyfills.server.mjs?

Looking at the changes to sharp between v0.33.2 and v0.33.3 there is 7bc74fe that introduces the use of structuredClone, which is the sort of thing that I might imagine could be pollyfilled in a non-standard way.

@rnenjoy
Copy link

rnenjoy commented May 10, 2024

Getting the same error on my apollo server.

await sharp(tmpFilePath) .resize({ width: 1500, height: 1500, fit: 'inside' }) .jpeg({ quality: 90 }) .toFile(tmpFileResizedPath)

both variables are paths /srv/http/blabla.jpg etc

error: TypeError: A string was expected at /srv/http/dev-net23/backend/node_modules/.pnpm/[email protected]/node_modules/sharp/lib/output.js:1534:15 at new Promise (<anonymous>) at Sharp._pipeline (/srv/http/dev-net23/backend/node_modules/.pnpm/[email protected]/node_modules/sharp/lib/output.js:1533:14) at Sharp.toFile (/srv/http/dev-net23/backend/node_modules/.pnpm/[email protected]/node_modules/sharp/lib/output.js:90:17) at Object.uploadAppWorkshopPicture (file:///srv/http/dev-net23/backend/src/modules/app/resolvers.js:35:6) { path: [ 'uploadAppWorkshopPicture' ], locations: [ { line: 2, column: 3 } ], extensions: [Object: null prototype] {}

@lovell
Copy link
Owner

lovell commented May 10, 2024

Please can you create a standalone repo with minimal dependencies that allows someone else to reproduce.

@rnenjoy
Copy link

rnenjoy commented May 10, 2024

Please can you create a standalone repo with minimal dependencies that allows someone else to reproduce.

Yes i will try to do that! I'm not that experienced in repos. Do i do that on github or some pastebin variant?

@lovell
Copy link
Owner

lovell commented May 10, 2024

Do i do that on github

Yes please, that would be ideal, I'd expect to see a package.json file with dependencies, a single test image and a single JS file that someone else can use to reproduce.

@lovell
Copy link
Owner

lovell commented May 16, 2024

@Enngage @rnenjoy Are either of you able to create a minimal repo that allows someone else to reproduce?

@rnenjoy
Copy link

rnenjoy commented May 18, 2024

@Enngage @rnenjoy Are either of you able to create a minimal repo that allows someone else to reproduce?

Sorry for late reply. I was working on the minimal repo but didnt mange to get the error. Then i saw that i now used 0.33.4. And the problem seems to be gone? So no need to work any more on it? :)

@lovell
Copy link
Owner

lovell commented Jun 1, 2024

@Enngage Were you able to make any progress with this?

@Enngage
Copy link
Author

Enngage commented Jun 1, 2024

Hey @lovell,

I've tried upgrading to 0.33.4, but that didn't help in my case. Honestly, the only way I can reproduce it is by pushing this to Azure (Node 20 LTS, linux). Even the simplest example fails there, no setup needed, just try to include import and call sharp and it fails.

@lovell
Copy link
Owner

lovell commented Jun 1, 2024

Thanks for the update.

Even the simplest example fails there, no setup needed

When you say "simplest example", is this with sharp as the only dependency, or are you still using Angular and therefore at the mercy of its polyfills.server.mjs?

Sadly, without a complete, minimal repo declaring the dependencies, code and build steps that will allow someone else to reproduce, there's not a lot anyone can do to help.

@rnenjoy
Copy link

rnenjoy commented Jun 1, 2024

Maybe my started working with 34 is because I upgraded to node 22 aswell ?

@lovell
Copy link
Owner

lovell commented Jul 2, 2024

@Enngage Were you able to make any progress with this, e.g. providing a minimal repo that allows someone else to reproduce?

@gkTim
Copy link

gkTim commented Jul 26, 2024

Had the same problem (node 20) in my case it helped to update to node 22.

@lovell
Copy link
Owner

lovell commented Jul 27, 2024

All the signs here point to an Angular-provided server-side polyfill that is incompatible with Node.js 20 and that you're seeing the effects of this via sharp.

I'll close this for now, but please feel free to re-open with a minimal set of code/dependencies that allows someone else to reproduce.

@lovell lovell closed this as completed Jul 27, 2024
@VapidLinus
Copy link

VapidLinus commented Sep 2, 2024

I'm not able to provide a reproduce repo right now, but I just want to mention for any future stumblers that we started having this problem with NextJS and Node 20 with 0.33.5 as well and downgrading to either 0.30.2 or 0.33.4 both solve the issue. Upgrading to Node 22 did not solve it. Only happens on our Linux machines and can't reproduce on Windows.

@donmccurdy
Copy link

I've been getting the same error since upgrading to v0.33.5, downgrading to v0.33.4 resolves the issue.

Details:

  • Node.js v20.10.0
  • MacOS 14.6.1

The errors are occurring in a test suite executed with tsx and ava. No intentional polyfills, but it's possible something else is happening internally. I've so far been unable to create a simple reproduction, other than cloning https://github.com/donmccurdy/glTF-Transform and running...

yarn && yarn dist
npx ava packages/functions/test/texture-compress.test.ts --no-worker-threads

... but I realize it's a large project and not a good minimal example, I'll post a better example if I can figure out how... so far my simpler tests have not recreated the issue.

  Rejected promise returned by test. Reason:

  TypeError {
    message: 'A string was expected',
  }

  TypeError: A string was expected
      at /Users/donmccurdy/Documents/Projects/glTF-Transform/packages/functions/node_modules/sharp/lib/output.js:1534:15
      at new Promise (<anonymous>)
      at Sharp._pipeline (/Users/donmccurdy/Documents/Projects/glTF-Transform/packages/functions/node_modules/sharp/lib/output.js:1533:14)
      at Sharp.toBuffer (/Users/donmccurdy/Documents/Projects/glTF-Transform/packages/functions/node_modules/sharp/lib/output.js:162:15)
      at savePixelsInternal (/Users/donmccurdy/Documents/Projects/glTF-Transform/packages/functions/node_modules/ndarray-pixels/src/node-save-pixels.ts:29:4)
      at savePixels (/Users/donmccurdy/Documents/Projects/glTF-Transform/packages/functions/node_modules/ndarray-pixels/src/index.ts:53:9)
      at <anonymous> (/Users/donmccurdy/Documents/Projects/glTF-Transform/packages/functions/test/texture-compress.test.ts:216:19)
      at Test.callFn (file:///Users/donmccurdy/Documents/Projects/glTF-Transform/node_modules/ava/lib/test.js:525:26)
      at Test.run (file:///Users/donmccurdy/Documents/Projects/glTF-Transform/node_modules/ava/lib/test.js:534:33)
      at Runner.runSingle (file:///Users/donmccurdy/Documents/Projects/glTF-Transform/node_modules/ava/lib/runner.js:281:33)

@lovell
Copy link
Owner

lovell commented Sep 7, 2024

@donmccurdy Thanks for the extra info, I can't reproduce locally but I do notice your monorepo has the potential for multiple versions of sharp to be involved:

$ yarn why sharp
├─ @gltf-transform/cli@workspace:packages/cli
│  └─ sharp@npm:0.33.4 (via npm:~0.33.4)
│
└─ ndarray-pixels@npm:4.1.0
   └─ sharp@npm:0.33.4 (via npm:^0.33.4)

I wonder if the wrong binary is sometimes being selected at dlopen time, perhaps due to package hoisting or require resolution? I guess the rather generic "A string was expected" message is the sort of thing that an error within error handling could make, and might be masking the real problem.

I tried upgrading only the packages/cli version of sharp and then used LD_DEBUG=files to verify that the correct chain of shared library files were dlopen-ed when running ava.

file=/glTF-Transform/node_modules/ndarray-pixels/node_modules/@img/sharp-linux-x64/lib/sharp-linux-x64.node [0];  dynamically loaded by /node/v20.17.0/bin/node [0]
file=libvips-cpp.so.42 [0];  needed by /glTF-Transform/node_modules/ndarray-pixels/node_modules/@img/sharp-linux-x64/lib/sharp-linux-x64.node [0]
file=libresolv.so.2 [0];  needed by /glTF-Transform/node_modules/ndarray-pixels/node_modules/@img/sharp-linux-x64/lib/../../sharp-libvips-linux-x64/lib/libvips-cpp.so.42 [0]

@danielobima
Copy link

Hello, I encountered the same issue with [email protected] when installed in a docker container node:18-alpine . I can provide a more elaborate example if required. I am using it in a next js standalone build

Code snippet:

 const image = sharp(file);
        const { data } = await image
          .resize(parseInt(width), parseInt(height), {
            fit: "cover",
          })
          .toBuffer({ resolveWithObject: true });

Docker file

# Start Dockerfile
ARG VERSION=18-alpine
ARG DIR=usr/src/app

FROM node:${VERSION} AS builder
# redeclare ARG because ARG not in build environment
ARG DIR 
WORKDIR /${DIR}
COPY package.json .
COPY package-lock.json .
RUN npm i
COPY . .
RUN npm run build

FROM node:${VERSION} AS runner

# redeclare ARG because ARG not in build environment
ARG DIR
WORKDIR /${DIR}

# Packages omitted from the standalone build
RUN npm i mysql2 tedious [email protected]

COPY --from=builder /${DIR}/public ./public
COPY --from=builder /${DIR}/styles ./styles
COPY --from=builder /${DIR}/.next/standalone .
COPY --from=builder /${DIR}/.next/static ./.next/static


EXPOSE 3000
ENTRYPOINT ["node", "server.js"]

error:

TypeError: A string was expected
    at /usr/src/app/node_modules/sharp/lib/output.js:1534:15
    at new Promise (<anonymous>)
    at Sharp._pipeline (/usr/src/app/node_modules/sharp/lib/output.js:1533:14)
    at Sharp.toBuffer (/usr/src/app/node_modules/sharp/lib/output.js:162:15)
    at handler (/usr/src/app/.next/server/pages/api/tasks/uploadFile.js:114:16)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async Object.apiResolver (/usr/src/app/node_modules/next/dist/server/api-utils/node.js:363:9)
    at async NextNodeServer.runApi (/usr/src/app/node_modules/next/dist/server/next-server.js:474:9)
    at async Object.fn (/usr/src/app/node_modules/next/dist/server/next-server.js:736:37)
    at async Router.execute (/usr/src/app/node_modules/next/dist/server/router.js:252:36)

I fixed it by downgrading to [email protected]

@lovell
Copy link
Owner

lovell commented Sep 13, 2024

I think this might relate to having 2 different patch versions of sharp in the same installation tree where they depend on 2 different patch versions of libvips. I've opened libvips/libvips#4141 to discuss possible solutions.

@lovell lovell reopened this Sep 13, 2024
@donmccurdy
Copy link

Thanks @lovell! I quickly tried yarn why sharp and it didn't immediately appear that I had two installations, but I have been traveling and didn't do a very thorough check. I'll try to look closer soon.

A different but related possibility would be that Sharp or a dependency uses both CJS and ESM exports, and we get duplicate copies of a dependency from the same version ('dual package hazard') ... not sure if that could cause similar issues.

@kleisauke
Copy link
Contributor

[...]
    at /Users/donmccurdy/Documents/Projects/glTF-Transform/packages/functions/node_modules/sharp/lib/output.js:1534:15
[...]
    at /usr/src/app/node_modules/sharp/lib/output.js:1534:15
[...]

Based on the stack traces above, both appear to be from v0.33.4 (or earlier), as line 1534 in v0.33.5 is a comment. See:

sharp.pipeline(this.options, (err, data, info) => {
(v0.33.4)
// output=promise, input=file/buffer
(v0.33.5)

@kleisauke
Copy link
Contributor

Could someone provide a standalone reproducer for this? I have tried this to no avail:

$ git clone https://github.com/donmccurdy/glTF-Transform.git
$ cd glTF-Transform/
$ git checkout a52dfc6804a03a3e2c11e24802896f08126853bf~1
HEAD is now at bdbecdd5 chore(deps): update devdependencies (non-major) (#1494)
$ sudo corepack enable
$ yarn && yarn dist
[...]
 Lerna (powered by Nx)   Successfully ran target dist for 7 projects (16s)
$ npx ava packages/functions/test/texture-compress.test.ts --no-worker-threads

  ✔ unknown format
  ✔ size increase
  ✔ original formats
  ✔ jpeg
  ✔ png
  ✔ webp
  ✔ incompatible format
  ✔ excluded slots
  ✔ jpeg / jpg
  ✔ resize - sharp
  ✔ fallback to ndarray-pixels
  ✔ resize - ndarray-pixels


  12 tests passed
$ yarn why sharp
├─ @gltf-transform/cli@workspace:packages/cli
│  └─ sharp@npm:0.33.5 (via npm:~0.33.4)

└─ ndarray-pixels@npm:4.1.0
   └─ sharp@npm:0.33.5 (via npm:^0.33.4)

@kleisauke
Copy link
Contributor

I was also unable to reproduce this issue using LD_PRELOAD. I tested this by preloading the libvips binaries from sharp v0.33.5 in sharp v0.33.4, and vice versa.

Details
$ mkdir sharp-test
$ cd sharp-test
$ npm init -y
$ npm install [email protected]
$ npm why @img/sharp-libvips-linux-x64
node_modules/@img/sharp-libvips-linux-x64
  optional @img/sharp-libvips-linux-x64@"1.0.4" from @img/[email protected]
  node_modules/@img/sharp-linux-x64
    optional @img/sharp-linux-x64@"0.33.5" from [email protected]
    node_modules/sharp
      sharp@"^0.33.5" from the root project
  optional @img/sharp-libvips-linux-x64@"1.0.4" from [email protected]
  node_modules/sharp
    sharp@"^0.33.5" from the root project
$ node -e "require('sharp')({text: { text: 'test' }}).toFile('x.png')"
$ mkdir subdir
$ cp x.png subdir/
$ cd subdir
$ npm init -y
$ npm install [email protected]
$ npm why @img/sharp-libvips-linux-x64
node_modules/@img/sharp-libvips-linux-x64
  optional @img/sharp-libvips-linux-x64@"1.0.2" from @img/[email protected]
  node_modules/@img/sharp-linux-x64
    optional @img/sharp-linux-x64@"0.33.4" from [email protected]
    node_modules/sharp
      sharp@"^0.33.4" from the root project
  optional @img/sharp-libvips-linux-x64@"1.0.2" from [email protected]
  node_modules/sharp
    sharp@"^0.33.4" from the root project
$ LD_PRELOAD=../node_modules/@img/sharp-libvips-linux-x64/lib/libvips-cpp.so.42 node -p "require('sharp')('x.png').toFile('out.png')"
Promise {
  {
    format: 'png',
    width: 22,
    height: 9,
    channels: 3,
    premultiplied: false,
    size: 390
  }
}
$ LD_DEBUG=bindings LD_PRELOAD=../node_modules/@img/sharp-libvips-linux-x64/lib/libvips-cpp.so.42 node -e "require('sharp')('x.png').toFile('out.png')" 2>&1 | grep "calling fini: .*libvips-cpp.so.42"
     10159:	calling fini: ../node_modules/@img/sharp-libvips-linux-x64/lib/libvips-cpp.so.42 [0]
$ cd ../
$ LD_PRELOAD=subdir/node_modules/@img/sharp-libvips-linux-x64/lib/libvips-cpp.so.42 node -p "require('sharp')('x.png').toFile('out.png')"
Promise {
  {
    format: 'png',
    width: 22,
    height: 9,
    channels: 3,
    premultiplied: false,
    size: 390
  }
}
$ LD_DEBUG=bindings LD_PRELOAD=subdir/node_modules/@img/sharp-libvips-linux-x64/lib/libvips-cpp.so.42 node -e "require('sharp')('x.png').toFile('out.png')" 2>&1 | grep "calling fini: .*libvips-cpp.so.42"
     10227:	calling fini: subdir/node_modules/@img/sharp-libvips-linux-x64/lib/libvips-cpp.so.42 [0]

It would be great if someone could verify this on macOS. Replace LD_PRELOAD= with DYLD_INSERT_LIBRARIES= and LD_DEBUG=bindings with DYLD_PRINT_BINDINGS=1. Additionally, the grep command will also likely need to be adjusted.

@kleisauke
Copy link
Contributor

I could reproduce this by applying this patch:

--- a/lib/output.js
+++ b/lib/output.js
@@ -159,7 +159,7 @@ function toBuffer (options, callback) {
   } else if (this.options.resolveWithObject) {
     this.options.resolveWithObject = false;
   }
-  this.options.fileOut = '';
+  this.options.fileOut = undefined;
   const stack = Error();
   return this._pipeline(is.fn(options) ? options : callback, stack);
 }
$ node -e "require('sharp')('x.png').toBuffer()"
/home/kleisauke/sharp-test/node_modules/sharp/lib/output.js:1536
        sharp.pipeline(this.options, (err, data, info) => {
              ^

TypeError: A string was expected
    at /home/kleisauke/sharp-test/node_modules/sharp/lib/output.js:1536:15
    at new Promise (<anonymous>)
    at Sharp._pipeline (/home/kleisauke/sharp-test/node_modules/sharp/lib/output.js:1535:14)
    at Sharp.toBuffer (/home/kleisauke/sharp-test/node_modules/sharp/lib/output.js:164:15)
    at [eval]:1:27
    at runScriptInThisContext (node:internal/vm:209:10)
    at node:internal/process/execution:118:14
    at [eval]-wrapper:6:24
    at runScript (node:internal/process/execution:101:62)
    at evalScript (node:internal/process/execution:136:3)

Node.js v22.9.0

So, this sounds like a bundling/minification issue.

@lovell
Copy link
Owner

lovell commented Oct 4, 2024

I can't reproduce the exact problem, but here's a way to "force" a version mismatch that someone else might be able to build upon.

package.json

{
  "dependencies": {
    "fast-average-color-node": "3.1.0",
    "sharp": "0.33.4"
  }
}

index.js

require("fast-average-color-node");
const sharp = require("sharp");

console.log(sharp.versions.vips);
console.log(require("@img/sharp-linux-x64/sharp.node").libvipsVersion());

Running node index produces:

8.15.2
8.15.3

Running LD_DEBUG=files node index 2>&1 | grep init produces:

     59847:	calling init: /lib64/ld-linux-x86-64.so.2
     59847:	calling init: /lib/x86_64-linux-gnu/libc.so.6
     59847:	calling init: /lib/x86_64-linux-gnu/libpthread.so.0
     59847:	calling init: /lib/x86_64-linux-gnu/libgcc_s.so.1
     59847:	calling init: /lib/x86_64-linux-gnu/libm.so.6
     59847:	calling init: /lib/x86_64-linux-gnu/libstdc++.so.6
     59847:	calling init: /lib/x86_64-linux-gnu/libdl.so.2
     59847:	initialize program: node
     59847:	calling init: /lib/x86_64-linux-gnu/libresolv.so.2
     59847:	calling init: /redacted/node_modules/fast-average-color-node/node_modules/@img/sharp-linux-x64/lib/../../sharp-libvips-linux-x64/lib/libvips-cpp.so.42
     59847:	calling init: /redacted/node_modules/fast-average-color-node/node_modules/@img/sharp-linux-x64/lib/sharp-linux-x64.node
     59847:	calling init: /redacted/node_modules/@img/sharp-linux-x64/lib/sharp-linux-x64.node

This demonstrates sharp v0.33.4 using libvips v8.15.3 from inside the fast-average-color-node dependency instead of its own v8.15.2 (due, I think, to the SONAME values being the same).

@kleisauke
Copy link
Contributor

I think it matches on filename rather than the SONAME in this case:

$ LD_DEBUG=libs node index 2>&1 | grep "libvips-cpp.so.42"
      9746:	find library=libvips-cpp.so.42 [0]; searching
      9746:	  trying file=/redacted/node_modules/fast-average-color-node/node_modules/@img/sharp-linux-x64/lib/../../sharp-libvips-linux-x64/lib/glibc-hwcaps/x86-64-v4/libvips-cpp.so.42
      9746:	  trying file=/redacted/node_modules/fast-average-color-node/node_modules/@img/sharp-linux-x64/lib/../../sharp-libvips-linux-x64/lib/glibc-hwcaps/x86-64-v3/libvips-cpp.so.42
      9746:	  trying file=/redacted/node_modules/fast-average-color-node/node_modules/@img/sharp-linux-x64/lib/../../sharp-libvips-linux-x64/lib/glibc-hwcaps/x86-64-v2/libvips-cpp.so.42
      9746:	  trying file=/redacted/node_modules/fast-average-color-node/node_modules/@img/sharp-linux-x64/lib/../../sharp-libvips-linux-x64/lib/libvips-cpp.so.42
      9746:	calling init: /redacted/node_modules/fast-average-color-node/node_modules/@img/sharp-linux-x64/lib/../../sharp-libvips-linux-x64/lib/libvips-cpp.so.42
      9746:	calling fini: /redacted/node_modules/fast-average-color-node/node_modules/@img/sharp-linux-x64/lib/../../sharp-libvips-linux-x64/lib/libvips-cpp.so.42 [0]

Since fast-average-color-node is required first it will use libvips v8.15.3, if you reorder the require()'s then this mismatch won't happen.

I recently started distributing the pyvips-binary PyPI package, which automagically renames the shared library to libvips-<unique>.so.42 as part of the auditwheel process.
pypa/auditwheel#24
https://github.com/pypa/auditwheel/blob/dd3df250063f520950f7f7c3e30544c701b5ec9c/src/auditwheel/repair.py#L131-L138

On Windows, a similar approach is followed:
https://github.com/adang1345/delvewheel/blob/master/README.md#name-mangling

On macOS, it assigns a unique install name id within the Python namespace:
matthew-brett/delocate@dabce49
https://github.com/matthew-brett/delocate/blob/b0c37814ae51a0f8ccef0678b4d61173f9a59919/delocate/tools.py#L685

I think sharp could implement a similar approach to avoid libvips version mismatches, though this would mean potentially having two instances of libvips running in the same process.

@kleisauke
Copy link
Contributor

Oh, I just noticed this for macOS:

If two libraries have the same install_name_id when loaded into the process, then OSX will raise an error unless their compatibility number matches.
From: https://github.com/matthew-brett/delocate/blob/b0c37814ae51a0f8ccef0678b4d61173f9a59919/Changelog.md?plain=1#L297C11-L299C11

libvips increases the compatibility version only for minor versions, not for patch versions, see:
https://github.com/libvips/libvips/blob/9d4c4f9105c8171f92c21c93155aebdc7024958f/meson.build#L30

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

8 participants