potential node-gyp options parsing issue when building native modules

[achou11]

Description:

For better-sqlite3, receive the following error when running ../../scripts/rebuild-android.sh better-sqlite3 arm:

/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/libtool: file: Release/obj.target/sqlite3/gen/sqlite3/sqlite3.o is not an object file (not allowed in a library)

In general, this process shouldn’t be using XCode’s libtool but instead should be using the libtool provided by the Android NDK.

Debugging

The outcomes of running node-gyp rebuild --release (armeabi-v7a) are slightly different:

Mobile via nodejs-mobile-react-native gradle config (Node 12, npm 6), which succeeds:

> better-sqlite3@7.6.2 build-release /Users/andrewchou/GitHub/digidem/mapeo-mobile/android/build/nodejs-native-assets-temp-build/nodejs-native-assets-armeabi-v7a/nodejs-project/node_modules/better-sqlite3
> node-gyp rebuild --release

  touch ba23eeee118cd63e16015df367567cb043fed872.intermediate
  LD_LIBRARY_PATH=/Users/andrewchou/GitHub/digidem/mapeo-mobile/android/build/nodejs-native-assets-temp-build/nodejs-native-assets-armeabi-v7a/nodejs-project/node_modules/better-sqlite3/build/Release/lib.host:/Users/andrewchou/GitHub/digidem/mapeo-mobile/android/build/nodejs-native-assets-temp-build/nodejs-native-assets-armeabi-v7a/nodejs-project/node_modules/better-sqlite3/build/Release/lib.target:$LD_LIBRARY_PATH; export LD_LIBRARY_PATH; cd ../deps; mkdir -p /Users/andrewchou/GitHub/digidem/mapeo-mobile/android/build/nodejs-native-assets-temp-build/nodejs-native-assets-armeabi-v7a/nodejs-project/node_modules/better-sqlite3/build/Release/obj/gen/sqlite3; node copy.js "/Users/andrewchou/GitHub/digidem/mapeo-mobile/android/build/nodejs-native-assets-temp-build/nodejs-native-assets-armeabi-v7a/nodejs-project/node_modules/better-sqlite3/build/Release/obj/gen/sqlite3" ""
  touch Release/obj.target/deps/locate_sqlite3.stamp
  /Users/andrewchou/Library/Android/sdk/ndk/21.4.7075529/toolchains/llvm/prebuilt/darwin-x86_64/bin/armv7a-linux-androideabi21-clang '-DNODE_GYP_MODULE_NAME=sqlite3' '-DUSING_UV_SHARED=1' '-DUSING_V8_SHARED=1' '-DV8_DEPRECATION_WARNINGS=1' '-DV8_DEPRECATION_WARNINGS' '-DV8_IMMINENT_DEPRECATION_WARNINGS' '-D_LARGEFILE_SOURCE' '-D_FILE_OFFSET_BITS=64' '-D__STDC_FORMAT_MACROS' '-DOPENSSL_NO_PINSHARED' '-DOPENSSL_THREADS' '-DSQLITE_DQS=0' '-DSQLITE_LIKE_DOESNT_MATCH_BLOBS' '-DSQLITE_THREADSAFE=2' '-DSQLITE_USE_URI=0' '-DSQLITE_DEFAULT_MEMSTATUS=0' '-DSQLITE_OMIT_DEPRECATED' '-DSQLITE_OMIT_GET_TABLE' '-DSQLITE_OMIT_TCL_VARIABLE' '-DSQLITE_OMIT_PROGRESS_CALLBACK' '-DSQLITE_OMIT_SHARED_CACHE' '-DSQLITE_TRACE_SIZE_LIMIT=32' '-DSQLITE_DEFAULT_CACHE_SIZE=-16000' '-DSQLITE_DEFAULT_FOREIGN_KEYS=1' '-DSQLITE_DEFAULT_WAL_SYNCHRONOUS=1' '-DSQLITE_ENABLE_MATH_FUNCTIONS' '-DSQLITE_ENABLE_DESERIALIZE' '-DSQLITE_ENABLE_COLUMN_METADATA' '-DSQLITE_ENABLE_UPDATE_DELETE_LIMIT' '-DSQLITE_ENABLE_STAT4' '-DSQLITE_ENABLE_FTS3_PARENTHESIS' '-DSQLITE_ENABLE_FTS3' '-DSQLITE_ENABLE_FTS4' '-DSQLITE_ENABLE_FTS5' '-DSQLITE_ENABLE_JSON1' '-DSQLITE_ENABLE_RTREE' '-DSQLITE_ENABLE_GEOPOLY' '-DSQLITE_INTROSPECTION_PRAGMAS' '-DSQLITE_SOUNDEX' '-DHAVE_STDINT_H=1' '-DHAVE_INT8_T=1' '-DHAVE_INT16_T=1' '-DHAVE_INT32_T=1' '-DHAVE_UINT8_T=1' '-DHAVE_UINT16_T=1' '-DHAVE_UINT32_T=1' '-D_GLIBCXX_USE_C99_MATH' '-DNDEBUG' -I/Users/andrewchou/GitHub/digidem/mapeo-mobile/node_modules/nodejs-mobile-react-native/android/libnode/include/node -I/Users/andrewchou/GitHub/digidem/mapeo-mobile/node_modules/nodejs-mobile-react-native/android/libnode/src -I/Users/andrewchou/GitHub/digidem/mapeo-mobile/node_modules/nodejs-mobile-react-native/android/libnode/deps/openssl/config -I/Users/andrewchou/GitHub/digidem/mapeo-mobile/node_modules/nodejs-mobile-react-native/android/libnode/deps/openssl/openssl/include -I/Users/andrewchou/GitHub/digidem/mapeo-mobile/node_modules/nodejs-mobile-react-native/android/libnode/deps/uv/include -I/Users/andrewchou/GitHub/digidem/mapeo-mobile/node_modules/nodejs-mobile-react-native/android/libnode/deps/zlib -I/Users/andrewchou/GitHub/digidem/mapeo-mobile/node_modules/nodejs-mobile-react-native/android/libnode/deps/v8/include -I./Release/obj/gen/sqlite3  -fPIC -Wall -Wextra -Wno-unused-parameter -std=c99 -w -O3 -O3 -fno-omit-frame-pointer -fPIC  -MMD -MF ./Release/.deps/Release/obj.target/sqlite3/gen/sqlite3/sqlite3.o.d.raw   -c -o Release/obj.target/sqlite3/gen/sqlite3/sqlite3.o Release/obj/gen/sqlite3/sqlite3.c
  rm -f Release/obj.target/deps/sqlite3.a && /Users/andrewchou/Library/Android/sdk/ndk/21.4.7075529/toolchains/llvm/prebuilt/darwin-x86_64/bin/arm-linux-androideabi-ar crs Release/obj.target/deps/sqlite3.a Release/obj.target/sqlite3/gen/sqlite3/sqlite3.o
  rm -rf "Release/sqlite3.a" && cp -af "Release/obj.target/deps/sqlite3.a" "Release/sqlite3.a"
  /Users/andrewchou/Library/Android/sdk/ndk/21.4.7075529/toolchains/llvm/prebuilt/darwin-x86_64/bin/armv7a-linux-androideabi21-clang++ '-DNODE_GYP_MODULE_NAME=better_sqlite3' '-DUSING_UV_SHARED=1' '-DUSING_V8_SHARED=1' '-DV8_DEPRECATION_WARNINGS=1' '-DV8_DEPRECATION_WARNINGS' '-DV8_IMMINENT_DEPRECATION_WARNINGS' '-D_LARGEFILE_SOURCE' '-D_FILE_OFFSET_BITS=64' '-D__STDC_FORMAT_MACROS' '-DOPENSSL_NO_PINSHARED' '-DOPENSSL_THREADS' '-DBUILDING_NODE_EXTENSION' '-D_GLIBCXX_USE_C99_MATH' '-DNDEBUG' -I/Users/andrewchou/GitHub/digidem/mapeo-mobile/node_modules/nodejs-mobile-react-native/android/libnode/include/node -I/Users/andrewchou/GitHub/digidem/mapeo-mobile/node_modules/nodejs-mobile-react-native/android/libnode/src -I/Users/andrewchou/GitHub/digidem/mapeo-mobile/node_modules/nodejs-mobile-react-native/android/libnode/deps/openssl/config -I/Users/andrewchou/GitHub/digidem/mapeo-mobile/node_modules/nodejs-mobile-react-native/android/libnode/deps/openssl/openssl/include -I/Users/andrewchou/GitHub/digidem/mapeo-mobile/node_modules/nodejs-mobile-react-native/android/libnode/deps/uv/include -I/Users/andrewchou/GitHub/digidem/mapeo-mobile/node_modules/nodejs-mobile-react-native/android/libnode/deps/zlib -I/Users/andrewchou/GitHub/digidem/mapeo-mobile/node_modules/nodejs-mobile-react-native/android/libnode/deps/v8/include -I./Release/obj/gen/sqlite3  -fPIC -Wall -Wextra -Wno-unused-parameter -std=c++14 -O3 -O3 -fno-omit-frame-pointer -fPIC -fno-rtti -fno-exceptions -std=gnu++1y -MMD -MF ./Release/.deps/Release/obj.target/better_sqlite3/src/better_sqlite3.o.d.raw   -c -o Release/obj.target/better_sqlite3/src/better_sqlite3.o ../src/better_sqlite3.cpp
  /Users/andrewchou/Library/Android/sdk/ndk/21.4.7075529/toolchains/llvm/prebuilt/darwin-x86_64/bin/armv7a-linux-androideabi21-clang++ -shared -rdynamic -fPIC  -Wl,-soname=better_sqlite3.node -o Release/obj.target/better_sqlite3.node -Wl,--start-group Release/obj.target/better_sqlite3/src/better_sqlite3.o Release/obj.target/deps/sqlite3.a -Wl,--end-group -llog /Users/andrewchou/GitHub/digidem/mapeo-mobile/node_modules/nodejs-mobile-react-native/android/libnode//bin/armeabi-v7a/libnode.so
  rm -rf "Release/better_sqlite3.node" && cp -af "Release/obj.target/better_sqlite3.node" "Release/better_sqlite3.node"
  /Users/andrewchou/Library/Android/sdk/ndk/21.4.7075529/toolchains/llvm/prebuilt/darwin-x86_64/bin/armv7a-linux-androideabi21-clang '-DNODE_GYP_MODULE_NAME=test_extension' '-DUSING_UV_SHARED=1' '-DUSING_V8_SHARED=1' '-DV8_DEPRECATION_WARNINGS=1' '-DV8_DEPRECATION_WARNINGS' '-DV8_IMMINENT_DEPRECATION_WARNINGS' '-D_LARGEFILE_SOURCE' '-D_FILE_OFFSET_BITS=64' '-D__STDC_FORMAT_MACROS' '-DOPENSSL_NO_PINSHARED' '-DOPENSSL_THREADS' '-DBUILDING_NODE_EXTENSION' '-D_GLIBCXX_USE_C99_MATH' '-DNDEBUG' -I/Users/andrewchou/GitHub/digidem/mapeo-mobile/node_modules/nodejs-mobile-react-native/android/libnode/include/node -I/Users/andrewchou/GitHub/digidem/mapeo-mobile/node_modules/nodejs-mobile-react-native/android/libnode/src -I/Users/andrewchou/GitHub/digidem/mapeo-mobile/node_modules/nodejs-mobile-react-native/android/libnode/deps/openssl/config -I/Users/andrewchou/GitHub/digidem/mapeo-mobile/node_modules/nodejs-mobile-react-native/android/libnode/deps/openssl/openssl/include -I/Users/andrewchou/GitHub/digidem/mapeo-mobile/node_modules/nodejs-mobile-react-native/android/libnode/deps/uv/include -I/Users/andrewchou/GitHub/digidem/mapeo-mobile/node_modules/nodejs-mobile-react-native/android/libnode/deps/zlib -I/Users/andrewchou/GitHub/digidem/mapeo-mobile/node_modules/nodejs-mobile-react-native/android/libnode/deps/v8/include -I./Release/obj/gen/sqlite3  -fPIC -Wall -Wextra -Wno-unused-parameter -O3 -O3 -fno-omit-frame-pointer -fPIC  -MMD -MF ./Release/.deps/Release/obj.target/test_extension/deps/test_extension.o.d.raw   -c -o Release/obj.target/test_extension/deps/test_extension.o ../deps/test_extension.c
  /Users/andrewchou/Library/Android/sdk/ndk/21.4.7075529/toolchains/llvm/prebuilt/darwin-x86_64/bin/armv7a-linux-androideabi21-clang++ -shared -rdynamic -fPIC  -Wl,-soname=test_extension.node -o Release/obj.target/test_extension.node -Wl,--start-group Release/obj.target/test_extension/deps/test_extension.o Release/obj.target/deps/sqlite3.a -Wl,--end-group -llog /Users/andrewchou/GitHub/digidem/mapeo-mobile/node_modules/nodejs-mobile-react-native/android/libnode//bin/armeabi-v7a/libnode.so
  rm -rf "Release/test_extension.node" && cp -af "Release/obj.target/test_extension.node" "Release/test_extension.node"
rm ba23eeee118cd63e16015df367567cb043fed872.intermediate

• notice: rm -f Release/obj.target/deps/sqlite3.a && /Users/andrewchou/Library/Android/sdk/ndk/21.4.7075529/toolchains/llvm/prebuilt/darwin-x86_64/bin/arm-linux-androideabi-ar crs Release/obj.target/deps/sqlite3.a Release/obj.target/sqlite3/gen/sqlite3/sqlite3.o
  • most likey derived from https://github.com/nodejs/node-gyp/blob/main/gyp/pylib/gyp/generator/make.py#L222

Mobile Next via rebuild-android.sh (Node 16, npm 8), which fails:

> better-sqlite3@7.6.2 build-release
> node-gyp rebuild --release

  touch ba23eeee118cd63e16015df367567cb043fed872.intermediate
  LD_LIBRARY_PATH=/Users/andrewchou/GitHub/digidem/mapeo-mobile-next/src/backend/node_modules/better-sqlite3/build/Release/lib.host:/Users/andrewchou/GitHub/digidem/mapeo-mobile-next/src/backend/node_modules/better-sqlite3/build/Release/lib.target:$LD_LIBRARY_PATH; export LD_LIBRARY_PATH; cd ../deps; mkdir -p /Users/andrewchou/GitHub/digidem/mapeo-mobile-next/src/backend/node_modules/better-sqlite3/build/Release/obj/gen/sqlite3; node copy.js "/Users/andrewchou/GitHub/digidem/mapeo-mobile-next/src/backend/node_modules/better-sqlite3/build/Release/obj/gen/sqlite3" ""
  touch Release/obj.target/deps/locate_sqlite3.stamp
  /Users/andrewchou/Library/Android/sdk/ndk/21.4.7075529/toolchains/llvm/prebuilt/darwin-x86_64/bin/armv7a-linux-androideabi21-clang -o Release/obj.target/sqlite3/gen/sqlite3/sqlite3.o Release/obj/gen/sqlite3/sqlite3.c '-DNODE_GYP_MODULE_NAME=sqlite3' '-DUSING_UV_SHARED=1' '-DUSING_V8_SHARED=1' '-DV8_DEPRECATION_WARNINGS=1' '-DV8_DEPRECATION_WARNINGS' '-DV8_IMMINENT_DEPRECATION_WARNINGS' '-D_LARGEFILE_SOURCE' '-D_FILE_OFFSET_BITS=64' '-D__STDC_FORMAT_MACROS' '-DOPENSSL_THREADS' '-DOPENSSL_NO_ASM' '-DSQLITE_DQS=0' '-DSQLITE_LIKE_DOESNT_MATCH_BLOBS' '-DSQLITE_THREADSAFE=2' '-DSQLITE_USE_URI=0' '-DSQLITE_DEFAULT_MEMSTATUS=0' '-DSQLITE_OMIT_DEPRECATED' '-DSQLITE_OMIT_GET_TABLE' '-DSQLITE_OMIT_TCL_VARIABLE' '-DSQLITE_OMIT_PROGRESS_CALLBACK' '-DSQLITE_OMIT_SHARED_CACHE' '-DSQLITE_TRACE_SIZE_LIMIT=32' '-DSQLITE_DEFAULT_CACHE_SIZE=-16000' '-DSQLITE_DEFAULT_FOREIGN_KEYS=1' '-DSQLITE_DEFAULT_WAL_SYNCHRONOUS=1' '-DSQLITE_ENABLE_MATH_FUNCTIONS' '-DSQLITE_ENABLE_DESERIALIZE' '-DSQLITE_ENABLE_COLUMN_METADATA' '-DSQLITE_ENABLE_UPDATE_DELETE_LIMIT' '-DSQLITE_ENABLE_STAT4' '-DSQLITE_ENABLE_FTS3_PARENTHESIS' '-DSQLITE_ENABLE_FTS3' '-DSQLITE_ENABLE_FTS4' '-DSQLITE_ENABLE_FTS5' '-DSQLITE_ENABLE_JSON1' '-DSQLITE_ENABLE_RTREE' '-DSQLITE_ENABLE_GEOPOLY' '-DSQLITE_INTROSPECTION_PRAGMAS' '-DSQLITE_SOUNDEX' '-DHAVE_STDINT_H=1' '-DHAVE_INT8_T=1' '-DHAVE_INT16_T=1' '-DHAVE_INT32_T=1' '-DHAVE_UINT8_T=1' '-DHAVE_UINT16_T=1' '-DHAVE_UINT32_T=1' '-D_GLIBCXX_USE_C99_MATH' '-DNDEBUG' -I/Users/andrewchou/GitHub/digidem/mapeo-mobile-next/node_modules/nodejs-mobile-react-native/android/libnode/include/node -I/Users/andrewchou/GitHub/digidem/mapeo-mobile-next/node_modules/nodejs-mobile-react-native/android/libnode/src -I/Users/andrewchou/GitHub/digidem/mapeo-mobile-next/node_modules/nodejs-mobile-react-native/android/libnode/deps/openssl/config -I/Users/andrewchou/GitHub/digidem/mapeo-mobile-next/node_modules/nodejs-mobile-react-native/android/libnode/deps/openssl/openssl/include -I/Users/andrewchou/GitHub/digidem/mapeo-mobile-next/node_modules/nodejs-mobile-react-native/android/libnode/deps/uv/include -I/Users/andrewchou/GitHub/digidem/mapeo-mobile-next/node_modules/nodejs-mobile-react-native/android/libnode/deps/zlib -I/Users/andrewchou/GitHub/digidem/mapeo-mobile-next/node_modules/nodejs-mobile-react-native/android/libnode/deps/v8/include -I./Release/obj/gen/sqlite3  -fasm-blocks -mpascal-strings -O3 -mmacosx-version-min=10.7 -arch x86_64 -w -std=c99 -MMD -MF ./Release/.deps/Release/obj.target/sqlite3/gen/sqlite3/sqlite3.o.d.raw   -c
  rm -f Release/sqlite3.a && ./gyp-mac-tool filter-libtool libtool  -static -o Release/sqlite3.a Release/obj.target/sqlite3/gen/sqlite3/sqlite3.o
rm ba23eeee118cd63e16015df367567cb043fed872.intermediate

• notice: rm -f Release/sqlite3.a && ./gyp-mac-tool filter-libtool libtool -static -o Release/sqlite3.a Release/obj.target/sqlite3/gen/sqlite3/sqlite3.o
  • most likely derived from https://github.com/nodejs/node-gyp/blob/main/gyp/pylib/gyp/generator/make.py#L210

---

There seems to be difference between how node-gyp flags and options are parsed between the projects. Whether it’s because of the difference in node+npm version or due to configuration errors in rebuild-android.sh is not yet clear.

There's a chance that the export npm_config_format="make-android" is not being used by node-gyp for the mobile-next case, causing it to resort to use mac as the build flavor and thus trying to use XCode's libtool. I tried setting GYP_GENERATORS=make-android but that didn't seem to change anything. Wondering if maybe because --ignore-environment needs to be passed to the node-gyp command somewhere in order to use it.

Another thing to consider is maybe differences in the node-gyp implementation? Didn't dig deep into this but doesn't seem likely.

Useful links for node-gyp:

• Entry where options and flags are used: https://github.com/nodejs/node-gyp/blob/main/gyp/pylib/gyp/__init__.py#L663
• Code that processes the node-gyp --formats flag (or GYP_GENERATORS env variable): https://github.com/nodejs/node-gyp/blob/main/gyp/pylib/gyp/__init__.py#L75
• Link command for Mac: https://github.com/nodejs/node-gyp/blob/main/gyp/pylib/gyp/generator/make.py#L210
• Link command for Android: https://github.com/nodejs/node-gyp/blob/main/gyp/pylib/gyp/generator/make.py#L222
• Where flavor (e.g. mac, linux, android, etc) is determined for GenerateOutput function: https://github.com/nodejs/node-gyp/blob/main/gyp/pylib/gyp/generator/make.py#L2399
  • Source for GetFlavor implmentation: https://github.com/nodejs/node-gyp/blob/main/gyp/pylib/gyp/common.py#L433
• Code that determines what link command is ran for the GenerateOutput function: https://github.com/nodejs/node-gyp/blob/main/gyp/pylib/gyp/generator/make.py#L2472-L2483
• Online documentation for Gyp: https://gyp.gsrc.io/

[achou11]

From @gmaclennan :

I tracked it down to node-gyp not respecting the npm_config_node_gyp environment variable with npm v8. Normally this would cause the node-gyp-nodejs-mobile to run when npm build is called, but instead it was calling the built-in node-gyp.

[achou11]

Looks like that bug was introduced in npm 7: npm/run-script#23

guess the short-term solution is to use npm 6 for now...

[achou11]

using npm 6 seems to move past the original issue. now running into this one:

The following command is run:

c++ '-DNODE_GYP_MODULE_NAME=better_sqlite3' '-DUSING_UV_SHARED=1' '-DUSING_V8_SHARED=1' '-DV8_DEPRECATION_WARNINGS=1' '-DV8_DEPRECATION_WARNINGS' '-DV8_IMMINENT_DEPRECATION_WARNINGS' '-D_LARGEFILE_SOURCE' '-D_FILE_OFFSET_BITS=64' '-D__STDC_FORMAT_MACROS' '-DOPENSSL_NO_PINSHARED' '-DOPENSSL_THREADS' '-DBUILDING_NODE_EXTENSION' '-D_GLIBCXX_USE_C99_MATH' '-DNDEBUG' -I/Users/andrewchou/GitHub/digidem/mapeo-mobile-next/node_modules/nodejs-mobile-react-native/android/libnode/include/node -I/Users/andrewchou/GitHub/digidem/mapeo-mobile-next/node_modules/nodejs-mobile-react-native/android/libnode/src -I/Users/andrewchou/GitHub/digidem/mapeo-mobile-next/node_modules/nodejs-mobile-react-native/android/libnode/deps/openssl/config -I/Users/andrewchou/GitHub/digidem/mapeo-mobile-next/node_modules/nodejs-mobile-react-native/android/libnode/deps/openssl/openssl/include -I/Users/andrewchou/GitHub/digidem/mapeo-mobile-next/node_modules/nodejs-mobile-react-native/android/libnode/deps/uv/include -I/Users/andrewchou/GitHub/digidem/mapeo-mobile-next/node_modules/nodejs-mobile-react-native/android/libnode/deps/zlib -I/Users/andrewchou/GitHub/digidem/mapeo-mobile-next/node_modules/nodejs-mobile-react-native/android/libnode/deps/v8/include -I./Release/obj/gen/sqlite3  -fPIC -Wall -Wextra -Wno-unused-parameter -std=c++14 -O3 -O3 -fno-omit-frame-pointer -fPIC -fno-rtti -fno-exceptions -std=gnu++1y -MMD -MF ./Release/.deps/Release/obj.target/better_sqlite3/src/better_sqlite3.o.d.raw   -c -o Release/obj.target/better_sqlite3/src/better_sqlite3.o ../src/better_sqlite3.cpp

Notice the c++ executable being used, which in my case points to /usr/bin/c++. That should instead be using the NDK toolkit e.g. /Users/andrewchou/Library/Android/sdk/ndk/21.4.7075529/toolchains/llvm/prebuilt/darwin-x86_64/bin/x86_64-linux-android21-clang++. We have two env variables that are set to this NDK path, CX and LINK, so it seems that one of them is not being properly picked up by node-gyp.

UPDATE: This was due to an error in the script. It should be CXX, not CX 😅

[achou11]

future note to self: nodejs-mobile-react-native v16.17.x attempts to handle the npm 7+ issue in a very fragile way by patching the install field in the target module's package.json. this may work in some cases but definitely doesn't in others, such as with better-sqlite3@7.6.2

[achou11]

Closing since no longer relevant