Support for custom libraries when using `sketch.yaml`

[dmohns]

Describe the request

In a project I am currently using different "kind" of libraries.

1. Versioned libraries available in via Arduino Library Manager
2. Non-versioned libraries available via their source code on Github
3. Non-versioned libraries available via their source code in local subfolder folder of my project (zipped or raw)

To ease collaboration on the project I would like to use a Sketch Project File and provide a set of dependencies that are confirmed working.

Unfortunately, the "build profiles" feature only seems to support 1. (versioned libraries available in Arduino Library Manager). Scope of this request is to add functionality to supply Github and local folder libraries to the libraries: attribute on sketch.yaml (in some fashion).

Describe the current behavior

Currently, I have been able to compile my project using

arduino-cli compile --fqbn "<FQBN>" --libraries=Library/

However, if I tried to use --profile and --libraries the compile command fails

You cannot use the --libraries flag while compiling with a profile.

Arduino CLI version

arduino-cli Version: nightly-20230329 Commit: b69ed33 Date: 2023-03-29T01:34:59Z

Operating system

macOS

Operating system version

12.6.3

Additional context

Additional requests

• Support for custom libraries when using sketch.yaml #2133 (comment)
• Support for custom libraries when using sketch.yaml #2133 (comment)
• Support for custom libraries when using sketch.yaml #2133 (comment)
• Detect libraries in Sketch Folder #2763
• https://arduino.stackexchange.com/questions/17802/is-it-possible-to-package-a-sketch-with-dependent-libraries-for-cross-platform-c
• https://forum.arduino.cc/t/local-libraries/1259854/20

Related

• arduino-cli should provide "environments" #108
• https://stackoverflow.com/questions/4705790/keeping-all-libraries-in-the-arduino-sketch-directory

Issue checklist

• I searched for previous requests in the issue tracker
• I verified the feature was still missing when using the nightly build
• My request contains all necessary details

[umbynos]

Seems a duplicate of #108

[jirutka]

Seems a duplicate of #108

It’s not a duplicate, #108 is just one of the ways it can be implemented (and more complicated than requested here).

[dmohns]

Yeah, just to clarify, this issue is about making sketch.yaml spec more flexible to support libraries from other sources than Arduino Library Manager.

One way that is backwards compatible would be

profiles:
  <PROFILE_NAME>:
    ....
    libraries:
      - <LIB_NAME> (<LIB_VERSION>)

to allow <LIB_VERSION> to be <LIB_VERSION_OR_URL> and take the form of VCS and local urls, for example

profiles:
  my_profile:
    ....
    libraries:
      - ArduinoIoTCloud (1.0.2)
      - HelloWorld (https://github.com/username/HelloWorld.git#v2.13)
      - HelloWorld2 (file://./libraries/HelloWorld2.zip)

Note: Above is just an example. There are other ways to implements this as well, like changing the schema of libraries to be a list of dicts rather list of strings. This would be more powerful and flexible but not be backwards compatible.

[eesoymilk]

I've been trying to install custom libraries through cli for ages now.
Is there a workaround for this issue?
The cli compile can never seem to find my custom lib in ~/Arduino/libraries.

[ubidefeo]

@eesoymilk
can you be more specific?
what are your custom libraries? libraries developed by you?
do they conform to the libraries specification?

[eesoymilk]

@ubidefeo I'm writing my own library. This is the repo: eesoymilk/AS608-Fingerprint-Arduino.

I believe it does conform to the spec since running arduino-lint only shows 1 warning (no license).

I put this repo directly in my ~/Arduino/libraries of course.
I was trying to compile it with arduino-cli and I have a sketch.yaml file right next to the targeted .ino file.
I added my library's into the skectch.yaml file in hope that all things works (arduino-cli, neovim lsp, etc.):

libraries:
  - AS608Fingerprint (0.1.0)

However, this is the root cause why arduino-cli compile won't correctly compile. It trys to download my library from the Arduino Library Manager (DNE btw) even though my library is listed in arduino-cli lib list.

In the end, removing my library from sketch.yaml solved this problem and the lsp also works.

[dmohns]

I put this repo directly in my ~/Arduino/libraries of course. I was trying to compile it with arduino-cli and I have a sketch.yaml file right next to the targeted .ino file. I added my library's into the skectch.yaml file in hope that all things works (arduino-cli, neovim lsp, etc.):

libraries:
  - AS608Fingerprint (0.1.0)

However, this is the root cause why arduino-cli compile won't correctly compile. It trys to download my library from the Arduino Library Manager (DNE btw) even though my library is listed in arduino-cli lib list.

You would need to run arduino-cli lib install --git-url https://github.com/eesoymilk/AS608-Fingerprint-Arduino.git then arduino-cli compile should work. Which is very tedious to do if you have more than one external library. You can find an example how we created a Makefile to automate some of this here: https://github.com/EnAccess/OpenSmartMeter/blob/27ad33a244d4f95201a7676a6d8307f7c766d15e/Firmware%20code/smart_energy_meter/Makefile#L23-L33

What you describe is exactly the motivation for why this feature request exists.

But let's keep this discussion about the feature request itself rather than workarounds for individual situations.