[Feature] Package loader for amber #668

b1ek · 2025-01-18T06:15:48Z

we would obviously need a package manager at some point.

i'd like to propose a system for loading packages on compiler level

with these package directories (i will refer to this as $PKGDIR from now on):

./amber_modules
~/.local/amber_modules
/usr/lib/amber_modules

when importing a path that starts with a letter, compiler should import it from $PKGDIR/package_name/main.ab, with this logic:

// pseudo code
const package_directories = [
    "./amber_modules",
    "~/.local/amber_modules",
    "/usr/bin/amber_modules"
];
function resolve_import(path: string): string? {
    if !path.startsWithLetter() {
        return resolve_local_import(path)
    }
    if path.split("/").length == 1 {
        path += "/main.ab";
    }
    if !path.endsWith(".ab") {
        path += ".ab";
    }

    for (pkgdir of package_directories) {
        if file_exists(path.join(pkgdir)) {
            return path.join(pkgdir);
        }
        if exists(path.join(pkgdir) + "/main.ab") {
            return path.join(pkgdir) + "/main.ab";
        }
    }
    
    return null;
}

resolve_import("package_name"); // import "package_name" -> "$PKGDIR/main.ab"
resolve_import("package_name/subpkg"); // import "package_name" -> "$PKGDIR/subpkg.ab" | "$PKGDIR/subpkg/main.ab"
resolve_import("package_name/subpkg.ab"); // import "package_name" -> "$PKGDIR/subpkg.ab" | "$PKGDIR/subpkg/main.ab"

for importing files in the working directory, users should use import "./local_file"

implementing this will allow for creating a package manager for amber, that would be maintained independently from compiler

this will also mean treating stdlib as a package, rather than as a hardcoded language feature

The text was updated successfully, but these errors were encountered:

lens0021 · 2025-01-18T08:08:31Z

I think $PKGDIR/main.ab is not enough, $PKGDIR/THE_SPECIFIED_VERSION_OF_THE_PACKAGE/main.ab is. So some counterparts of Cargo.toml and Cargo.lock are also required. So we should also decide to which format to use. (json, yaml, toml, pkl, kdl, etc).

And Jura is what I tried just for fun. But I will patch to match some of the requirements.

b1ek · 2025-01-18T09:27:54Z

I think $PKGDIR/main.ab is not enough, $PKGDIR/THE_SPECIFIED_VERSION_OF_THE_PACKAGE/main.ab is.

i wouldn't want to meddle with package versions, dependencies, and integrity checks on compiler level. i imagine it working like it is with node - a 3rd party app like yarn does all the smart work like downloading, versions and integrity checks, and puts it into node_modules for node to load. so like the compiler itself doesn't manage anything

also i think you have it a little bit confused - rust's compiler doesn't read cargo files. cargo is a totally independent program from rust's compiler

b1ek · 2025-01-18T09:32:42Z

i want to clarify that the purpose of this issue is not to create a package manager, but to create an interface for loading packages into the compiler. package management is not something that should be done by the compiler, but rather by a 3rd party program

lens0021 · 2025-01-20T05:11:18Z

Oh, I am happy adding more location to search modules. Thank you.

My opinions for details.

I am not sure if it is good to automatically load /main.ab when only package_name given. Because if both $PKGDIR/subpkg.ab and $PKGDIR/subpkg/main.ab exist, we should set priorities to them.
Shouldn't importing a path that starts with a letter be for the project root? Though we have not the concept of the project root, I think we should if we treat packages.

b1ek · 2025-01-25T11:12:38Z

I am not sure if it is good to automatically load /main.ab when only package_name given. Because if both $PKGDIR/subpkg.ab and $PKGDIR/subpkg/main.ab exist, we should set priorities to them.

you mean $PKGDIR/package_name(.ab|/main.ab), right? if it is, then i wrote in the pseudocode snippet that the $PKGDIR/package_name/main.ab should be loaded only, and not $PKGDIR/package_name.ab, with the $PKGDIR/package_name being the namespace for that package

Shouldn't importing a path that starts with a letter be for the project root? Though we have not the concept of the project root, I think we should if we treat packages.

we do not have project root at all for now so i dont know how thats relevant. you do mean working directory, right?

i just feel like that would cause way too much confusion about what is a package and what is a file in the working directory

b1ek · 2025-01-25T11:16:04Z

also i think that in files that are imported in a different directory than the first one, working directory should refer to the directory they are in. so if ./main.ab imports ./dir/main, and ./dir/main.ab imports ./file, ./file should refer to ./dir/file.ab in that context. and if ./dir/main.ab needs to import something from the parent directory, it should use ../file_in_parent

lens0021 · 2025-01-25T13:54:50Z

Thank you for the detailed explanation. I support your proposal.

Thesola10 · 2025-03-12T09:14:59Z

Fundamentally Amber doesn't have a notion of a "project directory". Each file is imported relative to its importer, which is clear and easy to understand. Having to work around this leads to what Jura currently does to work around the lack of a configurable import path.

I still strongly believe an AMBER_PATH variable is the simpler way to address this, as showcased in #681, since the source code is not aware of the developer's environment and assuming so will break portability with Nix or Meson, or just with users in general.

There will be default paths, and the variable will only be used as an override as needed by a build tool or non-standard user config. Making any further assumptions will hinder Amber's adoption, as I believe embedding it into a project built in a different language will be a big use case.

If we want to perform version checks, or siloed environments, we can use external tools (or turn amber itself into a wrapper for, say, amberc). That way, the environment variable can be used to build virtual environments as needed, combining "default" modules (like the stdlib, or modules installed through rpm/pacman/what have you) with "local" modules in a predictable manner.

b1ek · 2025-03-20T05:44:05Z

Fundamentally Amber doesn't have a notion of a "project directory".

is this relevant? this feature request doesn't use "project directory" in its terminology, but rather the working directory where the compiler is being run

I still strongly believe an AMBER_PATH variable is the simpler way to address this, as showcased in #681

could you make a technical description of how you see it work? preferably in its own feature request since you are pretty much just making a new proposal

I still strongly believe an AMBER_PATH variable is the simpler way to address this, as showcased in #681, since the source code is not aware of the developer's environment and assuming so will break portability with Nix or Meson, or just with users in general.

i dont get it. how is using a custom variable simpler than relying on ./amber_modules? its pretty much just the ./node_modules but renamed for amber. its location is predictable and rigidly defined, which allows for multiple tools (npm, yarn, pnpm) to load packages in the same way.

relying on a dynamically defined variable just straight up invites chaos the way i see it. because package manager 1 will store its packages in ./pkgmanager1_modules, package manager 2 will store its packages in ./pkgmanager2_modules and is a complete mess.

don't get me wrong though, i am not proposing relying on one directory specifically. my idea is that if there is no package "package" in ./amber_modules, it will first look for user-wide install of it and then for system-wide install of it. which allows per-project installations and preserving the option to install a package system-wide (DRY)

b1ek · 2025-03-20T05:50:03Z

its probably a wise idea to not hardcode /usr/lib for systems that do not have it, but i still insist on hardcoding amber_modules at the end for the sake of predictability and simplicity. maybe load from $PATH? or maybe forget about system-wide packages like rust does and only load from ./amber_modules

Thesola10 · 2025-03-20T08:25:07Z

The use of amber_modules isn't incompatible with AMBER_PATH. What I mean though is, where is amber_modules supposed to be if I start compilation in a subdirectory? As it stands, the "current directory" changes for each file compiled by Amber.

e.g. if I run amber build src/tests/test.ab, will it look for amber_modules in src/tests?

The advantage of using AMBER_PATH in front of the final compiler, is that it lets the compiler not have to know about this. If the user-facing solution ends up being amber_modules, then that will be by finding and setting that directory in AMBER_PATH, likely using a default frontend. I'm talking about the relationship between go and cgo, or cargo and rustc, or even gcc and cc1.

It's very important to allow other buildsystems to integrate Amber as they see fit, because Amber produces shell scripts, which are usually embedded as part of a larger project.

I'll open a detailed feature proposal for the frontend/backend architecture and how AMBER_PATH and amber_modules will tie into this.

b1ek added compiler enhancement New feature or request stdlib syntax labels Jan 18, 2025

github-project-automation bot added this to Amber Project Jan 18, 2025

github-project-automation bot moved this to 🆕 New in Amber Project Jan 18, 2025

Thesola10 linked a pull request Feb 27, 2025 that will close this issue

Implemented AMBER_PATH for import lookup #681

Draft

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Feature] Package loader for amber #668

[Feature] Package loader for amber #668

b1ek commented Jan 18, 2025 •

edited

Loading

lens0021 commented Jan 18, 2025 •

edited

Loading

b1ek commented Jan 18, 2025

b1ek commented Jan 18, 2025

lens0021 commented Jan 20, 2025 •

edited

Loading

b1ek commented Jan 25, 2025

b1ek commented Jan 25, 2025

lens0021 commented Jan 25, 2025

Thesola10 commented Mar 12, 2025 •

edited

Loading

b1ek commented Mar 20, 2025

b1ek commented Mar 20, 2025

Thesola10 commented Mar 20, 2025

[Feature] Package loader for amber #668

[Feature] Package loader for amber #668

Comments

b1ek commented Jan 18, 2025 • edited Loading

lens0021 commented Jan 18, 2025 • edited Loading

b1ek commented Jan 18, 2025

b1ek commented Jan 18, 2025

lens0021 commented Jan 20, 2025 • edited Loading

b1ek commented Jan 25, 2025

b1ek commented Jan 25, 2025

lens0021 commented Jan 25, 2025

Thesola10 commented Mar 12, 2025 • edited Loading

b1ek commented Mar 20, 2025

b1ek commented Mar 20, 2025

Thesola10 commented Mar 20, 2025

b1ek commented Jan 18, 2025 •

edited

Loading

lens0021 commented Jan 18, 2025 •

edited

Loading

lens0021 commented Jan 20, 2025 •

edited

Loading

Thesola10 commented Mar 12, 2025 •

edited

Loading