-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support a fddatautilities
umbrella package
#361
Comments
…ocally on daq.fnal.gov fddatautilities is an umbrella package which depends on a couple of far-detector-focused packages as well as the core set of packages (currently named "dunedaq" as it has for a couple of years, but soon to be renamed "coredaq"). dunedaq has been given a build variant called "subset" which when it assumes the value of "datautilities" will only depend on the subset of core packages which fddatautilities needs. Note that it may make sense to have the variant value match up perfectly with the name of the target package it's intended to support. In order to not try and solve everything at once, the package.py files for the umbrella packages needed for this build are located in a temporary subdirectory called issue361_dev_files. With this subdirectory I can sidestep worrying about the make-release-repo.py algorithm whenever I want to modify and test the relevant package.py files for the umbrella packages during the development process. I've successfully build fddatautilities using the files in this commit via calling build-release.sh manually inside of a container on daq.fnal.gov which is volume-mounted to a v2.1 externals area.
…reate an image with only externals needed for fdreadoututilities
…CHE-dness, force the layers to be redone in the image by sticking a command before them
… try to get the cache to be ignored so I can use a different staging location on daq.fnal.gov than normal
…ets up the externals area to be copied
Status update: What we have so far:
To get
and then run As you can see,
One question that will be persistent as we move forward is, how do we decide which packages should go into the core DUNE DAQ repository, and which should go into the umbrella-package-specific repository? There are tradeoffs involved: the choice I made here means that Another question: I imagine that we want to treat these new umbrella packages as exactly analogous to (*) Work on this Issue began before |
…ronment variables expected by wf-setup-tools.sh
…evel between fddaq/nddaq vs. fddatautilities during this Issue's development process
…uccessful fddatautilities build in a container
…-creating command in build-release.sh
…hat we have a successful Workflow build of fddatautilities (https://github.com/DUNE-DAQ/daq-release/actions/runs/8885383682)
…ck name is deduced from the nightly tag acronym to support adding a letter to denote a test build (e.g., FDT instead of FD)
…s a dependency of fddatautilities
…es image has access to the name of the nightly tag
…image as apparently it's basically impossible to copy a local absolute directory into an image via a Dockerfile
…utilities external image to create the lightest-weight fddatautilities external image possible
…lease-repo.py for fddatautilities
...which will depend directly only on
hdf5libs
andrawdatautils
, and can be used for data analysis but isn't a full blown DAQ system the wayfddaq
andnddaq
are.A major part of this Issue is that we want to have an image available to people who want to use
fddatautilities
which is much slimmer than thefddaq
/nddaq
analogue,ghcr.io/dune-daq/nightly-release-alma9:development_v5
and the image which it derives from,ghcr.io/dune-daq/alma9-slim-externals:v2.1
, both north of 10 GB.A secondary effect of this Issue is it should give us a real-world sense of what extending our framework to support diverse types of umbrella package entails; this topic is covered in more detail in Issue #338
The text was updated successfully, but these errors were encountered: