Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support multiple software stacks #338

Open
jcfreeman2 opened this issue Jan 16, 2024 · 1 comment
Open

Support multiple software stacks #338

jcfreeman2 opened this issue Jan 16, 2024 · 1 comment

Comments

@jcfreeman2
Copy link
Collaborator

As the DUNE DAQ software project evolves, different parts of it begin to diverge. E.g., writing run control software really isn't in theory heavily coupled with writing detector readout software. Currently we support a near detector stack and a far detector stack, but we should generalize this to more stacks:

  • Run control
  • Configuration
  • DQM
  • Raw data unpacking
  • Other?

To 0th order it seems that these would be in parallel with the ND and FD stacks. E.g., while on this year's April Fool's Day one could call dbt-setup-release or dbt-create for the far detector via NAFD24-04-01, one could also work with the raw data release via NARD24-04-01. Please note that if we want longer, more descriptive names - e.g., NARAWDATA24-04-01 - we'll need to reduce the path length of our releases.

From an implementation perspective, it seems like the strategy to employ would be for each stack to get its own subdirectory in daq-release/configs/ with its own release.yaml file. We could also, which keeping an externals repo and a base package repo in common for all the stacks, have different umbrella packages which pick up different subsets of the packages in those two repos. E.g., we'd rename the current externals umbrella package as externals_fd and externals_nd, and then add an externals_configuration which would drop dpdk but add qt (needed for the dbe package).

@jcfreeman2
Copy link
Collaborator Author

The first step here seems fairly straightforward, though perhaps I'm underthinking. The most important new umbrella package concerns data formats (I'll call it fddataformats here, though I'm totally open to a new name). As I understand it, the two highest level dependencies for fddataformats are rawdatautils and hdf5libs (*); given that these are already built as part of the fddaq build, it will literally add just a couple of seconds to our nightly build if I create an umbrella package called fddataformats. Then, a developer should be able to do something like:

dbt-create -n NFDDF_PROD4_240415_A9 <name of new work area>
cd <new work area>
dbt-workarea-env

...where FDDF stands for Far Detector Data Formats. Instead of the "traditional" fddaq umbrella package getting loaded, this will spack load in the fddataformats package, even though the hdf5libs and rawdatautils packages will be physically installed in the regular /cvmfs/dunedaq-development.opensciencegrid.org/nightly/NB_PROD4_240415_A9/spack-0.20.0/spack-repo and /cvmfs/dunedaq-development.opensciencegrid.org/nightly/NFD_PROD4_240415_A9/spack-0.20.0/spack-repo repositories, respectively.

(*) For now I'm setting aside the fact that rawdatautils will eventually probably need to be renamed fdrawdatautils and we'll want to add a ndrawdatautils.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant