JCF: overhaul of the documentation on how to interact with cvmfs

docs/create_release_spack.md

@@ -63,19 +63,8 @@ It's worth to do several checks before starting any test builds. These checks in
 * The release will be cut at the end of the testing period. The build of the final frozen release can be done in a similar way as the candidate releases. Choose "Build frozen release" in the workflows list, and trigger the build by specifying release name used in `configs` and the build number (starts from 1, increment it if second deployment to cvmfs is needed).
 * Deploying the frozen release to cvmfs is the same as for a candidate release  _except_ you want to log in to `oasiscfs01.fnal.gov` as `cvmfsdunedaq` instead of `cvmfsdunedaqdev` and of course you'll want to pass `frozen` rather than `candidate` to the publishing script
 * Do similar tests as shown in the section above for candidate releases
-* If there is a new version of `daq-buildtools` for the release, it will need to be deployed to cvmfs too. Otherwise, creating a symbolic link in cvmfs to the latest tagged version will be sufficient. 
-
-* Adding a new daq-buildtools version
-
-To do so, login to `[email protected]` as `cvmfsdunedaq`, then, [after setting up a cvmfs transaction](publish_to_cvmfs.md) for `dunedaq.opensciencegrid.org`, doing the following:
-  1. change directory to `/cvmfs/dunedaq.opensciencegrid.org/tools/dbt/`;
-  1. (if needed) download the latest tagged version of daq-buildtools if it is not deployed yet, and expand it in the directory, rename the directory name using the tag;
-  1. (if needed) move the `latest` link to the latest tag in the directory;
-  1. create a symbolic link using the release tag, and point it to the latest tag in the directory;
-Then publish the change to cvmfs. 
-
+* If there is a new version of `daq-buildtools` for the release, it will need to be deployed to cvmfs too. Otherwise, creating a symbolic link in cvmfs to the latest tagged version will be sufficient. How to do this is described in the [documentation on cvmfs](publish_to_cvmfs.md).
 * After the frozen release is rolled out, there will be remaining prep release and patch branches used in the production of the release. The software coordination team and the release coordinator should get in touch to establish if anything should be kept out of the merge to `develop`. The software coordination team will do the merge across all relevant repositories. Developers should handle any partial merge (cherry-pick).
-
 * The last step of making a frozen release is to create release tags for all packages used by the release. To do so, use the script `scripts/create-release-tag.py`:
     * make sure `daq-release` is tagged for the new release, and the version is updated in the release YAML file. It will be tagged by the `create-release-tag.py` script;
     * `scripts/create-release-tag.py -h` to show the usage of the script;

docs/publish_to_cvmfs.md

@@ -1,14 +1,14 @@
 # Publish files to cvmfs
 
+## Overview
+
 We use CERN's [CernVM-FS filesystem](https://cernvm.cern.ch/fs/),
 better known as "cvmfs", to make our software releases available to
-the collaboration. A good user tutorial for how to publish files to it can be found [here](https://cvmfs-contrib.github.io/cvmfs-tutorial-2021/04_publishing/), but this document will focus specifically on DUNE DAQ Software Coordination-specific use of cvmfs.
+the collaboration. A good user tutorial for how to publish files to it can be found [here](https://cvmfs-contrib.github.io/cvmfs-tutorial-2021/04_publishing/), but this page will focus on DUNE DAQ Software Coordination-specific use of cvmfs.
 
 We use two cvmfs repositories: `dunedaq.opensciencegrid.org` for our frozen releases, and `dunedaq-development.opensciencegrid.org` for our candidate and nightly releases. To alter the former, log in to `oasiscfs01.fnal.gov` as `cvmfsdunedaq`; to alter the latter, log in as `cvmfsdunedaqdev`. Once you've done that, you can use the `cvmfs_server` command (or a script which uses it) to modify cvmfs. 
 
-Publishing the releases typically does not involve running `cvmfs_server` directly. [The nightly is published directly](https://dune-daq-sw.readthedocs.io/en/latest/packages/daq-release/ci_github_action/#how-the-nightly-releases-are-made) via the GitHub Workflow, and [candidate and frozen releases are published via the `publish_release_to_cvmfs.sh` script](https://dune-daq-sw.readthedocs.io/en/latest/packages/daq-release/create_release_spack/#building-candidate-releases)
-
-The most common use case for directly calling `cvmfs_server` is when adding a new daq-buildtools version
+## The basics
 
 A cheat sheet for using `cvmfs_server` which is even simpler than the tutorial linked to above is the following:
 1. Open a cvmfs server transaction via `cvmfs_server transaction <name of cvmfs repo>`, where you'd use the name of one of our two cvmfs repos here
@@ -18,24 +18,31 @@ A cheat sheet for using `cvmfs_server` which is even simpler than the tutorial l
 
 The cvmfs area is relied upon by everyone so of course it's important to treat it carefully. In the event that something bad happens, however, cvmfs provides a method to roll back unwanted changes. Each cvmfs publish has an associated tag; if you run `cvmfs_server tag -l <name of cvmfs repo>` you can see the tag for each publish in the first column of the output. If you wish to roll back to a particular tag of the cvmfs repo's existence, you can run `cvmfs_server rollback -t <name of tag> <name of cvmfs repo>`. In general this would only be under exceptional circumstances and in consultation with other members of software coordination. 
 
-_JCF, Jan-10-2024: section below not yet updated_
-
-## Publishing to cvmfs
-
-1. Once the files are under the staging area mentioned above, login to `oasiscfs01.fnal.gov` as `cvmfsdunedaq` if publishing to `/cvmfs/dunedaq.opensciencegrid.org` (or as `cvmfsdunedaqdev` if publishing to `/cvmfs/dunedaq-development.opensciencegrid.org`);
-2. Run `~/bin/dunedaq-sync` (or `~/bin/dunedaq-dev-sync`) to sync the cvmfs repo to the staging area. There is a `~/dunedaq-sync-delete` script which uses `--delete` option when syncing with `rsync`. Be careful when using this script since it may delete exisiting files from the cvmfs repo and potentially break existing workflows.
-
-### Adding more top level directories to the repo
-
-The DUNE cvmfs repo utilizes `.cvmfsdirtab` in the top directory of the repo to help generating `.cvmfscatalog` files for subdirectories. If you need to add a new directory which is not covered by exisiting enties in the file, you will need to modify `~/bin/dune-split` with the new directory path, and run it once.
-
-### Publishing externals
-
-_JCF, Jan-10-2024: to be added_
+## Updating daq-buildtools on cvmfs
 
+Most uses of `cvmfs_server` are wrapped within scripts, but adding a
+new version of daq-buildtools to cvmfs is an exception; this section
+also doubles as a concrete example of the commands in the previous
+section. Log in to `oasiscfs01.fnal.gov` as `cvmfsdunedaq` and execute the following:
 
+1. cvmfs_server transaction dunedaq.opensciencegrid.org
+1. cd /cvmfs/dunedaq.opensciencegrid.org/tools/dbt/
+1. git clone https://github.com/DUNE-DAQ/daq-buildtools
+1. rm -rf daq-buildtools/.git   # No need for git info on cvmfs
+1. mv daq-buildtools <version>
+1. rm latest
+1. ln -s <version> latest
+1. cd ~   # you can't run the next command from within /cvmfs
+1. cvmfs_server publish dunedaq.opensciencegrid.org
 
+## Updating releases on cvmfs
 
+Publishing the releases typically does not involve running `cvmfs_server` directly. [The nightly is published directly](https://dune-daq-sw.readthedocs.io/en/latest/packages/daq-release/ci_github_action/#how-the-nightly-releases-are-made) via GitHub Workflows, and [candidate and frozen releases are published via the `publish_release_to_cvmfs.sh` script](https://dune-daq-sw.readthedocs.io/en/latest/packages/daq-release/create_release_spack/#building-candidate-releases)
 
+## Updating a particular directory on cvmfs
 
-
+As a member of Software Coordination, if you have an account on `mwts.fnal.gov` with a staging area called `/home/<your username>/docker-scratch/cvmfs_dunedaq`, the `scripts/cvmfs/publish_directory_to_cvmfs.sh` script will synchronize a given subdirectory in your staging area to a given subdirectory on `/cvmfs`. E.g., if you've made changes to the v2.0 externals in your staging area, you can do the following if you've logged in to `oasiscfs01.fnal.gov` as `cvmfsdunedaq` and are in the base of a freshly-updated daq-release repo:
+```
+./scripts/cvmfs/publish_directory_to_cvmfs.sh spack/externals/ext-v2.0 
+``` 
+You'll be prompted a couple of times if you're sure you want to go ahead, since of course modifying `/cvmfs` is a sensitive expert action. _Be aware_ that this script behaves as a "clobber", so if along with adding files to your staging-area `spack/externals/ext-v2.0` directory you've also removed some files, the corresponding files on cvmfs will also be removed.