[Hold] Port Skylab ingest CI to jedi-ci

[ashley314]

Description

Provide a detailed description of what this PR does. What problem does it fix? What new capability does it add?

Issue(s) addressed

Resolves https://github.com/JCSDA-internal/skylab/issues/889

Dependencies

None?

Clean up PR for skylab: https://github.com/JCSDA-internal/skylab/pull/892

Impact

Expected impact on downstream repositories:

Manual Testing Instructions (optional)

Here is how I tested this PR using the CI docker image. From an existing EC2 instance:

# Authenticate to ECR and pull the container image
aws ecr get-login-password --region us-east-2 \
  | docker login --username AWS --password-stdin 747101682576.dkr.ecr.us-east-2.amazonaws.com
docker pull 747101682576.dkr.ecr.us-east-2.amazonaws.com/jedi-gcc-openmpi-dev:2.0

# Create the host directory layout
mkdir -p ~/skylab-test/build
git clone -b develop https://github.com/jcsda-internal/jedi-bundle ~/skylab-test/bundle

# Copy the jedi-ci shell scripts into the bundle
git clone -b feature/skylab-tests https://github.com/jcsda-internal/jedi-ci 
cp -r ~/jedi-ci/shell ~/skylab-test/bundle/jedi_ci_resources

# Launch the container
docker run -it \
  -v ~/skylab-test/bundle:/workdir/bundle \
  -v ~/skylab-test/build:/workdir/build \
  -e WORKDIR=/workdir \
  -e JEDI_COMPILER=gcc \
  747101682576.dkr.ecr.us-east-2.amazonaws.com/jedi-gcc-openmpi-dev:2.0 \
  bash

# Inside the container, set up git credentials and activate spack
source /opt/spack-environment/activate.sh
git config --global credential.helper store
echo "https://<YOUR_GITHUB_USERNAME>:<YOUR_TOKEN>@github.com" > ~/.git-credentials
git config --global --add safe.directory '*'

# Run the script, bypassing the GitHub App validation
export GITHUB_APP_PRIVATE_KEY=/dev/null
export GITHUB_APP_ID=0
export GITHUB_INSTALL_ID=0
bash /workdir/bundle/jedi_ci_resources/run_skylab_ingest.sh

# The script will print results to your screen as it runs
# Expected end results:
+ echo 'Skylab ingest complete'
Skylab ingest complete

Checklist

• I have performed a self-review of my own code
• I have made corresponding changes to the documentation
• I have run the unit tests before creating the PR

[ashley314]

I ended up removing the obs bias from our ingest suite for the CI. I have tested this out and will include testing instructions for future use. We will put this PR on hold until CI is ready for another test.

[ashley314]

@eap Tagging you as a reviewer. We talked about holding this PR until later. I successfully tested it so hopefully it should be good to go at a later date. Instructions for testing are in the description. Thanks!