DRAFT: Add project ingestion to the DB to the project sync cron job

[alasdairwilson]

Adds syncing and ingestion for external project repos into the existing auth/access DB, so deployed static projects get registered automatically and linked to real users.

One behaviour that i am not sure about... at the moment if a new project is inserted and the owner does not exist in the Users table then we throw a RuntimeError so then you would have to make the account and then wait for the hourly resync I guess?

Maybe that is fine...

[alasdairwilson]

I have marked this as a draft because I havent really let this change perculate in my head very long so I don't know if it is fully "right" but I am on AL so I figured if I atleast made the PR people could have a look.

[codecov]

Codecov Report

❌ Patch coverage is 73.05699% with 52 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
vertex/project_ingestion.py	73.05%	52 Missing ⚠️

Files with missing lines	Coverage Δ
vertex/project_ingestion.py	73.05% <73.05%> (ø)

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[sr-murthy]

Looks OK, but I had a few questions below.

[sr-murthy]

Might be useful for debugging to include the time diff end_ts - start_ts in the echo message to get an idea of sync times, so something like:

time_diff=$(echo "$(date --date=$end_ts +%s) - $(date --date=$start_ts +%s)" |  bc -l)
echo "[${end_ts}] Sync finished successfully in $time_diff seconds"

Had to install GNU coreutils on Mac to test the time diff with gdate (not date which errors), but this should work.

[sr-murthy]

We now have a VERTEX CLI. I'm wondering whether it would be worth adding this functionality, for example, in an admin command subgroup.

Obviously, the CLI is user-facing, but this particular script of yours requires certain credentials and information only admins would have, so I don't see a problem with it.

I think this looks fine in its current form, but it would be nice to have a uniform common CLI that can be used by users, developers and admins.

[read-the-docs-community]

Documentation build overview

📚 ISARIC VERTEX | 🛠️ Build #32932662 | 📁 Comparing d0fce03 against latest (8e78bf1)

  🔍 Preview build  

7 files changed · + 5 added · ± 2 modified

+ Added

• sources/api-reference.html
• sources/app.html
• sources/cli.html
• sources/contributors.html
• sources/vertex/cli.html

± Modified

• index.html
• sources/getting-started.html

[lithomson]

The code looks good to me. I tested it using my local database and test projects for ISARICAccount. Nothing loaded when one of the emails didn't exist. Should this one just be skipped?

% python -m vertex.project_ingestion
--projects-dir /Users/laurat/PycharmProjects/IsaricAccount/dev/vertex
--database-url sqlite:////Users/laurat/PycharmProjects/IsaricAccount/instance/dev.db

2026-06-01 11:34:33 [ERROR] main: Static project ingestion failed: Cannot insert project: owner user does not exist for no_a_user@gmail.com. Parsed config={"is_public": true, "name": "project_3", "project_dir": "/Users/laurat/PycharmProjects/IsaricAccount/dev/vertex/project_3", "project_id": "this-wont-be-loaded", "project_owner": "no_a_user@gmail.com"}

I changed to an existing owner and they all loaded (as expected).

2026-06-01 11:35:42 [INFO] main: Static project ingestion summary: seen=5 inserted=5 existing=0 owner_links_inserted=5 owner_links_existing=0 owner_pending_users=0 owner_immutable_skipped=0

And then no reload attempt for the third run (as expected).

2026-06-01 11:38:20 [INFO] main: Static project ingestion summary: seen=5 inserted=0 existing=5 owner_links_inserted=0 owner_links_existing=0 owner_pending_users=0 owner_immutable_skipped=5