[ADMINAPI-1369] Create DbInstance Jobs

[josephcampos-gap]

This pull request introduces enhancements to the AddDbInstance feature, primarily adding job scheduling for database instance creation and improving multi-tenancy support. It also refines validation logic for instance names and updates related tests to cover the new behavior.

Job Scheduling and Multi-Tenancy Enhancements:

• The AddDbInstance.Handle method now schedules a Quartz job (CreateInstanceJob) to create the database instance after the request is validated and persisted. The job is tenant-aware if multi-tenancy is enabled, ensuring jobs are scheduled with the correct tenant context.
• Added new constants to JobConstants for job and data map keys, supporting the new job scheduling logic.
• The AppSettings class now includes configuration for the database instance creation job sweep interval and max retry attempts, supporting job execution management.

Validation Improvements:

• The maximum allowed length for a database instance name is now dynamically calculated to ensure the synchronized ODS instance name fits within 100 characters, with the validator updated to enforce this and provide a clear error message.

Test Coverage Updates:

• Unit tests for AddDbInstance have been updated to accommodate the new job scheduling logic. New tests verify that jobs are scheduled correctly, including tenant-aware scenarios, and all validation tests are updated to use the new method signature and dependencies. [1] [2] [3] [4] [5] [6] [7] [8]

Summary of Most Important Changes:

Job Scheduling and Multi-Tenancy:

• AddDbInstance.Handle now schedules a CreateInstanceJob using Quartz, including tenant information when multi-tenancy is enabled.
• Added job-related constants to JobConstants and new job configuration properties to AppSettings. [1] [2]

Validation:

• Updated validation to enforce a stricter, dynamically calculated maximum database instance name length to ensure compatibility with ODS naming constraints.

Testing:

• Refactored and expanded unit tests to cover job scheduling and multi-tenancy scenarios, and updated all tests to use the new handler signature and dependencies. [1] [2] [3] [4] [5] [6] [7] [8]

[github-actions]

Test Results

   14 files     14 suites   36s ⏱️
  775 tests   774 ✅ 1 💤 0 ❌
1 715 runs  1 714 ✅ 1 💤 0 ❌

Results for commit 68ba2ac.

♻️ This comment has been updated with latest results.

[Copilot]

Pull request overview

This PR enhances the Admin API v2 POST /v2/dbinstances workflow by making DbInstance provisioning asynchronous via Quartz jobs, improving multi-tenant support for background execution, and tightening request validation around naming/length constraints.

Changes:

• Schedule CreateInstanceJob after persisting a Pending DbInstance and add a recurring CreatePendingDbInstancesDispatcherJob for sweep-based recovery and capped retries.
• Improve multi-tenant behavior by propagating tenant identity into job keys/data, enabling tenant-specific connection string resolution, and updating Docker compose defaults accordingly.
• Expand validation (allowed characters, uniqueness across DbInstances and OdsInstances, portable DB-name length) and update/add unit/E2E/docs to reflect the async job pipeline.

Reviewed changes

Copilot reviewed 50 out of 50 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
docs/http/dbinstances.http	Updates HTTP examples to include Tenant usage and new request naming; adds delete examples.
docs/developer.md	Documents the asynchronous provisioning flow and required configuration.
docs/design/PLAN-D-ACCEPT-RISK.md	Adds “accept risk” remediation plan documentation for tenant context race.
docs/design/PLAN-C-EXPLICIT-CONNECTION-STRING-PARAM.md	Adds “explicit connection string param” remediation plan documentation.
docs/design/PLAN-B-HTTPACCESSOR-SPLIT-STORAGE.md	Adds “split storage” remediation plan documentation.
docs/design/PLAN-A-ASYNC-LOCAL-CONTEXT-STORAGE.md	Adds “AsyncLocal context storage” remediation plan documentation.
docs/design/DBINSTANCE-PROVISIONING-JOBS.md	Adds detailed durable design doc for provisioning jobs, identity, retries, and risks.
Docker/V2/Compose/pgsql/SingleTenant/compose-build-ods.yml	Adds EdFi_Ods/EdFi_Master connection strings for single-tenant pgsql.
Docker/V2/Compose/pgsql/SingleTenant/compose-build-idp-dev.yml	Adds EdFi_Ods/EdFi_Master connection strings for single-tenant pgsql.
Docker/V2/Compose/pgsql/SingleTenant/compose-build-idp-binaries.yml	Adds EdFi_Ods/EdFi_Master connection strings for single-tenant pgsql.
Docker/V2/Compose/pgsql/SingleTenant/compose-build-dev.yml	Adds EdFi_Ods/EdFi_Master connection strings; trims whitespace.
Docker/V2/Compose/pgsql/SingleTenant/compose-build-binaries.yml	Adds EdFi_Ods/EdFi_Master connection strings for single-tenant pgsql.
Docker/V2/Compose/pgsql/MultiTenant/compose-build-ods-multi-tenant.yml	Adds default + tenant-specific EdFi_Ods/EdFi_Master connection strings.
Docker/V2/Compose/pgsql/MultiTenant/compose-build-idp-dev-multi-tenant.yml	Adds default + tenant-specific EdFi_Ods/EdFi_Master connection strings.
Docker/V2/Compose/pgsql/MultiTenant/compose-build-idp-binaries-multi-tenant.yml	Adds default + tenant-specific EdFi_Ods/EdFi_Master connection strings.
Docker/V2/Compose/pgsql/MultiTenant/compose-build-dev-multi-tenant.yml	Adds default + tenant-specific EdFi_Ods/EdFi_Master connection strings.
Docker/V2/Compose/pgsql/MultiTenant/compose-build-binaries-multi-tenant.yml	Adds default + tenant-specific EdFi_Ods/EdFi_Master connection strings.
Docker/V2/Compose/mssql/SingleTenant/compose-build-ods.yml	Adds EdFi_Ods/EdFi_Master connection strings for single-tenant mssql.
Docker/V2/Compose/mssql/SingleTenant/compose-build-idp-dev.yml	Adds EdFi_Ods/EdFi_Master connection strings for single-tenant mssql.
Docker/V2/Compose/mssql/SingleTenant/compose-build-idp-binaries.yml	Adds EdFi_Ods/EdFi_Master connection strings for single-tenant mssql.
Docker/V2/Compose/mssql/SingleTenant/compose-build-dev.yml	Adds EdFi_Ods/EdFi_Master connection strings for single-tenant mssql.
Docker/V2/Compose/mssql/SingleTenant/compose-build-binaries.yml	Adds EdFi_Ods/EdFi_Master connection strings for single-tenant mssql.
Docker/V2/Compose/mssql/MultiTenant/compose-build-ods-multi-tenant.yml	Adds default + tenant-specific EdFi_Ods/EdFi_Master connection strings.
Docker/V2/Compose/mssql/MultiTenant/compose-build-idp-dev-multi-tenant.yml	Adds default + tenant-specific EdFi_Ods/EdFi_Master connection strings.
Docker/V2/Compose/mssql/MultiTenant/compose-build-idp-binaries-multi-tenant.yml	Adds default + tenant-specific EdFi_Ods/EdFi_Master connection strings.
Docker/V2/Compose/mssql/MultiTenant/compose-build-dev-multi-tenant.yml	Adds default + tenant-specific EdFi_Ods/EdFi_Master connection strings.
Docker/V2/Compose/mssql/MultiTenant/compose-build-binaries-multi-tenant.yml	Adds default + tenant-specific EdFi_Ods/EdFi_Master connection strings.
Application/EdFi.Ods.AdminApi/appsettings.json	Adds defaults for sweep interval and max retries.
Application/EdFi.Ods.AdminApi/appsettings.Development.json	Adds dev defaults for sweep interval and max retries.
Application/EdFi.Ods.AdminApi/Program.cs	Schedules recurring dispatcher (and refactors tenant initialization / scheduler retrieval).
Application/EdFi.Ods.AdminApi/Infrastructure/WebApplicationBuilderExtensions.cs	Registers new jobs and makes ConfigConnectionStringsProvider transient for sandbox provisioning.
Application/EdFi.Ods.AdminApi/Infrastructure/Services/Jobs/CreatePendingDbInstancesDispatcherJob.cs	Adds sweep-based dispatcher job implementation.
Application/EdFi.Ods.AdminApi/Infrastructure/Services/Jobs/CreateInstanceJob.cs	Adds worker job to provision DB + sync OdsInstance + status transitions.
Application/EdFi.Ods.AdminApi/Features/DbInstances/DbInstanceDatabaseNameFormatter.cs	Adds canonical DB-name generation + portable length constant.
Application/EdFi.Ods.AdminApi/Features/DbInstances/AddDbInstance.cs	Schedules CreateInstanceJob from the API and expands validation (chars, uniqueness, portable DB-name length).
Application/EdFi.Ods.AdminApi/E2E Tests/V2/Bruno Admin API E2.0 refactor/v2/DbInstances/POST - DbInstances - Sample Template.bru	Uses unique name per run; improves diagnostics; adjusts ID parsing; sets timeout.
Application/EdFi.Ods.AdminApi/E2E Tests/V2/Bruno Admin API E2.0 refactor/v2/DbInstances/POST - DbInstance - For Delete Test.bru	Removes obsolete pre-delete setup request file.
Application/EdFi.Ods.AdminApi/E2E Tests/V2/Bruno Admin API E2.0 refactor/v2/DbInstances/DELETE - DbInstance - Success.bru.disabled	Adds disabled delete-success E2E with documented CI prerequisites.
Application/EdFi.Ods.AdminApi/E2E Tests/V2/Bruno Admin API E2.0 refactor/v2/DbInstances/DELETE - DbInstance - Pending Status.bru	Removes pending-status delete test file.
Application/EdFi.Ods.AdminApi/E2E Tests/V2/Bruno Admin API E2.0 refactor/v2/DbInstances/DELETE - DbInstance - Not Found.bru	Minor formatting + timeout setting.
Application/EdFi.Ods.AdminApi.UnitTests/Infrastructure/Services/Jobs/CreatePendingDbInstancesDispatcherJobTests.cs	Adds unit tests for dispatcher scheduling and retry gating.
Application/EdFi.Ods.AdminApi.UnitTests/Infrastructure/Services/Jobs/CreateInstanceJobTests.cs	Adds unit tests for worker job behavior, tenant context, and DB-name formatting.
Application/EdFi.Ods.AdminApi.UnitTests/Infrastructure/Helpers/ConfigConnectionStringsProviderTests.cs	Adds unit tests for tenant override behavior in connection string provider.
Application/EdFi.Ods.AdminApi.UnitTests/Features/DbInstances/AddDbInstanceTests.cs	Updates tests for new handler signature and validates job scheduling + new validation rules.
Application/EdFi.Ods.AdminApi.InstanceManagement/Provisioners/SandboxProvisionerBase.cs	Avoids caching EdFi_Master connection string; resolves at call time for multi-tenant jobs.
Application/EdFi.Ods.AdminApi.Common/Settings/AppSettings.cs	Adds settings for sweep interval and max retries.
Application/EdFi.Ods.AdminApi.Common/Infrastructure/MultiTenancy/TenantConfigurationProvider.cs	Adds tenant-level EdFi_Ods and EdFi_Master extraction.
Application/EdFi.Ods.AdminApi.Common/Infrastructure/MultiTenancy/TenantConfiguration.cs	Adds OdsConnectionString and MasterConnectionString properties.
Application/EdFi.Ods.AdminApi.Common/Infrastructure/Jobs/JobConstants.cs	Adds job constants for DbInstance provisioning jobs.
Application/EdFi.Ods.AdminApi.Common/Infrastructure/Helpers/ConfigConnectionStringsProvider.cs	Adds tenant-aware overlay logic and makes mapping dynamic per call.

[Copilot]

This schedules the CreateInstance job directly via scheduler.ScheduleJob(...) without checking whether the same job key was already scheduled. Because the dispatcher can also schedule CreateInstanceJob for newly-inserted Pending rows, there’s a real race where the dispatcher schedules first and this call throws ObjectAlreadyExistsException, causing the API to fail even though the job is already queued. Consider using the existing QuartzJobScheduler.ScheduleJob<TJob>(...) helper (or an explicit CheckExists guard) here for idempotent scheduling.

Suggested change

	var trigger = TriggerBuilder.Create()
	.StartNow()
	.Build();
	var scheduler = await schedulerFactory.GetScheduler();
	await scheduler.ScheduleJob(jobBuilder.Build(), trigger);
	var job = jobBuilder.Build();
	var trigger = TriggerBuilder.Create()
	.StartNow()
	.Build();
	var scheduler = await schedulerFactory.GetScheduler();
	try
	{
	await scheduler.ScheduleJob(job, trigger);
	}
	catch (ObjectAlreadyExistsException)
	{
	// Another path may have already scheduled the same CreateInstanceJob.
	// Treat duplicate scheduling as success so this endpoint remains idempotent.
	}

[josephcampos-gap]

Fixed in this PR. Added a try/catch ObjectAlreadyExistsException guard around scheduler.ScheduleJob(...) so that if the dispatcher has already scheduled the same job key, the duplicate is silently swallowed and the endpoint still returns 202 Accepted.