[FEATURE-NOK] Open Edge Orchestrator - Upgrade failure: Cert-manager 0.1.2 → 0.1.4 fails when Observability 0.1.3 → 0.1.5 is present

[nagarjunaha056]

Bug Description

Upgrade of the observability and cert-manager packages fails due to dependency/conflict issues when both components coexist.

Observed Behavior:

1. Upgrade of Observability from 0.1.3 → 0.1.5 fails.

2. Upgrade of Cert-manager from 0.1.2 → 0.1.4 fails when Observability is still installed.

3. Upgrade of Cert-manager succeeds only when Observability is deleted first (workaround).

Expected Behavior:

1. The system should allow sequential upgrades — Cert-manager should upgrade successfully even if the Observability package is still present.
2. Observability upgrade should also complete smoothly after the cert-manager upgrade without requiring manual deletion.
   OR
3. When upgrading the Observability package, if it depends on a newer Cert-manager version:
   a. The system should automatically upgrade Cert-manager first.
   b. Once Cert-manager upgrade succeeds, proceed with Observability package upgrade.

User should not need to manually delete or manage dependency packages.

System Setup

Orchestrator v3.1.2

Reproducible Steps

1. Deploy Observability v0.1.3 and there by the system deploys Cert-manager v0.1.2.
2. Attempt to upgrade Observability to v0.1.5.
   → Upgrade fails.
3. Attempt to upgrade Cert-manager to v0.1.4 while Observability v0.1.3 is present.
   → Upgrade fails.
4. Delete Observability package.
5. Retry Cert-manager upgrade.
   → Upgrade succeeds.

Workaround:
Manually delete the Observability package before upgrading Cert-manager.

Impact:
Users are forced to remove the Observability package to perform Cert-manager upgrades, which results in monitoring downtime and breaks the ideal upgrade flow.

Root Cause Analysis

No response

[scottmbaker]

I think this should be addressed as a feature request, not a bug fix. I also think the feature requires more investigation and design work. My concerns:

Expected Behavior 1+2 - allow cert-manager to be manually upgraded by the user

1. Consider that if we allow cert-manager to be upgraded first, then it may be upgraded to a version that is incompatible with the current observability package. This could lead to temporary instability, or it could even lead to a failure for the observability package to be upgradeable if, for example, observability entered an unhealthy state.
2. What if the user is unaware of the dependencies? For example, let's say they update cert-manager and do not know that observability depends on it. Will they accidentally break observability and not notice it? Will there be any indicators in the GUI to let them know that additional packages need to be upgraded?

Expected Behavior 3 - upgrading observability should automatically upgrade cert-manager

1. What happens in the case of partial failure? Are we able to roll back?
2. What if there are two services that rely on cert-manager? For example, we upgrade observability which triggers an automatic upgrade of cert-manager, but a third hypothetical package, package3, also relies on cert-manager. Does the upgrade propagate to package3 as well? What if there is no package3 in the catalog that is compatible with the new cert-manager?

We need to write up an ADR that describes both Expected Behavior 1+2 and Expected Behavior 3 and discusses the various corner cases that might occur. Then we need to make a decision on which one is the right one to implement.

Until then, I feel the system is working as intended -- it is displaying an error message that an upgrade cannot be performed while there are dependencies.

[soniabha-intc]

@nagarjunaha056 We are changing this to a feature request.

[nagarjunaha056]

@soniabha-intc Any updates on this feature request?