Retrieve operator & HO log via http api endpoint, without journald

[0405ysj]

This allows to retrieve operator & Host Orchestrator log via HTTP/HTTPS API endpoint through nginx, but it doesn't rely on systemd-journal-gatewayd.

Context: b/385123612

Verified on podman:

• GET /_journal/entries?_SYSTEMD_UNIT=cuttlefish-operator.service
• GET /_journal/entries?_SYSTEMD_UNIT=cuttlefish-host_orchestrator.service
• GET /hostlogs/cuttlefish-operator.service
• GET /hostlogs/cuttlefish-host_orchestrator.service
• cat /run/cuttlefish/operator.log
• cat /run/cuttelfish/host_orchestrator.log
• sudo systemctl status cuttlefish-operator
• sudo systemctl status cuttlefish-host_orchestrator

Verified on docker:

• GET /hostlogs/cuttlefish-operator.service
• GET /hostlogs/cuttlefish-host_orchestrator.service
• cat /run/cuttlefish/operator.log
• cat /run/cuttelfish/host_orchestrator.log

[ser-io]

https://github.com/google/android-cuttlefish/pull/1832/files#r2581910272

[ser-io]

This workaround LGTM.

However, do not merge the PR until it's actually needed. This new code is only needed for Docker, hence it should be merged after Docker is integrated.

[0405ysj]

However, do not merge the PR until it's actually needed. This new code is only needed for Docker, hence it should be merged after Docker is integrated.

SG. I'm currently trying GPU utilization on podman. Let's decide afterwards, by whether GPU utilization works or not.

[ser-io]

However, do not merge the PR until it's actually needed. This new code is only needed for Docker, hence it should be merged after Docker is integrated.

SG. I'm currently trying GPU utilization on podman. Let's decide afterwards, by whether GPU utilization works or not.

Providing HO/Operator via http is only needed by Tradefed and it's not a blocker. This mean, this PR would be needed when there are actual ATP tests using Docker, that would be the time to land this PR.

[ikicha]

This workaround LGTM.

However, do not merge the PR until it's actually needed. This new code is only needed for Docker, hence it should be merged after Docker is integrated.

QQ: This change works well even outside docker without regression, doesn't it? If so, I think merging it now wouldn't be harmful..

[ser-io]

This workaround LGTM.
However, do not merge the PR until it's actually needed. This new code is only needed for Docker, hence it should be merged after Docker is integrated.

QQ: This change works well even outside docker without regression, doesn't it? If so, I think merging it now wouldn't be harmful..

Providing HO/Operator via http is only needed by Tradefed, until there's an actual ATP test using the new Docker container solution this code is not needed hence landing this PR can wait till then.

[0405ysj]

Providing HO/Operator via http is only needed by Tradefed, until there's an actual ATP test using the new Docker container solution this code is not needed hence landing this PR can wait till then.

The circumstance is changed. Now I validated GPU can be utilized on podman instance. Also I validated a container image is available to be run on both docker and podman.

Based on recent validations, I think the best option is deploying single container image which can be run on both, and keeping it stable via doing E2E test on both. We all know that there are some places to be able to use podman, not docker. On the other hand, Cloud Orchestrator via on-premise configuration is getting to have real use cases, but docker is the only integrated solution today.

The problem starts from sharing container image from both. Podman can start init process for itself, but docker can't. Even if it's possible, not recommended. But without starting init process, systemd-journal-gatewayd won't be working properly. Fortunately, applying this PR makes available to expose operator or HO logs instead of systemd-journal-gatewayd. And I believe any use cases will require this PR soon, with converting their setup into container-friendly setup. Thus, why don't we apply this method?

BTW, I want to raise a concern for using systemd-journal-gatewayd. IIKC there are some cases that using systemd-journal-gatewayd to retrieve kernel log or other containers' log. Are these really something we want to support, by making systemd-journal-gatewayd be dependent of Host Orchestrator? IMO it's out of scope of Host Orchestrator, and also risky to expose those information via network by just installing Host Orchestrator. How do you think? @ser-io

[ser-io]

And I believe any use cases will require this PR soon, with converting their setup into container-friendly setup. Thus, why don't we apply this method?

When will "soon" be? This is not clear, hence the position to wait and then merge later when needed. The moment there's an actual use case for this change, this PR can definitely land.