chore: Update Dockerfile to ubi9/ubi-micro:9.4

[ckadner]

Re: #42 (review)

We should use a specific version to ensure reproducible builds and/or to trace back in time when something last worked how/why what changed when ...

Alternatively, to stay on UBI 8, we could use the "latest" UBI 8 image ubi8/ubi-micro:8.10-7

[oss-prow-bot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: ckadner

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[ckadner]

TODO: Create architectural decision record issue to summarize why we want to use latest images:

Cons of using latest:

• MM image builds might fail unexpectedly
• hard to debug/trace back when failing changes happened
• last working image may not be possible to recreate for a new MM release without any changes
• last working image might not be available even if identified
• even if builds succeed, there might be unexpected changes in behavior which happens quite often with 3rd party dependencies of Python packages (on a tangent: when we update to Go 1.19 a while ago, the go get CLI tools install commands "succeed" but no longer installed executables, though we would not use ubi8/go-toolset:latest)

Pros of using latest:

• keeping specific version tag, we keep outdated code, old vulnerabilities
• we don't have to keep updating to newer patched versions of base images when upstream base images get updated frequently

Builder vs final images:

• using latest on a final build stage (not builder image) should be fairly "safe", no 3rd party libraries are installed, just files copied from builder image.
• however, it's the base of the builder images where most problems/vulnerabilities are coming in/need to be fixed, are getting fixed upstream,
• so really, all build stages need to build on a latest upstream (UBI) base image

Finally, we should follow the same strategy for all MM images consistently.

[spolti]

For go-toolset I do agree, no latest tag.

I didn't understand this last part:

so really, all build stages need to build on a latest upstream (UBI) base image

[ckadner]

Let's hold off on updating to UBI 9 for now. As discussed on kserve/modelmesh-serving#508 (comment) we will update the final build stages in all MM images to use UBI 8 latest.