locale-gen.sh does not work because of missing referenced locales

[JelteF]

The file /usr/share/i18n/locales contains only en_US, but that file references other locale files that are not part of the Mariner installation.

So when you run locale-gen.sh you get the following error:

en_US.ISO-8859-1...[error] cannot open locale definition file `en_GB': No such file or directory

To make locale generation work I had to copy the following locales from my Ubuntu install:

1. en_GB
2. i18n
3. i18n_ctype
4. iso14651_t1
5. iso14651_t1_common
6. translit_* (it was missing many of these, but maybe not all are necessary)

[bossmc]

I just ran into this and found a workaround - install glibc-i18n which (at a whopping 15Mb!) includes all the additional locale definition files.

Still seems like locale-gen.sh should work out of the box without needing to install other packages manually.

[Malateshk007]

@JelteF, Install the package glibc-i18n since it is not pre-installed. We are closing the issue since it is not a bug.

[JelteF]

I really don't understand why you are not considering this a bug. The en_US.ISO-8859-1 locale does not work!

[Malateshk007]

@JelteF , as we informed you, we have created internal ADO bug to handle the issue and we are re-opening this issue.

[JelteF]

To document my clarification of the problem here too. The locale-gen.sh command that's included in the base image fails when run.

❯ docker run -it mcr.microsoft.com/cbl-mariner/base/core:2.0 /bin/bash
root [ / ]# locale-gen.sh
Generating locales...
  en_US.ISO-8859-1...[error] cannot open locale definition file `en_GB': No such file or directory

I feel like there are two options to resolve this:

1. Don't include locale-gen.sh in the base image and only install it when glibc-i18n is installed. And the same should then be done for the file /usr/share/i18n/locales/en_US.
2. Make sure that locale-gen.sh doesn't fail when run by removing the en_US lines from /etc/locale-gen.conf. (and probably also removing /usr/share/i18n/locales/en_US.
3. Make sure that locale-gen.sh doesn't fail when run by adding all the necessary files for en_US to be compiled to the base image.

[omerfsen]

Issue is reported on 2022 and now Azure Flexible Servers are using CBL Mariner OS (AzureLinux) and we can't create DBs with UTF8 encoding for en_GB

psql:/tmp/dbSetup.sql:55: ERROR:  encoding "UTF8" does not match locale "en_GB.utf8"
DETAIL:  The chosen LC_CTYPE setting requires encoding "LATIN9".

as collencoding is 16 (it should be 6 for UTF8 but 16 for LATIN9)

SELECT *                                                                                                                        
FROM pg_collation
WHERE collname like 'en_GB.utf8';
  oid  |  collname  | collnamespace | collowner | collprovider | collisdeterministic | collencoding | collcollate | collctype  | collversion 
-------+------------+---------------+-----------+--------------+---------------------+--------------+-------------+------------+-------------
 12518 | en_GB.utf8 |            11 |        10 | c            | t                   |           16 | en_GB.utf8  | en_GB.utf8 | 2.35

[omerf-hmcts]

Issue is reported on 2022 and now Azure Flexible Servers are using CBL Mariner OS (AzureLinux) and we can't create DBs with UTF8 encoding for en_GB

psql:/tmp/dbSetup.sql:55: ERROR:  encoding "UTF8" does not match locale "en_GB.utf8"
DETAIL:  The chosen LC_CTYPE setting requires encoding "LATIN9".

as collencoding is 16 (it should be 6 for UTF8 but 16 for LATIN9)

SELECT *                                                                                                                        
FROM pg_collation
WHERE collname like 'en_GB.utf8';
  oid  |  collname  | collnamespace | collowner | collprovider | collisdeterministic | collencoding | collcollate | collctype  | collversion 
-------+------------+---------------+-----------+--------------+---------------------+--------------+-------------+------------+-------------
 12518 | en_GB.utf8 |            11 |        10 | c            | t                   |           16 | en_GB.utf8  | en_GB.utf8 | 2.35

As on Azure Flexible Server you can't install a new package (ie. glibc-i18n or others) I think that needs to be handled on Azure side

[ext-omerf]

Issue is still valid for Mariner CBL 2.0 docker image (distroless or distrobased) and https://github.com/microsoft/openjdk-docker/blob/main/docker/mariner/Dockerfile.msopenjdk-21-jdk as this JDK bases on Mariner CBL 2.0 (both distro based and distroless)

So apps running on those docker images cant have locales other than en_US.UTF-8

[ext-omerf]

Also from above only en_US locale exists as file