Fresh clone can't connect to mariadb

[roddds]

Apologies if I'm missing any critical debug information. No files in the repository have been changed. The interrupted stack traces at the end continue indefinitely, several times per second. I checked out the most recent release tag, 0.7.2, and got the same results:

$ git --version
git version 2.25.1

$ docker --version
Docker version 20.10.22, build 3a2c30b

$ docker-compose --version
docker-compose version 1.29.2, build 5becea4c

$ uname -a
Linux cosmos 5.15.0-58-generic #64~20.04.1-Ubuntu SMP Fri Jan 6 16:42:31 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

$ sudo sysctl -w vm.max_map_count=262144
[sudo] password for rodrigo: 
vm.max_map_count = 262144

$ sudo docker-compose up
Creating network "docker-compose_default" with the default driver
Creating docker-compose_mariadb_1       ... done
Creating docker-compose_elasticsearch_1 ... done
Creating docker-compose_kibiter_1       ... done
Creating docker-compose_hatstall_1      ... done
Creating docker-compose_mordred_1       ... done
Attaching to docker-compose_elasticsearch_1, docker-compose_mariadb_1, docker-compose_kibiter_1, docker-compose_mordred_1, docker-compose_hatstall_1
elasticsearch_1  | OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
hatstall_1       | Running Hatstall
mariadb_1        | 2023-01-25 20:24:12+00:00 [Note] [Entrypoint]: Entrypoint script for MariaDB Server 1:10.6.11+maria~ubu2004 started.
mariadb_1        | 2023-01-25 20:24:12+00:00 [Note] [Entrypoint]: Switching to dedicated user 'mysql'
mariadb_1        | 2023-01-25 20:24:12+00:00 [Note] [Entrypoint]: Entrypoint script for MariaDB Server 1:10.6.11+maria~ubu2004 started.
mariadb_1        | 2023-01-25 20:24:12+00:00 [Note] [Entrypoint]: Initializing database files
hatstall_1       | Site 000-default disabled.
hatstall_1       | To activate the new configuration, you need to run:
hatstall_1       |   service apache2 reload
hatstall_1       | Enabling site apache-hatstall.
hatstall_1       | To activate the new configuration, you need to run:
hatstall_1       |   service apache2 reload
hatstall_1       | Considering dependency setenvif for ssl:
hatstall_1       | Module setenvif already enabled
hatstall_1       | Considering dependency mime for ssl:
hatstall_1       | Module mime already enabled
hatstall_1       | Considering dependency socache_shmcb for ssl:
hatstall_1       | Enabling module socache_shmcb.
hatstall_1       | Enabling module ssl.
hatstall_1       | See /usr/share/doc/apache2/README.Debian.gz on how to configure SSL and create self-signed certificates.
hatstall_1       | To activate the new configuration, you need to run:
hatstall_1       |   service apache2 restart
hatstall_1       | Django configured for deployment (secret, debug, allowed_hosts) in django_hatstall/settings.py
hatstall_1       | No changes detected
elasticsearch_1  | [2023-01-25T20:24:14,220][INFO ][o.e.e.NodeEnvironment    ] [U1YqLnr] using [1] data paths, mounts [[/ (overlay)]], net usable_space [97.6gb], net total_space [466.2gb], types [overlay]
elasticsearch_1  | [2023-01-25T20:24:14,222][INFO ][o.e.e.NodeEnvironment    ] [U1YqLnr] heap size [1.9gb], compressed ordinary object pointers [true]
elasticsearch_1  | [2023-01-25T20:24:14,223][INFO ][o.e.n.Node               ] [U1YqLnr] node name derived from node ID [U1YqLnr8TPOTu09tenvQag]; set [node.name] to override
elasticsearch_1  | [2023-01-25T20:24:14,224][INFO ][o.e.n.Node               ] [U1YqLnr] version[6.8.6], pid[1], build[oss/docker/3d9f765/2019-12-13T17:11:52.013738Z], OS[Linux/5.15.0-58-generic/amd64], JVM[AdoptOpenJDK/OpenJDK 64-Bit Server VM/13.0.1/13.0.1+9]
elasticsearch_1  | [2023-01-25T20:24:14,224][INFO ][o.e.n.Node               ] [U1YqLnr] JVM arguments [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.io.tmpdir=/tmp/elasticsearch-1012499374263332485, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -Djava.locale.providers=COMPAT, -XX:UseAVX=2, -Xms2g, -Xmx2g, -Des.path.home=/usr/share/elasticsearch, -Des.path.conf=/usr/share/elasticsearch/config, -Des.distribution.flavor=oss, -Des.distribution.type=docker]
hatstall_1       | Operations to perform:
hatstall_1       |   Apply all migrations: admin, auth, contenttypes, sessions
hatstall_1       | Running migrations:
hatstall_1       |   Applying contenttypes.0001_initial... OK
hatstall_1       |   Applying auth.0001_initial... OK
hatstall_1       |   Applying admin.0001_initial... OK
hatstall_1       |   Applying admin.0002_logentry_remove_auto_add... OK
hatstall_1       |   Applying admin.0003_logentry_add_action_flag_choices... OK
hatstall_1       |   Applying contenttypes.0002_remove_content_type_name... OK
hatstall_1       |   Applying auth.0002_alter_permission_name_max_length... OK
hatstall_1       |   Applying auth.0003_alter_user_email_max_length... OK
hatstall_1       |   Applying auth.0004_alter_user_username_opts... OK
hatstall_1       |   Applying auth.0005_alter_user_last_login_null... OK
hatstall_1       |   Applying auth.0006_require_contenttypes_0002... OK
hatstall_1       |   Applying auth.0007_alter_validators_add_error_messages... OK
hatstall_1       |   Applying auth.0008_alter_user_username_max_length... OK
hatstall_1       |   Applying auth.0009_alter_user_last_name_max_length... OK
hatstall_1       |   Applying auth.0010_alter_group_name_max_length... OK
hatstall_1       |   Applying auth.0011_update_proxy_permissions... OK
hatstall_1       |   Applying auth.0012_alter_user_first_name_max_length... OK
hatstall_1       |   Applying sessions.0001_initial... OK
elasticsearch_1  | [2023-01-25T20:24:14,864][INFO ][o.e.p.PluginsService     ] [U1YqLnr] loaded module [aggs-matrix-stats]
elasticsearch_1  | [2023-01-25T20:24:14,865][INFO ][o.e.p.PluginsService     ] [U1YqLnr] loaded module [analysis-common]
elasticsearch_1  | [2023-01-25T20:24:14,865][INFO ][o.e.p.PluginsService     ] [U1YqLnr] loaded module [ingest-common]
elasticsearch_1  | [2023-01-25T20:24:14,865][INFO ][o.e.p.PluginsService     ] [U1YqLnr] loaded module [ingest-geoip]
elasticsearch_1  | [2023-01-25T20:24:14,865][INFO ][o.e.p.PluginsService     ] [U1YqLnr] loaded module [ingest-user-agent]
elasticsearch_1  | [2023-01-25T20:24:14,865][INFO ][o.e.p.PluginsService     ] [U1YqLnr] loaded module [lang-expression]
elasticsearch_1  | [2023-01-25T20:24:14,865][INFO ][o.e.p.PluginsService     ] [U1YqLnr] loaded module [lang-mustache]
elasticsearch_1  | [2023-01-25T20:24:14,865][INFO ][o.e.p.PluginsService     ] [U1YqLnr] loaded module [lang-painless]
elasticsearch_1  | [2023-01-25T20:24:14,865][INFO ][o.e.p.PluginsService     ] [U1YqLnr] loaded module [mapper-extras]
elasticsearch_1  | [2023-01-25T20:24:14,865][INFO ][o.e.p.PluginsService     ] [U1YqLnr] loaded module [parent-join]
elasticsearch_1  | [2023-01-25T20:24:14,865][INFO ][o.e.p.PluginsService     ] [U1YqLnr] loaded module [percolator]
elasticsearch_1  | [2023-01-25T20:24:14,865][INFO ][o.e.p.PluginsService     ] [U1YqLnr] loaded module [rank-eval]
elasticsearch_1  | [2023-01-25T20:24:14,865][INFO ][o.e.p.PluginsService     ] [U1YqLnr] loaded module [reindex]
elasticsearch_1  | [2023-01-25T20:24:14,865][INFO ][o.e.p.PluginsService     ] [U1YqLnr] loaded module [repository-url]
elasticsearch_1  | [2023-01-25T20:24:14,865][INFO ][o.e.p.PluginsService     ] [U1YqLnr] loaded module [transport-netty4]
elasticsearch_1  | [2023-01-25T20:24:14,866][INFO ][o.e.p.PluginsService     ] [U1YqLnr] loaded module [tribe]
elasticsearch_1  | [2023-01-25T20:24:14,866][INFO ][o.e.p.PluginsService     ] [U1YqLnr] no plugins loaded
mariadb_1        | 
mariadb_1        | 
mariadb_1        | PLEASE REMEMBER TO SET A PASSWORD FOR THE MariaDB root USER !
mariadb_1        | To do so, start the server, then issue the following command:
mariadb_1        | 
mariadb_1        | '/usr/bin/mysql_secure_installation'
mariadb_1        | 
mariadb_1        | which will also give you the option of removing the test
mariadb_1        | databases and anonymous user created by default.  This is
mariadb_1        | strongly recommended for production servers.
mariadb_1        | 
mariadb_1        | See the MariaDB Knowledgebase at https://mariadb.com/kb
mariadb_1        | 
mariadb_1        | Please report any problems at https://mariadb.org/jira
mariadb_1        | 
mariadb_1        | The latest information about MariaDB is available at https://mariadb.org/.
mariadb_1        | 
mariadb_1        | Consider joining MariaDB's strong and vibrant community:
mariadb_1        | https://mariadb.org/get-involved/
mariadb_1        | 
mariadb_1        | 2023-01-25 20:24:15+00:00 [Note] [Entrypoint]: Database files initialized
mariadb_1        | 2023-01-25 20:24:15+00:00 [Note] [Entrypoint]: Starting temporary server
mariadb_1        | 2023-01-25 20:24:15+00:00 [Note] [Entrypoint]: Waiting for server startup
mariadb_1        | 2023-01-25 20:24:15 0 [Note] mariadbd (server 10.6.11-MariaDB-1:10.6.11+maria~ubu2004) starting as process 132 ...
mariadb_1        | 2023-01-25 20:24:15 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
mariadb_1        | 2023-01-25 20:24:15 0 [Note] InnoDB: Number of pools: 1
mariadb_1        | 2023-01-25 20:24:15 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions
mariadb_1        | 2023-01-25 20:24:15 0 [Note] mariadbd: O_TMPFILE is not supported on /tmp (disabling future attempts)
hatstall_1       | 
hatstall_1       | 163 static files copied to '/home/grimoirelab/grimoirelab-hatstall/django-hatstall/static'.
mariadb_1        | 2023-01-25 20:24:15 0 [Note] InnoDB: Using Linux native AIO
mariadb_1        | 2023-01-25 20:24:15 0 [Note] InnoDB: Initializing buffer pool, total size = 134217728, chunk size = 134217728
mariadb_1        | 2023-01-25 20:24:15 0 [Note] InnoDB: Completed initialization of buffer pool
mariadb_1        | 2023-01-25 20:24:15 0 [Note] InnoDB: 128 rollback segments are active.
mariadb_1        | 2023-01-25 20:24:15 0 [Note] InnoDB: Creating shared tablespace for temporary tables
mariadb_1        | 2023-01-25 20:24:15 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
mariadb_1        | 2023-01-25 20:24:15 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
mariadb_1        | 2023-01-25 20:24:15 0 [Note] InnoDB: 10.6.11 started; log sequence number 42120; transaction id 14
mariadb_1        | 2023-01-25 20:24:15 0 [Note] Plugin 'FEEDBACK' is disabled.
mariadb_1        | 2023-01-25 20:24:15 0 [Warning] 'user' entry 'root@63097e223174' ignored in --skip-name-resolve mode.
mariadb_1        | 2023-01-25 20:24:15 0 [Warning] 'proxies_priv' entry '@% root@63097e223174' ignored in --skip-name-resolve mode.
mariadb_1        | 2023-01-25 20:24:15 0 [Note] mariadbd: ready for connections.
mariadb_1        | Version: '10.6.11-MariaDB-1:10.6.11+maria~ubu2004'  socket: '/run/mysqld/mysqld.sock'  port: 0  mariadb.org binary distribution
hatstall_1       | User for django admin created: admin/admin as login
kibiter_1        | {"type":"log","@timestamp":"2023-01-25T20:24:15Z","tags":["status","plugin:[email protected]","info"],"pid":1,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
kibiter_1        | {"type":"log","@timestamp":"2023-01-25T20:24:15Z","tags":["status","plugin:[email protected]","info"],"pid":1,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
kibiter_1        | {"type":"log","@timestamp":"2023-01-25T20:24:15Z","tags":["status","plugin:[email protected]","info"],"pid":1,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
kibiter_1        | {"type":"log","@timestamp":"2023-01-25T20:24:15Z","tags":["status","plugin:[email protected]","info"],"pid":1,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
kibiter_1        | {"type":"log","@timestamp":"2023-01-25T20:24:15Z","tags":["status","plugin:[email protected]","info"],"pid":1,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
kibiter_1        | {"type":"log","@timestamp":"2023-01-25T20:24:15Z","tags":["status","plugin:[email protected]","info"],"pid":1,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
kibiter_1        | {"type":"log","@timestamp":"2023-01-25T20:24:15Z","tags":["status","plugin:[email protected]","info"],"pid":1,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
kibiter_1        | {"type":"log","@timestamp":"2023-01-25T20:24:15Z","tags":["error","elasticsearch","admin"],"pid":1,"message":"Request error, retrying\nHEAD http://elasticsearch:9200/ => connect ECONNREFUSED 192.168.160.2:9200"}
kibiter_1        | {"type":"log","@timestamp":"2023-01-25T20:24:15Z","tags":["warning","elasticsearch","admin"],"pid":1,"message":"Unable to revive connection: http://elasticsearch:9200/"}
kibiter_1        | {"type":"log","@timestamp":"2023-01-25T20:24:15Z","tags":["warning","elasticsearch","admin"],"pid":1,"message":"No living connections"}
kibiter_1        | {"type":"log","@timestamp":"2023-01-25T20:24:15Z","tags":["status","plugin:[email protected]","error"],"pid":1,"state":"red","message":"Status changed from yellow to red - Unable to connect to Elasticsearch.","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
hatstall_1       | AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 192.168.160.6. Set the 'ServerName' directive globally to suppress this message
hatstall_1       | AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 192.168.160.6. Set the 'ServerName' directive globally to suppress this message
hatstall_1       | [Wed Jan 25 20:24:15.988463 2023] [mpm_event:notice] [pid 71:tid 139860522615936] AH00489: Apache/2.4.38 (Debian) OpenSSL/1.1.1d mod_wsgi/4.6.5 Python/3.7 configured -- resuming normal operations
hatstall_1       | [Wed Jan 25 20:24:15.988537 2023] [core:notice] [pid 71:tid 139860522615936] AH00094: Command line: '/usr/sbin/apache2 -D FOREGROUND'
mariadb_1        | 2023-01-25 20:24:16+00:00 [Note] [Entrypoint]: Temporary server started.
elasticsearch_1  | [2023-01-25T20:24:17,147][INFO ][o.e.d.DiscoveryModule    ] [U1YqLnr] using discovery type [zen] and host providers [settings]
elasticsearch_1  | [2023-01-25T20:24:17,466][INFO ][o.e.n.Node               ] [U1YqLnr] initialized
elasticsearch_1  | [2023-01-25T20:24:17,466][INFO ][o.e.n.Node               ] [U1YqLnr] starting ...
elasticsearch_1  | [2023-01-25T20:24:17,579][INFO ][o.e.t.TransportService   ] [U1YqLnr] publish_address {192.168.160.2:9300}, bound_addresses {0.0.0.0:9300}
elasticsearch_1  | [2023-01-25T20:24:17,588][INFO ][o.e.b.BootstrapChecks    ] [U1YqLnr] bound or publishing to a non-loopback address, enforcing bootstrap checks
mariadb_1        | 2023-01-25 20:24:17+00:00 [Note] [Entrypoint]: Securing system users (equivalent to running mysql_secure_installation)
mariadb_1        | 
mariadb_1        | 2023-01-25 20:24:17+00:00 [Note] [Entrypoint]: Stopping temporary server
mariadb_1        | 2023-01-25 20:24:17 0 [Note] mariadbd (initiated by: unknown): Normal shutdown
mariadb_1        | 2023-01-25 20:24:17 0 [Note] InnoDB: FTS optimize thread exiting.
mariadb_1        | 2023-01-25 20:24:17 0 [Note] InnoDB: Starting shutdown...
mariadb_1        | 2023-01-25 20:24:17 0 [Note] InnoDB: Dumping buffer pool(s) to /var/lib/mysql/ib_buffer_pool
mariadb_1        | 2023-01-25 20:24:17 0 [Note] InnoDB: Buffer pool(s) dump completed at 230125 20:24:17
mariadb_1        | 2023-01-25 20:24:18 0 [Note] InnoDB: Removed temporary tablespace data file: "./ibtmp1"
mariadb_1        | 2023-01-25 20:24:18 0 [Note] InnoDB: Shutdown completed; log sequence number 42132; transaction id 15
mariadb_1        | 2023-01-25 20:24:18 0 [Note] mariadbd: Shutdown complete
mariadb_1        | 
mariadb_1        | 2023-01-25 20:24:18+00:00 [Note] [Entrypoint]: Temporary server stopped
mariadb_1        | 
mariadb_1        | 2023-01-25 20:24:18+00:00 [Note] [Entrypoint]: MariaDB init process done. Ready for start up.
mariadb_1        | 
mariadb_1        | 2023-01-25 20:24:18 0 [Note] mariadbd (server 10.6.11-MariaDB-1:10.6.11+maria~ubu2004) starting as process 1 ...
mariadb_1        | 2023-01-25 20:24:18 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
mariadb_1        | 2023-01-25 20:24:18 0 [Note] InnoDB: Number of pools: 1
mariadb_1        | 2023-01-25 20:24:18 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions
mariadb_1        | 2023-01-25 20:24:18 0 [Note] mariadbd: O_TMPFILE is not supported on /tmp (disabling future attempts)
mariadb_1        | 2023-01-25 20:24:18 0 [Note] InnoDB: Using Linux native AIO
mariadb_1        | 2023-01-25 20:24:18 0 [Note] InnoDB: Initializing buffer pool, total size = 134217728, chunk size = 134217728
mariadb_1        | 2023-01-25 20:24:18 0 [Note] InnoDB: Completed initialization of buffer pool
mariadb_1        | 2023-01-25 20:24:18 0 [Note] InnoDB: 128 rollback segments are active.
mariadb_1        | 2023-01-25 20:24:18 0 [Note] InnoDB: Creating shared tablespace for temporary tables
mariadb_1        | 2023-01-25 20:24:18 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
mariadb_1        | 2023-01-25 20:24:18 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
mariadb_1        | 2023-01-25 20:24:18 0 [Note] InnoDB: 10.6.11 started; log sequence number 42132; transaction id 14
mariadb_1        | 2023-01-25 20:24:18 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
mariadb_1        | 2023-01-25 20:24:18 0 [Note] Plugin 'FEEDBACK' is disabled.
mariadb_1        | 2023-01-25 20:24:18 0 [Note] InnoDB: Buffer pool(s) load completed at 230125 20:24:18
mariadb_1        | 2023-01-25 20:24:18 0 [Warning] You need to use --log-bin to make --expire-logs-days or --binlog-expire-logs-seconds work.
mariadb_1        | 2023-01-25 20:24:18 0 [Note] Server socket created on IP: '0.0.0.0'.
mariadb_1        | 2023-01-25 20:24:18 0 [Note] Server socket created on IP: '::'.
mariadb_1        | 2023-01-25 20:24:18 0 [Note] mariadbd: ready for connections.
mariadb_1        | Version: '10.6.11-MariaDB-1:10.6.11+maria~ubu2004'  socket: '/run/mysqld/mysqld.sock'  port: 3306  mariadb.org binary distribution
kibiter_1        | {"type":"log","@timestamp":"2023-01-25T20:24:18Z","tags":["warning","elasticsearch","admin"],"pid":1,"message":"Unable to revive connection: http://elasticsearch:9200/"}
kibiter_1        | {"type":"log","@timestamp":"2023-01-25T20:24:18Z","tags":["warning","elasticsearch","admin"],"pid":1,"message":"No living connections"}
elasticsearch_1  | [2023-01-25T20:24:20,628][INFO ][o.e.c.s.MasterService    ] [U1YqLnr] zen-disco-elected-as-master ([0] nodes joined), reason: new_master {U1YqLnr}{U1YqLnr8TPOTu09tenvQag}{hcq_sVHzRKioNxddK6q8Pw}{192.168.160.2}{192.168.160.2:9300}
elasticsearch_1  | [2023-01-25T20:24:20,631][INFO ][o.e.c.s.ClusterApplierService] [U1YqLnr] new_master {U1YqLnr}{U1YqLnr8TPOTu09tenvQag}{hcq_sVHzRKioNxddK6q8Pw}{192.168.160.2}{192.168.160.2:9300}, reason: apply cluster state (from master [master {U1YqLnr}{U1YqLnr8TPOTu09tenvQag}{hcq_sVHzRKioNxddK6q8Pw}{192.168.160.2}{192.168.160.2:9300} committed version [1] source [zen-disco-elected-as-master ([0] nodes joined)]])
elasticsearch_1  | [2023-01-25T20:24:20,686][INFO ][o.e.h.n.Netty4HttpServerTransport] [U1YqLnr] publish_address {192.168.160.2:9200}, bound_addresses {0.0.0.0:9200}
elasticsearch_1  | [2023-01-25T20:24:20,686][INFO ][o.e.n.Node               ] [U1YqLnr] started
elasticsearch_1  | [2023-01-25T20:24:20,697][INFO ][o.e.g.GatewayService     ] [U1YqLnr] recovered [0] indices into cluster_state
kibiter_1        | {"type":"log","@timestamp":"2023-01-25T20:24:20Z","tags":["status","plugin:[email protected]","info"],"pid":1,"state":"green","message":"Status changed from red to green - Ready","prevState":"red","prevMsg":"Unable to connect to Elasticsearch."}
kibiter_1        | {"type":"log","@timestamp":"2023-01-25T20:24:21Z","tags":["info","migrations"],"pid":1,"message":"Creating index .kibana_1."}
elasticsearch_1  | [2023-01-25T20:24:21,123][INFO ][o.e.c.m.MetaDataCreateIndexService] [U1YqLnr] [.kibana_1] creating index, cause [api], templates [], shards [1]/[1], mappings [doc]
elasticsearch_1  | [2023-01-25T20:24:21,129][INFO ][o.e.c.r.a.AllocationService] [U1YqLnr] updating number_of_replicas to [0] for indices [.kibana_1]
elasticsearch_1  | [2023-01-25T20:24:21,326][INFO ][o.e.c.r.a.AllocationService] [U1YqLnr] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[.kibana_1][0]] ...]).
kibiter_1        | {"type":"log","@timestamp":"2023-01-25T20:24:21Z","tags":["info","migrations"],"pid":1,"message":"Pointing alias .kibana to .kibana_1."}
kibiter_1        | {"type":"log","@timestamp":"2023-01-25T20:24:21Z","tags":["info","migrations"],"pid":1,"message":"Finished in 386ms."}
kibiter_1        | {"type":"log","@timestamp":"2023-01-25T20:24:21Z","tags":["listening","info"],"pid":1,"message":"Server running at http://0.0.0.0:5601"}
mordred_1        | Exception in thread Global tasks:
mordred_1        | Traceback (most recent call last):
mordred_1        |   File "/usr/local/lib/python3.8/site-packages/urllib3/connection.py", line 174, in _new_conn
mordred_1        |     conn = connection.create_connection(
mordred_1        |   File "/usr/local/lib/python3.8/site-packages/urllib3/util/connection.py", line 95, in create_connection
mordred_1        |     raise err
mordred_1        |   File "/usr/local/lib/python3.8/site-packages/urllib3/util/connection.py", line 85, in create_connection
mordred_1        |     sock.connect(sa)
mordred_1        | ConnectionRefusedError: [Errno 111] Connection refused
mordred_1        | 
mordred_1        | During handling of the above exception, another exception occurred:
mordred_1        | 
mordred_1        | Traceback (most recent call last):
mordred_1        |   File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 703, in urlopen
mordred_1        |     httplib_response = self._make_request(
mordred_1        |   File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 398, in _make_request
mordred_1        |     conn.request(method, url, **httplib_request_kw)
mordred_1        |   File "/usr/local/lib/python3.8/site-packages/urllib3/connection.py", line 239, in request
mordred_1        |     super(HTTPConnection, self).request(method, url, body=body, headers=headers)
mordred_1        |   File "/usr/local/lib/python3.8/http/client.py", line 1256, in request
mordred_1        |     self._send_request(method, url, body, headers, encode_chunked)
mordred_1        |   File "/usr/local/lib/python3.8/http/client.py", line 1302, in _send_request
mordred_1        |     self.endheaders(body, encode_chunked=encode_chunked)
mordred_1        |   File "/usr/local/lib/python3.8/http/client.py", line 1251, in endheaders
mordred_1        |     self._send_output(message_body, encode_chunked=encode_chunked)
mordred_1        |   File "/usr/local/lib/python3.8/http/client.py", line 1011, in _send_output
mordred_1        |     self.send(msg)
mordred_1        |   File "/usr/local/lib/python3.8/http/client.py", line 951, in send
mordred_1        |     self.connect()
mordred_1        |   File "/usr/local/lib/python3.8/site-packages/urllib3/connection.py", line 205, in connect
mordred_1        |     conn = self._new_conn()
mordred_1        |   File "/usr/local/lib/python3.8/site-packages/urllib3/connection.py", line 186, in _new_conn
mordred_1        |     raise NewConnectionError(
mordred_1        | urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f6bb73d9430>: Failed to establish a new connection: [Errno 111] Connection refused
mordred_1        | 
mordred_1        | During handling of the above exception, another exception occurred:
mordred_1        | 
mordred_1        | Traceback (most recent call last):
mordred_1        |   File "/usr/local/lib/python3.8/site-packages/requests/adapters.py", line 489, in send
mordred_1        |     resp = conn.urlopen(
mordred_1        |   File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 787, in urlopen
mordred_1        |     retries = retries.increment(
mordred_1        |   File "/usr/local/lib/python3.8/site-packages/urllib3/util/retry.py", line 592, in increment
mordred_1        |     raise MaxRetryError(_pool, url, error or ResponseError(cause))
mordred_1        | urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='mariadb', port=8000): Max retries exceeded with url: /api/ (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6bb73d9430>: Failed to establish a new connection: [Errno 111] Connection refused'))
mordred_1        | 
mordred_1        | During handling of the above exception, another exception occurred:
mordred_1        | 
mordred_1        | Traceback (most recent call last):
mordred_1        |   File "/usr/local/lib/python3.8/site-packages/sortinghat/cli/client/client.py", line 105, in connect
mordred_1        |     result = requests.get(self.url, headers={'Accept': 'text/html'})
mordred_1        |   File "/usr/local/lib/python3.8/site-packages/requests/api.py", line 73, in get
mordred_1        |     return request("get", url, params=params, **kwargs)
mordred_1        |   File "/usr/local/lib/python3.8/site-packages/requests/api.py", line 59, in request
mordred_1        |     return session.request(method=method, url=url, **kwargs)
mordred_1        |   File "/usr/local/lib/python3.8/site-packages/requests/sessions.py", line 587, in request
mordred_1        |     resp = self.send(prep, **send_kwargs)
mordred_1        |   File "/usr/local/lib/python3.8/site-packages/requests/sessions.py", line 701, in send
mordred_1        |     r = adapter.send(request, **kwargs)
mordred_1        |   File "/usr/local/lib/python3.8/site-packages/requests/adapters.py", line 565, in send
mordred_1        |     raise ConnectionError(e, request=request)
mordred_1        | requests.exceptions.ConnectionError: HTTPConnectionPool(host='mariadb', port=8000): Max retries exceeded with url: /api/ (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6bb73d9430>: Failed to establish a new connection: [Errno 111] Connection refused'))
mordred_1        | 
mordred_1        | During handling of the above exception, another exception occurred:
mordred_1        | 
mordred_1        | Traceback (most recent call last):
mordred_1        |   File "/usr/local/lib/python3.8/threading.py", line 932, in _bootstrap_inner
mordred_1        |     self.run()
mordred_1        |   File "/usr/local/lib/python3.8/site-packages/sirmordred/task_manager.py", line 83, in run
mordred_1        |     task = tc(self.config)
mordred_1        |   File "/usr/local/lib/python3.8/site-packages/sirmordred/task_panels.py", line 268, in __init__
mordred_1        |     super().__init__(conf)
mordred_1        |   File "/usr/local/lib/python3.8/site-packages/sirmordred/task.py", line 62, in __init__
mordred_1        |     self.client.connect()
mordred_1        |   File "/usr/local/lib/python3.8/site-packages/sortinghat/cli/client/client.py", line 108, in connect
mordred_1        |     if result.status_code != 400:
mordred_1        | UnboundLocalError: local variable 'result' referenced before assignment
mordred_1        | Exception in thread Global tasks:
mordred_1        | Traceback (most recent call last):
mordred_1        |   File "/usr/local/lib/python3.8/site-packages/urllib3/connection.py", line 174, in _new_conn
mordred_1        |     conn = connection.create_connection(
mordred_1        |   File "/usr/local/lib/python3.8/site-packages/urllib3/util/connection.py", line 95, in create_connection
mordred_1        |     raise err
mordred_1        |   File "/usr/local/lib/python3.8/site-packages/urllib3/util/connection.py", line 85, in create_connection
mordred_1        |     sock.connect(sa)
mordred_1        | ConnectionRefusedError: [Errno 111] Connection refused
mordred_1        | 
mordred_1        | During handling of the above exception, another exception occurred:
mordred_1        | 
mordred_1        | Traceback (most recent call last):
mordred_1        |   File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 703, in urlopen
mordred_1        |     httplib_response = self._make_request(
mordred_1        |   File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 398, in _make_request
mordred_1        |     conn.request(method, url, **httplib_request_kw)
mordred_1        |   File "/usr/local/lib/python3.8/site-packages/urllib3/connection.py", line 239, in request
mordred_1        |     super(HTTPConnection, self).request(method, url, body=body, headers=headers)
mordred_1        |   File "/usr/local/lib/python3.8/http/client.py", line 1256, in request
mordred_1        |     self._send_request(method, url, body, headers, encode_chunked)
mordred_1        |   File "/usr/local/lib/python3.8/http/client.py", line 1302, in _send_request
mordred_1        |     self.endheaders(body, encode_chunked=encode_chunked)
mordred_1        |   File "/usr/local/lib/python3.8/http/client.py", line 1251, in endheaders
mordred_1        |     self._send_output(message_body, encode_chunked=encode_chunked)
mordred_1        |   File "/usr/local/lib/python3.8/http/client.py", line 1011, in _send_output
mordred_1        |     self.send(msg)
mordred_1        |   File "/usr/local/lib/python3.8/http/client.py", line 951, in send
mordred_1        |     self.connect()
mordred_1        |   File "/usr/local/lib/python3.8/site-packages/urllib3/connection.py", line 205, in connect
mordred_1        |     conn = self._new_conn()
mordred_1        |   File "/usr/local/lib/python3.8/site-packages/urllib3/connection.py", line 186, in _new_conn
mordred_1        |     raise NewConnectionError(
mordred_1        | urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f6bb73f6400>: Failed to establish a new connection: [Errno 111] Connection refused
mordred_1        | 
mordred_1        | During handling of the above exception, another exception occurred:
mordred_1        | 
mordred_1        | Traceback (most recent call last):
mordred_1        |   File "/usr/local/lib/python3.8/site-packages/requests/adapters.py", line 489, in send
mordred_1        |     resp = conn.urlopen(
mordred_1        |   File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 787, in urlopen
mordred_1        |     retries = retries.increment(
mordred_1        |   File "/usr/local/lib/python3.8/site-packages/urllib3/util/retry.py", line 592, in increment
mordred_1        |     raise MaxRetryError(_pool, url, error or ResponseError(cause))
mordred_1        | urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='mariadb', port=8000): Max retries exceeded with url: /api/ (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6bb73f6400>: Failed to establish a new connection: [Errno 111] Connection refused'))
mordred_1        | 
mordred_1        | During handling of the above exception, another exception occurred:
mordred_1        | 
mordred_1        | Traceback (most recent call last):
mordred_1        |   File "/usr/local/lib/python3.8/site-packages/sortinghat/cli/client/client.py", line 105, in connect
mordred_1        |     result = requests.get(self.url, headers={'Accept': 'text/html'})
mordred_1        |   File "/usr/local/lib/python3.8/site-packages/requests/api.py", line 73, in get
mordred_1        |     return request("get", url, params=params, **kwargs)
mordred_1        |   File "/usr/local/lib/python3.8/site-packages/requests/api.py", line 59, in request
mordred_1        |     return session.request(method=method, url=url, **kwargs)
mordred_1        |   File "/usr/local/lib/python3.8/site-packages/requests/sessions.py", line 587, in request
mordred_1        |     resp = self.send(prep, **send_kwargs)
mordred_1        |   File "/usr/local/lib/python3.8/site-packages/requests/sessions.py", line 701, in send
mordred_1        |     r = adapter.send(request, **kwargs)
mordred_1        |   File "/usr/local/lib/python3.8/site-packages/requests/adapters.py", line 565, in send
mordred_1        |     raise ConnectionError(e, request=request)
mordred_1        | requests.exceptions.ConnectionError: HTTPConnectionPool(host='mariadb', port=8000): Max retries exceeded with url: /api/ (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6bb73f6400>: Failed to establish a new connection: [Errno 111] Connection refused'))
mordred_1        | 
mordred_1        | During handling of the above exception, another exception occurred:
mordred_1        | 
mordred_1        | Traceback (most recent call last):
mordred_1        |   File "/usr/local/lib/python3.8/threading.py", line 932, in _bootstrap_inner
mordred_1        |     self.run()
mordred_1        |   File "/usr/local/lib/python3.8/site-packages/sirmordred/task_manager.py", line 83, in run
mordred_1        |     task = tc(self.config)
mordred_1        |   File "/usr/local/lib/python3.8/site-packages/sirmordred/task.py", line 62, in __init__
mordred_1        |     self.client.connect()
mordred_1        |   File "/usr/local/lib/python3.8/site-packages/sortinghat/cli/client/client.py", line 108, in connect
mordred_1        |     if result.status_code != 400:
mordred_1        | UnboundLocalError: local variable 'result' referenced before assignment
^CGracefully stopping... (press Ctrl+C again to force)
Stopping docker-compose_hatstall_1      ... done
Stopping docker-compose_mordred_1       ... done
Stopping docker-compose_kibiter_1       ... done
Stopping docker-compose_mariadb_1       ... done
Stopping docker-compose_elasticsearch_1 ... done

[zhquan]

Hi @roddds

The latest stable version is grimoirelab/grimoirelab:0.7.1. Could you try this version instead?

I guess you are using this docker-compose.yml

[Rafaeltheraven]

I've been running into this exact same issue. I think it's because the example setup.cfg in this repository has the sortinghat.host key set to mariadb, suggesting that these are the mysql credentials. However, sirmordred is actually using this value to attempt to connect to the sortinghat server, which AFAIK is being hosted by the hatstall image, so the host should be set to hatstall. Having done this, Mordred no longer crashes, but I think there needs to be a bit more tweaking because I don't think the identities phase properly runs.

[Rafaeltheraven]

Right, I think I figured it out.

The issue is that sortinghat has moved over to a service-based architecture, but this repository does not properly reflect that yet.

My relevant modifications to the docker-compose are as follows:

services:
    mariadb:
      image: mariadb:10.6
      expose:
        - "3306"
      environment:
        - MYSQL_ROOT_PASSWORD=
        - MYSQL_ALLOW_EMPTY_PASSWORD=yes
      healthcheck:
        test: ["CMD", "/usr/local/bin/healthcheck.sh", "--su=root", "--connect", "--innodb_initialized"]

    redisdb:
      image: redis:latest
      ports:
        - 6379:6379
      healthcheck:
        test: [ "CMD", "redis-cli", "--raw", "incr", "ping" ]
    
    sortinghat:
      image: grimoirelab/sortinghat
      environment:
        - SORTINGHAT_SECRET_KEY=secret
        - SORTINGHAT_DB_HOST=mariadb
        - SORTINGHAT_REDIS_HOST=redisdb
        - SORTINGHAT_SUPERUSER_USERNAME=root
        - SORTINGHAT_SUPERUSER_PASSWORD=root
        - SORTINGHAT_DB_DATABASE=sortinghat_db
        - SORTINGHAT_HTTP_DEV=0.0.0.0:8000
        - SORTINGHAT_ALLOWED_HOST=sortinghat,.localhost,127.0.0.1,[::1]
        - DEV_FLAG=1
      ports:
        - 8000:8000
        - 9314:9314
      depends_on:
        mariadb:
          condition: service_healthy
        redisdb:
          condition: service_healthy

    sortinghat_worker:
      image: grimoirelab/sortinghat-worker
      environment:
        - SORTINGHAT_SECRET_KEY=secret
        - SORTINGHAT_DB_HOST=mariadb
        - SORTINGHAT_REDIS_HOST=redisdb
      depends_on:
        - sortinghat

Then, also modify the setup.cfg so the sortinghat config looks like this:

[sortinghat]
host = sortinghat
user = root
password = root
port = 8000
path = api/
ssl = false
database = sortinghat_db
load_orgs = true
orgs_file = /home/bitergia/organizations.json
autoprofile = [github, pipermail, git]
matching = [email,username]
sleep_for = 100
unaffiliated_group = Unknown
affiliate = true
strict_mapping = false
reset_on_load = false
identities_file = [/home/bitergia/conf/identities.yml]
identities_format = grimoirelab

Ideally, there would be an nginx/apache/whatever container acting as a proxy for sortinghat:9314 but to keep it simple, I'm using the dev http server here, which runs on 8000.

[zhquan]

Hi @Rafaeltheraven

The latest stable version (grimoirelab/grimoirelab:0.7.1) does not need sortinghat and sortinghat_worker containers, only need MariaDB.

We are preparing significant changes for the next release (0.8.0), including the new SortingHat (sortinghat and sortinghat-worker). We will add sortinghat sections on the docker-compose/docker-compose.yml and update the default-grimoirelab-settings/setup.cfg when the 0.8.0 release is ready.

I guess you are using grimoirelab/grimoirelab:latest and it needs sortinghat and sortinghat_worker containers but the latest version is not stable and it still has some bugs.

Could you try this docker-compose but using grimoirelab/grimoirelab:0.7.1 instead of grimoirelab/grimoirelab:latest?

[yangzhangs]

I also encountered this problem. I changed the tag and everything did appear to start up normal. Great!

Hi @roddds

The latest stable version is grimoirelab/grimoirelab:0.7.1. Could you try this version instead?

I guess you are using this docker-compose.yml

[Rafaeltheraven]

grimoirelab:0.7.1 works, though the compose file in this repository still uses latest.

As for me, I'm using the latest version of Mordred for my own reasons, so I kinda need latest in a sense. Anyhow, if anybody else wants to use the latest version, they can tentatively use the docker-compose I described.

[mykaul]

Generally, I'm not sure using 'latest' is a good idea - it doesn't give a consistent experience for users.

Waiting for 0.8.0 with a fixed yml..