elasticsearch index_not_found_exception

[hschwicht]

Hi all, we run into problems running the docker compose setup (downloaded today). We prepared the setup as described. Maybe we have to do anything else, every help is welcome.

starting gen3 services using docker-compose up the guppy-service and nginx fails directly. It seems to be that nginx is failing due to the guppy and guppy failing. Restarting guppyservice seems to be caused by elasticsearch:

[16:02:04] INFO: [ES] connected to elasticsearch at http://esproxy-service:9200.
(node:6) UnhandledPromiseRejectionWarning: Error: [ES.initialize] error getting mapping from ES index "etl": index_not_found_exception
    at client.indices.getMapping.then.err (/guppy/dist/server/es/index.js:170:13)
    at process._tickCallback (internal/process/next_tick.js:68:7)
(node:6) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
(node:6) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

regards, Horst

[vzpgb]

Hi @hschwicht ! Have you run bash ./guppy_setup.sh to create/re-create ES indices?
This section describes guppy configuration https://github.com/uc-cdis/compose-services#configuring-guppy-for-exploration-page

[vzpgb]

@hschwicht , I wonder if you are using Docker Desktop? Our developers mentioned case when Compose Services failed to start because of Docker Desktop settings: it was set to make 4 CPUs available to the Docker Engine, and start working after limiting it to 1 CPU in Docker > Preferences > Advanced. Could you confirm your settings?

[rolinge]

Hello @vzpgb I am using docker-compose on a fresh Ubuntu 18.04 image in Azure, but the guppy-setup script fails. I get the following error...

'NoneType' object has no attribute '__getitem__'
ERROR when running transformation
Traceback (most recent call last):
  File "run_etl.py", line 32, in run_transform
    interpreter.run_transform(translators)
  File "/tube/tube/etl/indexers/interpreter.py", line 48, in run_transform
    t.write(df)
  File "/tube/tube/etl/indexers/base/translator.py", line 53, in write
    self.writer.create_guppy_array_config(self.parser.name, self.parser.types)
  File "/tube/tube/etl/outputs/es/writer.py", line 102, in create_guppy_array_config
    latest_transaction_time = get_latest_utc_transaction_time()
  File "/tube/tube/etl/outputs/es/timestamp.py", line 29, in get_latest_utc_transaction_time
    return to_utc_time(execute_sql_query_return_first_item(query)[2])
TypeError: 'NoneType' object has no attribute '__getitem__'

The other services start, except revproxy-service which fails due to relying on guppy. I comment that stanza out and I can start the proxy and get the application to run, but I'm betting guppy is needed. Any ideas?

osadmin@vm01:/compose-services$ docker version
Client: Docker Engine - Community
 Version:           19.03.8
 API version:       1.40
 Go version:        go1.12.17
 Git commit:        afacb8b7f0
 Built:             Wed Mar 11 01:25:46 2020
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.8
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.12.17
  Git commit:       afacb8b7f0
  Built:            Wed Mar 11 01:24:19 2020
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.2.13
  GitCommit:        7ad184331fa3e55e52b890ea95e65ba581ae3429
 runc:
  Version:          1.0.0-rc10
  GitCommit:        dc9208a3303feef5b3839f4323d9beb36df0a9dd
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

osadmin@vm01:/compose-services$ docker-compose version
docker-compose version 1.25.0, build 0a186604
docker-py version: 4.1.0
CPython version: 3.7.4
OpenSSL version: OpenSSL 1.1.0l  10 Sep 2019

[vzpgb]

Hi @rolinge !

You did absolutely right by commenting out guppy! If you are running Compose Services for the first time or have no data in the database, you may want to comment out following lines in the nginx.conf:

#        location /guppy/ {
#                proxy_pass http://guppy-service/;
#        }

The reason is that you don't have data in your commons yet so guppy can not be configured. Once you put data, you can configure guppy as described here