📈 GraphQL endpoint is not picking up data

[KagemniKarimu]

Self Checks

• This is only for bug report, if you would like to ask a question, please head to Discussions.
• I have searched for existing issues search for existing issues, including closed ones.
• I have searched for existing pull requests search for existing pull requests, including closed ones.

Environment

📜 Manuscript Core:
Version: 1.1.0

🏗️ Build Information:
Go Version: go1.22.5
Rust Version: 1.83.0
Git Commit: 003f13b
Built: 2024-12-13:21:19:06

💻 System Information:
Unix Name: Linux fedora-workstation 6.11.11-300.fc41.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Dec 5 18:38:25 UTC 2024 x86_64 GNU/Linux
OS/Arch: linux/amd64

🐳 Docker Environment:
Docker: '27.4.0'
Docker Desktop: '4.36.0'

Steps to reproduce

1. manuscript-cli init
2. select directory
3. select name
4. select database
5. select table set
6. confirm selection
7. visit Hasura endpoint

✔️ Expected Behavior

GraphQL should populate with the data set and be visible on Hasura.

❌ Actual Behavior

In f-postgres-1 log:

2024-12-13 15:21:56 2024-12-13 21:21:56.470 UTC [1] LOG:  starting PostgreSQL 16.4 (Debian 16.4-1.pgdg120+2) on x86_64-pc-linux-gnu, compiled by gcc (Debian 12.2.0-14) 12.2.0, 64-bit
2024-12-13 15:21:56 2024-12-13 21:21:56.471 UTC [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
2024-12-13 15:21:56 2024-12-13 21:21:56.471 UTC [1] LOG:  listening on IPv6 address "::", port 5432
2024-12-13 15:21:56 2024-12-13 21:21:56.494 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2024-12-13 15:21:56 2024-12-13 21:21:56.518 UTC [65] LOG:  database system was shut down at 2024-12-13 21:21:56 UTC
2024-12-13 15:21:56 2024-12-13 21:21:56.537 UTC [1] LOG:  database system is ready to accept connections
2024-12-13 15:22:00 2024-12-13 21:22:00.593 UTC [69] FATAL:  database "ethereum" does not exist
2024-12-13 15:22:00 2024-12-13 21:22:00.715 UTC [70] FATAL:  database "ethereum" does not exist
2024-12-13 15:22:00 2024-12-13 21:22:00.740 UTC [71] FATAL:  database "ethereum" does not exist
2024-12-13 15:22:00 2024-12-13 21:22:00.865 UTC [72] FATAL:  database "ethereum" does not exist
2024-12-13 15:22:14 2024-12-13 21:22:14.810 UTC [76] ERROR:  relation "hdb_catalog.hdb_version" does not exist at character 69
2024-12-13 15:22:14 2024-12-13 21:22:14.810 UTC [76] STATEMENT:  

GraphQL /healthz endpoint reports status OK

[skrfffe1]

Hello_Sir this is ctrlaltdel7963
Steps to produce:

1. manuscript-cli init
2. assign project directory, project name
3. Postgres
4. Ethereum network, transaction table.
5. inside the project directory, manuscript-cli deploy manuscript.yaml --env=local

I only use the generated yaml templates which is the manuscript.yaml and docker-compose.yaml to test the deployment.

[Liquidwe]

@skrfffe1 @KagemniKarimu thank you for the feedback, I will confirm and fix this issue

[KagemniKarimu]

@Liquidwe I just pulled your changes.
Unfortunately, this did not fix my issue.
It did not cause the graphQL endpoint to populate.

( ⭐ #105 does seem to make state detection a little less precise 😢 , as in it's now reporting running/ unknown/stopped instead of accurate states like failed. We may consider restoring old state detection logic. )

My job manager container and hasura containers still seem to be erroring out. What I think are the relevant lines

from jobmanager-1 logs:

2024-12-17 06:23:35 2024-12-17 12:23:35,671 INFO  com.chainbase.manuscript.ETLProcessor                        [] - Schema created or already exists: public
2024-12-17 06:23:35 2024-12-17 12:23:35,684 INFO  com.chainbase.manuscript.ETLProcessor                        [] - Creating table: CREATE TABLE IF NOT EXISTS public.contracts (block_timestamp TIMESTAMP(3) WITHOUT TIME ZONE, block_number BIGINT, block_hash VARCHAR(66), transaction_hash VARCHAR(66), transaction_index INTEGER, from_address VARCHAR(42), address VARCHAR(42), bytecode BYTEA, function_sighashes ARRAY<VARCHAR(10)>, is_erc20 BOOLEAN, is_erc721 BOOLEAN, __pk INTEGER, PRIMARY KEY (block_number))
2024-12-17 06:23:35 2024-12-17 12:23:35,691 ERROR com.chainbase.manuscript.ETLProcessor                        [] - Error creating database or table for PostgreSQL sink: ERROR: syntax error at or near "ARRAY"
2024-12-17 06:23:35   Position: 278

from combined cluster logs:

2024-12-17 06:23:20 postgres-1  | 2024-12-17 12:23:20.502 UTC [65] LOG:  database system was shut down at 2024-12-17 12:23:20 UTC
2024-12-17 06:23:20 postgres-1  | 2024-12-17 12:23:20.504 UTC [66] FATAL:  the database system is starting up
2024-12-17 06:23:20 postgres-1  | 2024-12-17 12:23:20.523 UTC [1] LOG:  database system is ready to accept connections
2024-12-17 06:23:20 postgres-1  | 2024-12-17 12:23:20.639 UTC [70] FATAL:  database "ethereum" does not exist
2024-12-17 06:23:20 postgres-1  | 2024-12-17 12:23:20.661 UTC [71] FATAL:  database "ethereum" does not exist
2024-12-17 06:23:20 postgres-1  | 2024-12-17 12:23:20.786 UTC [72] FATAL:  database "ethereum" does not exist
2024-12-17 06:23:28 postgres-1  | 2024-12-17 12:23:28.317 UTC [73] FATAL:  database "ethereum" does not exist
2024-12-17 06:23:28 postgres-1  | 2024-12-17 12:23:28.444 UTC [74] FATAL:  database "ethereum" does not exist
2024-12-17 06:23:28 postgres-1  | 2024-12-17 12:23:28.470 UTC [75] FATAL:  database "ethereum" does not exist
2024-12-17 06:23:28 postgres-1  | 2024-12-17 12:23:28.597 UTC [76] FATAL:  database "ethereum" does not exist
2024-12-17 06:23:35 postgres-1  | 2024-12-17 12:23:35.685 UTC [78] ERROR:  syntax error at or near "ARRAY" at character 278
2024-12-17 06:23:35 postgres-1  | 2024-12-17 12:23:35.685 UTC [78] STATEMENT:  CREATE TABLE IF NOT EXISTS public.contracts (block_timestamp TIMESTAMP(3) WITHOUT TIME ZONE, block_number BIGINT, block_hash VARCHAR(66), transaction_hash VARCHAR(66), transaction_index INTEGER, from_address VARCHAR(42), address VARCHAR(42), bytecode BYTEA, function_sighashes ARRAY<VARCHAR(10)>, is_erc20 BOOLEAN, is_erc721 BOOLEAN, __pk INTEGER, PRIMARY KEY (block_number))
2024-12-17 06:23:42 postgres-1  | 2024-12-17 12:23:42.395 UTC [79] ERROR:  relation "hdb_catalog.hdb_version" does not exist at character 69
2024-12-17 06:23:42 postgres-1  | 2024-12-17 12:23:42.395 UTC [79] STATEMENT:  
2024-12-17 06:23:42 postgres-1  |               SELECT ee_client_id::text, ee_client_secret
2024-12-17 06:23:42 postgres-1  |                 FROM hdb_catalog.hdb_version
2024-12-17 06:23:42 postgres-1  |             
2024-12-17 06:25:28 postgres-1  | 2024-12-17 12:25:28.505 UTC [88] ERROR:  syntax error at or near "ARRAY" at character 278
2024-12-17 06:25:28 postgres-1  | 2024-12-17 12:25:28.505 UTC [88] STATEMENT:  CREATE TABLE IF NOT EXISTS public.contracts (block_timestamp TIMESTAMP(3) WITHOUT TIME ZONE, block_number BIGINT, block_hash VARCHAR(66), transaction_hash VARCHAR(66), transaction_index INTEGER, from_address VARCHAR(42), address VARCHAR(42), bytecode BYTEA, function_sighashes ARRAY<VARCHAR(10)>, is_erc20 BOOLEAN, is_erc721 BOOLEAN, __pk INTEGER, PRIMARY KEY (block_number))

See complete logs here:
🗒️ https://gist.github.com/KagemniKarimu/ae3f8ca8d64c06f09766fc77dd8b7bf7

[Liquidwe]

thx @KagemniKarimu This error is very detailed, I think I should know what the problem is now

[skrfffe1]

it seems some dependencies are not behaved what they supposed to be, I keep on watching them and they gradually turning off out of service, I am starting to hypothesize that there is something to do with the service provider regardless of its configurations.

[Liquidwe]

@skrfffe1 Can you provide the chain and table you have chosen? I will follow up on this issue

[skrfffe1]

ethereum transaction table

…

On Thu, Dec 19, 2024, 11:40 AM Liquid ***@***.***> wrote: @skrfffe1 <https://github.com/skrfffe1> Can you provide the chain and table you have chosen? I will follow up on this issue — Reply to this email directly, view it on GitHub <#97 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AOHIN7UNS4XA5Z2OUGYELUD2GI52JAVCNFSM6AAAAABTS3UGSOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKNJSG4YDQMJXGA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[KagemniKarimu]

Issue is ongoing. I don't seem to be able to produce data reliably when launching manuscripts. :(
On our last community code, the jobmanager ran the job of getting data for 15 minutes while the logs cycled multiple checkpoints. Task manager and hasura showed activity but no completion.