k8s-infra-otel-agent - Failed to process entry - regex parser - k3s

[IzioDev]

Hi,

I've just installed Signoz on my k3s cluster, everything's fine but the 'automatic k8s metrics collection' seems to not work, by default.

2024-06-02T13:06:46.725Z	error	helper/transformer.go:98	Failed to process entry	{"kind": "receiver", "name": "filelog/k8s", "data_type": "logs", "operator_id": "parser-crio", "operator_type": "regex_parser", "error": {"description": "time parser: parsing time \"2024-06-02T15:06:46.70246648+02:00\" as \"2006-01-02T15:04:05.000000000-07:00\": cannot parse \".70246648+02:00\" as \".000000000\""}, "action": "send", "entry": {"observed_timestamp":"2024-06-02T13:06:46.725811448Z","timestamp":"0001-01-01T00:00:00Z","body":"2024-06-02T15:06:46.70246648+02:00 stdout F 10.42.0.1 - - [02/Jun/2024:13:06:46 +0000] \"GET / HTTP/1.1\" 200 10880 \"-\" \"kube-probe/1.28\" \"-\"","attributes":{"log":"10.42.0.1 - - [02/Jun/2024:13:06:46 +0000] \"GET / HTTP/1.1\" 200 10880 \"-\" \"kube-probe/1.28\" \"-\"","log.file.path":"/var/log/pods/signoz_signoz-frontend-86c6cd7958-pqnxb_64fd1dea-c7ec-44d1-a8f6-525551daa6be/signoz-frontend/0.log","logtag":"F","stream":"stdout","time":"2024-06-02T15:06:46.70246648+02:00"},"severity":0,"scope_name":""}}
github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza/operator/helper.(*TransformerOperator).HandleEntryError
	github.com/open-telemetry/opentelemetry-collector-contrib/pkg/[email protected]/operator/helper/transformer.go:98
github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza/operator/helper.(*ParserOperator).ParseWith
	github.com/open-telemetry/opentelemetry-collector-contrib/pkg/[email protected]/operator/helper/parser.go:175
github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza/operator/helper.(*ParserOperator).ProcessWithCallback
	github.com/open-telemetry/opentelemetry-collector-contrib/pkg/[email protected]/operator/helper/parser.go:112
github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza/operator/helper.(*ParserOperator).ProcessWith
	github.com/open-telemetry/opentelemetry-collector-contrib/pkg/[email protected]/operator/helper/parser.go:98
github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza/operator/parser/regex.(*Parser).Process
	github.com/open-telemetry/opentelemetry-collector-contrib/pkg/[email protected]/operator/parser/regex/regex.go:99
github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza/operator/transformer/router.(*Transformer).Process
	github.com/open-telemetry/opentelemetry-collector-contrib/pkg/[email protected]/operator/transformer/router/router.go:130
github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza/operator/helper.(*WriterOperator).Write
	github.com/open-telemetry/opentelemetry-collector-contrib/pkg/[email protected]/operator/helper/writer.go:53
github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza/operator/input/file.(*Input).emit
	github.com/open-telemetry/opentelemetry-collector-contrib/pkg/[email protected]/operator/input/file/file.go:52
github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza/fileconsumer/internal/reader.(*Reader).ReadToEnd
	github.com/open-telemetry/opentelemetry-collector-contrib/pkg/[email protected]/fileconsumer/internal/reader/reader.go:106
github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza/fileconsumer.(*Manager).consume.func1
	github.com/open-telemetry/opentelemetry-collector-contrib/pkg/[email protected]/fileconsumer/file.go:156

otel image: docker.io/otel/opentelemetry-collector-contrib:0.88.0
signoz chart version: latest

[nityanandagohain]

The error seems to be for log, is your metrics issue resolved ?

[grandwizard28]

Hi @IzioDev,
The latest release of k8s-infra has the new container-parser (https://opentelemetry.io/blog/2024/otel-collector-container-log-parser/) operator for logs. We have also bumped up the collector to 0.109.0.

Can you test with the latest release please?