Simplest way of sending access.log to remote SigNoz instance?

[rwjack]

Hi,

I've read the entire https://signoz.io/docs/ yesterday, but now I have too much information 😅

Coming from datadog, I didn't go into the OTEL docs as much, but I find it uneasy to configure an agent/forwarder to send data to SigNoz.

Basically I'm trying to comprehend what would be the simplest way to send access.log lines to a remtoe SigNoz instance?
From here I see that exporting nfs shares from the webserver to the signoz host is an option, but that seems kind of overkill.

As far as I could tell, OTEL collectors are the equivalent to DD private locations, so what would be the equivalent of a DD agent?

TLDR; Is there a stupid simple way of sending logs over TCP with TLS?

I think it would be great if SigNoz had some sort of agent that simplifies all of the above into a single package/container.

[rwjack]

To recap, I installed the otelcollector deb package (otelcol_0.72.0_linux_amd64.deb) with the following config.yaml:

receivers:
  filelog:
    include: [ /var/log/traefik/access.log ]
    start_at: beginning
    operators:
      - type: json_parser
        timestamp:
          parse_from: attributes.time
          layout: '%Y-%m-%d,%H:%M:%S %z'
      - type: move
        from: attributes.message
        to: body
      - type: remove
        field: attributes.time
processors:
  batch:
    send_batch_size: 10000
    send_batch_max_size: 11000
    timeout: 10s
exporters:
  otlp/log:
    endpoint: http://SIGNOZ_IP:4138
    tls:
      insecure: true
service:
  pipelines:
    logs:
      receivers: [ filelog ]
      processors: [ batch ]
      exporters: [ otlp/log ]

But I keep getting this error:

* error decoding 'receivers': unknown type: "filelog" for id: "filelog" (valid values: [jaeger kafka opencensus prometheus zipkin otlp hostmetrics])

Managed to get this sorted by installing otelcol-contrib, which contains the filelog receiver.

[nityanandagohain]

Yeah, the otelcol-contrib contains all the third-party receivers and processors.

[nityanandagohain]

You can use two otel collectors to send data from application instance to Signoz instance.

One otel collector deployed on the application instance will collect the logs and transfer it over OTLP to the otel collector deployed on the SigNoz instance.

Here is an example https://signoz.io/docs/userguide/collect_logs_from_file/#steps-for-collecting-logs-if-signoz-is-running-on-a-different-host

[rwjack]

Yep, managed to get it working, thanks!