Traces & logs in S3 Cold Storage

[Flooze-IT]

Hi everyone,

I actually use Signoz in my company to exploit logs and traces.

Recently we have setup a cold storage with S3 buckets.

The question is: Traces and logs moved in S3 are always available for requesting from the UI ?

And another question: It's possible to backup the data (in S3) in another signoz instance in another Kubernetes cluster ?

Many thanks

[nityanandagohain]

Traces and logs moved in S3 are always available for requesting from the UI ?

Yes you can -> though it will be slower.

It's possible to backup the data (in S3) in another signoz instance in another Kubernetes cluster ?

@prashant-shahi can help with this

[Flooze-IT]

@prashant-shahi if you have any ideas, i will take it :)

[prashant-shahi]

It's possible to backup the data (in S3) in another signoz instance in another Kubernetes cluster ?

S3 support of SigNoz is not exactly for backup per se, but for cold storage.
It is enabled using tiered storage policy of ClickHouse to store new data in ClickHouse volume and moving older data to S3.

Hence, I don't think it would be possible to use the data in S3 directly in new signoz instance.
You would have to copy clickhouse data as well since to my understanding, the default disk contains metadata of S3 objects as well.

/cc @ankitnayan @srikanthccv

[Flooze-IT]

News :)

I backup data on S3 and i tried to connect a new instance of Signoz in my S3 where i stored the backup but the data is not retrieved.
I don't see any data from my previous signoz instance.

Any ideas ? @prashant-shahi @ankitnayan @srikanthccv

[srikanthccv]

I don't see any data from my previous signoz instance.

How are you testing this? Did you try querying for older data?

[Flooze-IT]

Metadata are not backup. So when i change of instance data is not found.

I succeeded with clickhouse-backup tool and an external clickhouse instance.

I think you can evolve backup in signoz with this tool. It is very useful.