replace logstash with fluentd

[mmguero]

@mmguero cloned issue idaholab/Malcolm#103 on 2022-06-08:

Now that we're not using Elastic any more, it may make sense to replace Logstash with fluentd. See #102 for the client side of things. There is a lot of logic surrounding the logstash pipelines, though. We may decide to keep logstash (for now at least) as there is an official OpenSearch output plugin for it, even if we decide to move away from beats.

[mmguero]

@mmguero commented on 2023-06-15:

If/when we decide to do this, actually fluent bit with [lua](https://docs.fluentbit.io/manual/pipeline/filters/lua] scripting will be the way we go, i think.

[mmguero]

@mmguero commented on 2023-06-15:

https://docs.fluentbit.io/manual/about/fluentd-and-fluent-bit

[mmguero]

A customer suggested Kafka as an alternative to fluent, so that might be something to look at.

[mavam]

There's also Tenzir, which comes with a BSD 3-clause core.

Disclaimer: I run Tenzir, but genuinely feel that it could be a good fit here as a security-native ETL tool.