Hello,

Currently we use the netskope log shipper to get logs from netskope, but it seems that this will be deprecated and the new way to get logs is using the Log Streaming.

The integration supports getting Alerts and Events using the Log Streaming mode, but it seems that it cannot get both at the same time.

On Netskope side, when using Log Streaming the Events and Alerts are streamed together. [docs]

When admins select Alerts and Events, they are streamed together.

So, you would have logs from alerts and events on the same bucket, which would end up on the same SQS queue.

On the integration side, you have different configurations for Alerts and Events, the ingest pipeline for Alerts will drop Events logs and the ingest pipeline for Events will drop Alerts logs.

You cannot use the same SQS queue in both configurations as this would create a competition between the inputs and you would lose events and alerts, and using SNS to send notifications to 2 different SQS queues is an unnecessary extra work and would lead to the data being processed twice.

And you also cannot have multiple streams of the same data type on Netskope side, as mentioned in netskope documentation.

You cannot stream a data collection multiple times

The integration needs to have a routing ingest pipeline that would direct the Alerts to the Alerts pipeline and the Events to the Events pipeline.

The way it currently works the user needs to choose to collect Alerts or to collect Events.

On AWS is possible to use a SNS to fan out the notifications to different SQS queues, but on S3 polling mode and on GCS/Azure you would need to poll the logs twice, which leads to increase CPU usage and egress costs.