Request: Using the fields: key to define the values() from a |stats command in correlation searches

[joshnck]

Currently the correlations search can only reveal the data that is included in a detection if it is part of the explicit logic of the detection or if it is part of the group-by functionality. This is a limitation of how |stats works in Splunk and if we want to include extra context for the analyst from our detection, we need to use values() or some comparable function to bring data from the log into the table.

Currently we use fields: to build a |table with detections which allows us to tell Splunk which fields are important for an analyst to investigate - this should translate further into the |stats command.

title: Example Detection
name: base_rule
date: 2024/03/26
status: experimental
author: burnsn1
description: Test Rule
logsource:
    category: process_creation
    product: windows
detection:
    susp_exec:
        process_path:
        -   'C:\Windows'
    condition: susp_exec
fields:
-  process_path
-  process_name
---
title: Multiple occurrences of base event
correlation:
    type: event_count
    rules:
        - base_rule
    group-by:
        - process_path
    timespan: 24h
    condition:
        gte: 10

which then should ideally convert to:

process_path="C:\\Windows" | table process_path,process_name

| bin _time span=24h
| stats count as event_count values(process_name) as process_name by _time process_path

| search event_count >= 10

This is further useful because the converted element then retains full information that is useful for the analysis. Otherwise, you're dropping fields that may be necessary for context.

We will need to compare the fields: values to the group-by: values to make sure the searches are valid and only listed once in the final query.