Skip to content

[cisco_ftd] Add support for Security Group Tag and Endpoint Group fields #15652

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 6 commits into
base: main
Choose a base branch
from
Open

[cisco_ftd] Add support for Security Group Tag and Endpoint Group fields #15652

wants to merge 6 commits into from
+338 −5

Conversation

@yahyaghani
Copy link
Contributor

Summary

This PR adds support for parsing 6 Security Group Tag (SGT) and Endpoint Group (EPG) fields from Cisco FTD connection event syslog messages (message IDs 430002 and 430003).

Related Issue

Fixes #15204

Problem

The Cisco FTD integration was not parsing SGT/EGT-related fields from connection event messages. These fields were present in the event.original field but were not being extracted into structured, queryable fields, making it difficult to search and analyze security group information in Elastic.

Solution

Added parsing support for the following 6 fields:

Field Target Field Type Description
SourceSecurityGroup cisco.ftd.security_event.source_security_group keyword Security Group of the source
SourceSecurityGroupTag cisco.ftd.security_event.source_security_group_tag keyword Numeric SGT attribute of source
SourceSecurityGroupType cisco.ftd.security_event.source_security_group_type keyword Source SGT type (Inline, Session Directory, SXP)
DestinationIP_DynamicAttribute cisco.ftd.security_event.destination_ip_dynamic_attribute keyword Destination IP dynamic attribute (EPG info)
DestinationSecurityGroup cisco.ftd.security_event.destination_security_group keyword Security Group of the destination
DestinationSecurityGroupTag cisco.ftd.security_event.destination_security_group_tag keyword Numeric SGT attribute of destination

Changes Made

1. Ingest Pipeline (default.yml)

  • Added 6 field mappings in the script processor params section
  • Added field targets to security_event_list array to ensure fields are placed in cisco.ftd.security_event group (consistent with other connection event fields)
  • Fields are configured for message IDs ["430002", "430003"]

2. Field Definitions (fields.yml)

  • Added 6 field definitions under cisco.ftd.security_event group
  • All fields typed as keyword to support both string and numeric values
  • Added descriptions based on official Cisco documentation

3. Testing

  • Created new test file test-sgt.log with 2 sample connection events containing SGT/EGT fields
  • Test covers both 430002 (connection start) and 430003 (connection end) message types
  • Validates extraction of both string values (e.g., "SGT_TEST_GROUP") and numeric values (e.g., "2005")
  • All 39 pipeline tests passing ✅

Implementation Notes

Fields are placed in cisco.ftd.security_event rather than the legacy cisco.ftd.security field for consistency and maintainability.
All new fields use keyword type to handle both string and numeric values.

Testing Performed

  • Pipeline tests pass (39/39)
  • Fields extract correctly
  • Correct ECS placement
  • No regressions

References

Checklist

  • Field definitions added
  • Pipeline updated
  • Tests added
  • Docs/links included
  • All tests passing

@yahyaghani yahyaghani requested a review from a team as a code owner October 15, 2025 02:31
yahyaghani added a commit to yahyaghani/integrations that referenced this pull request Oct 15, 2025
@yahyaghani
docs: update changelog with PR link elastic#15652
e37b946
@elastic-vault-github-plugin-prod
Copy link

🚀 Benchmarks report

Package cisco_ftd 👍(0) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
log 892.06 524.93 -367.13 (-41.16%) 💔

To see the full report comment with /test benchmark fullreport

P1llus
P1llus reviewed Oct 15, 2025
View reviewed changes
bhapas
bhapas reviewed Oct 15, 2025
View reviewed changes
packages/cisco_ftd/data_stream/log/_dev/test/pipeline/test-sgt.log Outdated
@@ -0,0 +1,3 @@
2025-09-01T12:00:00Z firepower : %FTD-6-430003: EventPriority: Low, DeviceUUID: d697c8ca-9fe4-43e6-aeb5-33e277e5ffea, InstanceID: 11, FirstPacketSecond: 2025-09-01T12:35:00Z, ConnectionID: 39416, AccessControlRuleAction: Trust, SrcIP: 10.0.100.30, DstIP: 10.0.1.20, SrcPort: 56799, DstPort: 53, Protocol: udp, IngressInterface: inside, EgressInterface: outside, SourceSecurityGroup: SGT_TEST_GROUP, SourceSecurityGroupTag: 2003, SourceSecurityGroupType: Session Directory, DestinationIP_DynamicAttribute: APIC_EPG_TEST_GROUP, IngressVRF: Global, EgressVRF: Global, Endpoint Profile: Workstation:Microsoft-Workstation:Windows11-Workstation, ACPolicy: ACP-Access, AccessControlRuleName: Test-Rule-1, Prefilter Policy: Default Prefilter Policy, User: testuser, Client: DNS, ApplicationProtocol: DNS, ConnectionDuration: 0, InitiatorPackets: 1, ResponderPackets: 1, InitiatorBytes: 31, ResponderBytes: 238, NAPPolicy: Balanced Security and Connectivity
2025-09-01T14:00:00Z firepower : %FTD-6-430002: EventPriority: Low, DeviceUUID: d697c8ca-9fe4-43e6-aeb5-33e277e5ffea, InstanceID: 4, FirstPacketSecond: 2025-09-01T14:00:03Z, ConnectionID: 36584, AccessControlRuleAction: Block, SrcIP: 10.0.100.30, DstIP: 10.0.1.20, SrcPort: 56799, DstPort: 22, Protocol: tcp, IngressInterface: inside, EgressInterface: outside, SourceSecurityGroup: 2005, SourceSecurityGroupTag: 2005, DestinationSecurityGroup: 9, DestinationSecurityGroupTag: 9, SourceSecurityGroupType: Session Directory, DestinationSecurityGroupType: SXP, IngressVRF: Global, EgressVRF: Global, Endpoint Profile: Invalid ID, ACPolicy: ACP-Management, AccessControlRuleName: Default Deny, Prefilter Policy: Management Prefilter Policy, InitiatorPackets: 1, ResponderPackets: 0, InitiatorBytes: 70, ResponderBytes: 0, NAPPolicy: Balanced Security and Connectivity

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you can remove this empty log line

packages/cisco_ftd/changelog.yml Outdated
@@ -1,4 +1,9 @@
# newer versions go on top
- version: "3.10.3"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this be 3.11.0 since we are adding support for new types in this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

implemented

@andrewkroh andrewkroh added documentation Improvements or additions to documentation. Applied to PRs that modify *.md files. Integration:cisco_ftd Cisco FTD Team:Integration-Experience Security Integrations Integration Experience [elastic/integration-experience] labels Oct 15, 2025
@elasticmachine
Copy link

Pinging @elastic/integration-experience (Team:Integration-Experience)

@yahyaghani
docs: address PR review feedback for cisco_ftd SGT/EPG fields
3833710 
- Bump version to 3.11.0 (enhancement requires minor version bump)
- Update changelog link to PR elastic#15652
- Remove pr.md file (not needed in integration packages)
- Remove empty lines from test-sgt.log
- Anonymize test DeviceUUID values

Addresses review comments from @P1llus and @bhapas
@yahyaghani yahyaghani force-pushed the feat/cisco_ftd-sgt-failing-tests branch from 0d61098 to 3833710 Compare October 26, 2025 20:15
@yahyaghani yahyaghani requested review from P1llus and bhapas October 26, 2025 20:17
bhapas
bhapas approved these changes Oct 27, 2025
View reviewed changes
Copy link
Contributor

@bhapas bhapas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please fix the manifest entry with the new version for CI to go green.. Else LGTM

@elasticmachine
Copy link

elasticmachine commented Oct 28, 2025 •
edited
Loading

💔 Build Failed

Failed CI Steps

History

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@P1llus P1llus P1llus approved these changes

@bhapas bhapas bhapas approved these changes

Assignees

No one assigned

Labels

documentation Improvements or additions to documentation. Applied to PRs that modify *.md files. Integration:cisco_ftd Cisco FTD Team:Integration-Experience Security Integrations Integration Experience [elastic/integration-experience]

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[cisco_ftd]: parse additional fields related to SGT/EGT

5 participants

@yahyaghani @elasticmachine @P1llus @bhapas @andrewkroh