Request to add a VRL function to parse logs generate by auditd

[7czl]

A note for the community

Please vote on this issue by adding a 👍 [reaction](https://blog.github.com/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/) to the original issue to help the community and maintainers prioritize this request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

Use Cases
Using vector gathering the local security logs(/var/log/secure, /var/log/message, /var/log/sudo.log ....etc) to kafka and export logs from kafka to opensearch.

I tried to parse the logs generated by auditd which stored in /var/log/audit/audit.log like the format below. I tried parse_* functions listed in the document but all failed.

type=CRED_ACQ msg=audit(1729236001.971:1172784): pid=10824 uid=0 auid=4294967295 ses=4294967295 msg='op=PAM:setcred grantors=pam_env,pam_unix acct="root" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success'

Attempted Solutions
I have tried parse* family function to parse logs generate by auditd but failed.
https://vector.dev/docs/reference/vrl/functions/#parse-functions
Currently struggling on using parse_regex to extract useful field from the log. Would you please add a function to parse the logs generated by auditd?

No response
Proposal

No response
References

No response
Version

No response

[jorgehermo9]

Duplicated of vectordotdev/vrl#66 (that repository should be used instead for VRL-related feature request).

I'm currently working on it in vectordotdev/vrl#1010. That PR was blocked until recently due to a licensing discussion that recently got resolved hillu/linux-audit-parser-rs#1 (comment). If I find enough time these weeks I may work on this.

As a workaround meanwhile, maybe you can use vectordotdev/vrl#66 (comment).

However, it would be very useful for me to have more parsing thoughts in vectordotdev/vrl#66 and to know how would you prefer the parsing to look like. For example, we have doubts about vectordotdev/vrl#1010 (comment).

Thank you very much for the request though :)

[jorgehermo9]

Although, it would be interesting to add a decoding.codec=auditd to vector's sources after it is done in VRL, the implementation would be very similar and in simple cases it would be more efficient to use a codec than a remap transform

@jszwedko is this interesting enough to create a tracking issue for it? I could address it after my VRL PR

[7czl]

Thank you for the detailed explanation, it is appropriate to put this issue to vrl repository, I will close this and use the workaround. from and will check out the parsing style discussion later on

[jszwedko]

Although, it would be interesting to add a decoding.codec=auditd to vector's sources after it is done in VRL, the implementation would be very similar and in simple cases it would be more efficient to use a codec than a remap transform

@jszwedko is this interesting enough to create a tracking issue for it? I could address it after my VRL PR

That would be interesting, but seems like a bit of a niche case. vrl is already usable as a codec so it would be possible to use the new function there. Granted it would have some overhead vs. a native decoder.