009

IEP009: IntelMQ Data Format: Describe product and mark vulnerabilities

Today we see increasing use in tracking known vulnerabilities in specific online accessible software. IntelMQ Data Format does not natively support any kind of identification of which product is affected, or presentation of the vulnerability list. The only existing method is to use the extra field.

This IEP aims to address the increasing use of vulnerability reports by proposing additional harmonisation fields.

Background

You can see an increasing amount of vulnerability-related data being handled by IntelMQ (such as vulnerability reports from ShadowServer [1] [2]). Vulnerable online-accessible devices are also an ongoing problem [3 (de)] [4 (de)], and informing the constituencies about them is an important task for many security teams.

Use cases

Assuming that events are properly tagged with product and vulnerability information, the following use cases are possible:

• Clearly indicate the affected software/device to network administrators, thereby improving their response time.
• Deduplicate vulnerability information from different sources.
• Group vulnerabilities that affect the same product, e.g. for bulk distribution.
• Easily analyse collected data for vulnerability statistics.

Solution considerations

Existing solutions actively use e.g. the extra field to mark affected devices and vulnerabilities (e.g. [5]), but do not allow for compatibility between between data from different sources.

Another important question is how much information should be included in the IDF itself. Both product and vulnerability may have several related pieces of information (product: versions, release dates, available updates; vulnerability: an ID, affected versions, common weakness enumeration IDS, ...). Including all possible information in the IDF would make it too complex without a real improvement in data handling.

In terms of interoperability and vulnerability identification, the important reference is the Common Platform Enumeration [6]. This is de facto a standard for identifying affected devices.

Proposal

This IEP focuses changes on the identification of the existing product and vulnerabilities, which will easily retrieve other related information. For the best interoperability and querying, it's recommended to follo w the CPE naming convention [7] and/or the official CPE Product Dictionary [6], where applicable. The terms vendor, product, version are used here in the sense defined by CPE specification.

In addition, the following assumptions are made:

• The primary role of an IntelMQ event in this context is to identify a vulnerable product on an Internet-facing device. Therefore, only the currently used product should be listed; if a vulnerability affects multiple products or their versions, it's not relevant information.
• An event may list multiple vulnerabilities in an observed product, but not in multiple products on the same device. If a server contains multiple vulnerable products, multiple events should be generated by the parser or used in communication between IntelMQ instances. The IntelMQ operator can later decide to batch them in their own workflow.
• All but one of the suggested fields are strictly for machine processing. It is recommended to follow the CPE convention and it's dictionary, but to keep the flexibility (e.g. when CPE information is not available), only the lower case string needs to be used. Also the vulnerability IDs, which are not part of the CPE, should use lower case for better comparability between different sources.
• There are some existing fields such as event_description.text, event_description.url that can be used to store additional information for human operators. As such, the proposed fields do not include any description other than the product name.
• Machine-readable fields can still be used to retrieve additional information, e.g. query vulnerability databases for advisory URL or CPE product dictionary to get the human-readable product name.
• As there is an open discussion about a severity field [8], it's not a part of this IEP. However, if introduced, it can be used to mark the severity of a vulnerability.

Thus, the following new fields are proposed:

Group	Name	IDF type	Description	Example »
Product	product.full_name	String	A human readable product name. If a machine-readable format isn't available, this field should be used. It can directly use the version identification strings presented by the product. If not given, a good enough value can usually be constructed by concatenating product.product and product.version, or by consulting external sources such as the CPE Product Dictionary.	openssh_/8.9
Product	product.vendor	LowercaseString	Vendor name, recommended being as vendor in the CPE format.	openbsd
Product	product.name	LowercaseString	Product name, recommended being as the product in the CPE format.	openssh
Product	product.version	LowercaseString	Product version, recommended being as version in the CPE format	8.9
Product	product.vulnerabilities	LowercaseString	List of vulnerability IDs, separated by semicolons. It's recommended to use a CVE ID where available, and other easily retrievable IDs in other cases, e.g. Github Advisory Database ID [9]. Each vulnerability should only be listed once, and multiple values should be used if there are several different vulnerabilities. However, it's not necessary for a source to list all possible vulnerabilities for a given piece of software.	cve-2023-38408;cve-2023-28531;cve-2008-3844;cve-2007-2768

This should only be added to the event schema.