Skip to content

Releases: cisagov/Malcolm

Malcolm v23.08.0

15 Aug 17:55
@mmguero mmguero
v23.08.0
d5ee03e
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
Malcolm v23.08.0

Malcolm v23.08.0 is a minor release with a few improvements, bug fixes and component updates.

EDIT: I've discovered a regression in the Hedgehog Linux startup script that formats drives to make them available for artifact capture. I'm investigating now. If this affects you, you might want to avoid this release until I put out a patch.

v23.07.1...v23.08.0

  • Features and enhancements

  • Bug fixes

    • Fixed extracting Malcolm version during ISO build
    • Workaround for wireshark no longer publishing raw manuf (OUI) list (idaholab#230)
    • Remove news feed from default NetBox dashboard (as it would try to reach out to the web for RSS updates)

  • Component version updates

Malcolm and Hedgehog Linux may be obtained by pulling or building the Docker images and/or building the ISO installer images as described in the documentation. Unofficial ISO installer images for Malcolm and Hedgehog Linux are not hosted on GitHub, but may be downloaded from https://malcolm.fyi/.

Assets 8
Loading

Malcolm v23.07.1

20 Jul 18:21
@mmguero mmguero
v23.07.1
06b9562
Compare
Choose a tag to compare
Malcolm v23.07.1

Malcolm v23.07.1 is a patch release fixing a single bug.

v23.07.0...v23.07.1

Malcolm and Hedgehog Linux may be obtained by pulling or building the Docker images and/or building the ISO installer images as described in the documentation. Unofficial ISO installer images for Malcolm and Hedgehog Linux are not hosted on GitHub, but may be downloaded from https://malcolm.fyi/.

Assets 8
Loading

Malcolm v23.07.0 [see EDIT at the top of the release notes]

19 Jul 14:55
@mmguero mmguero
v23.07.0
432a90a
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
Malcolm v23.07.0 [see EDIT at the top of the release notes]

EDIT - A bug in how Modbus traffic was parsed was discovered shortly after this release. A v23.07.1 release will be put out in the next day or so, you may want to wait for that.

Malcolm v23.07.0 is a feature release with a number of improvements, bux fixes and component updates.

v23.05.1...v23.07.0

  • New features

    • scan docker images built via GitHub actions for vulnerabilities using Trivy (idaholab#218)
    • document building and deplolying Malcolm with an AWS AMI image (idaholab#205)
    • handle Arkime field actions (idaholab#200)
    • kubernetes: document how to get running on Amazon EKS (idaholab#194)
    • Populate NetBox inventory via passively-gathered network traffic metadata (basic functionality, work in progress) (idaholab#135)

  • Enhancements

    • use .tar.xz instead of .tar.gz for packaging Malcolm docker images for better compression (and smaller ISO file size)
    • Malcolm documentation edits (idaholab#204)
    • add option to enable SSH via password in hedgehog's configure-interfaces.py script (idaholab#158)
    • updated "Network Traffic Analysis with Malcolm" slides
    • use an init container in Kubernetes container startup to ensure necessary directories get created under PersistentVolume objects before startup
    • improvements to identifying source of third-party logs sent via fluent bit
    • don't do unnecessary clone of Zeek plugins, just install using URL
    • parse bacnet_device_control.log produced by the icsnpp-bacnet parser for Zeek

  • Bug fixes

    • maxlogins value includes tmux sessions, can lock user out of SSH (idaholab#214)
    • curl rc file for connecting to external OpenSearch without auth enabled causes logstash startup to fail (idaholab#209)
    • failure to parse some suricata alerts due to integer type which should be indexed as long (idaholab#206)
    • netbox-restore doesn't work in Kubernetes (idaholab#202)
    • PCAP File with no - in pcapng Fails to Upload (#265)
    • disable NetBox telemetry

  • Component version updates

Malcolm and Hedgehog Linux may be obtained by pulling or building the Docker images and/or building the ISO installer images as described in the documentation. Unofficial ISO installer images for Malcolm and Hedgehog Linux are not hosted on GitHub, but may be downloaded from https://malcolm.fyi/.

Assets 8
Loading

Malcolm v23.05.1

16 May 18:28
@mmguero mmguero
v23.05.1
8221433
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
Malcolm v23.05.1

Malcolm v23.05.1 is a minor release with a few component version updates and bug fixes, particularly to fix an issue with install.py where the ownership of .env files in the config directory may get incorrectly set to root rather than the unprivileged user.

v23.05.0...v23.05.1

  • Enhancements and bug fixes

    • install.py can create .env files 0:0 ownership instead of unprivileged user ownership (#253)
    • both zeek and zeek-live containers are trying to pull intel feeds on startup (idaholab#196)
    • Make sure a few Arkime fields (http.xff*) get created in the index template with the right field types to avoid aggregation query issues
    • Tweaks to convenience scripts (malcolmmonitor and sensormonitor) in ISO-installed Malcolm and Hedgehog Linux environments
    • Added some .service files for the ISO-installed version of Malcolm to be able to feed itself resource statistics via Fluent Bit
    • Documentation updates

  • Component version updates

Malcolm and Hedgehog Linux may be obtained by pulling or building the Docker images and/or building the ISO installer images as described in the documentation. Unofficial ISO installer images for Malcolm and Hedgehog Linux are not hosted on GitHub, but may be downloaded from https://malcolm.fyi/.

Assets 8
Loading

Malcolm v23.05.0

01 May 21:46
@mmguero mmguero
v23.05.0
c02a6e9
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
Malcolm v23.05.0

Malcolm v23.05.0 is a major release with new features, enhancements, component version updates and bug fixes.

IMPORTANT NOTE: Malcolm v23.05.0 has completely changed the way it manages its settings: rather than using environment variables found at the top of the docker-compose.yml file, it uses environment variables in .env files inside of the config directory. The locations of a number of configuration files have also changed. It's not recommended to update to Malcolm v23.05.0 from a previous version of Malcolm. Instead, shut down Malcolm, rename your old Malcolm installation directory to something else, and reconfigure Malcolm using ./scripts/configure and ./scripts/auth_setup.

v23.04.0...v23.05.0

Malcolm and Hedgehog Linux may be obtained by pulling or building the Docker images and/or building the ISO installer images as described in the documentation. Unofficial ISO installer images for Malcolm and Hedgehog Linux are not hosted on GitHub, but may be downloaded from https://malcolm.fyi/.

Assets 8
Loading

Malcolm v23.04.0

05 Apr 14:50
@mmguero mmguero
v23.04.0
37bb7f7
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
Malcolm v23.04.0

Malcolm v23.04.0 is a release with enhancements, component version updates and bug fixes.

IMPORTANT NOTE: In March 2023 Docker Inc. announced its decision to sunset the "Docker Free Team" plan, which prompted us to decide to migrate away from Docker Hub to the Github Container Registry or "ghcr" (see idaholab#163). Due to public backlash, Docker Inc. reversed its decision. However, the Malcolm project will continue with the decision to use GHCR beginning with this release (Malcolm v23.04.0) and moving forward. If you're updating an existing instance of Malcolm, it's recommended that you back up your docker-compose.yml and docker-compose-standalone.yml files, replace them with the ones from this release and re-run ./scripts/install.py --configure to ensure that you're pointing at the latest images (this is actually always good practice when moving to a new release of Malcolm).

v23.03.0...v23.04.0

  • Enhancements

    • autostart install.py --configure on Malcolm ISO first boot (idaholab#157)
    • clarify information about auth_setup's use of external OpenSearch connections (idaholab#160)
    • migrate away from DockerHub container registry (idaholab#163)
    • give easier option for transferring SSL client files from Malcolm to forwarder (idaholab#177)
      • added tx-rx-secure.sh script as wrapper around croc automatically creating and using a local-only relay

  • Component version updates

  • Fixes

    • XFCE4's "save session on exit" causes conflict with Hedgehog kiosk mode if firefox instance is started upon session restore (idaholab#164)
    • docker-compose move from go-yaml/v3 breaks Malcolm's docker-compose YAML files (idaholab#178, docker/compose#10411)
    • increase index.mapping.nested_fields.limit in opensearch index template (idaholab#180)

Malcolm and Hedgehog Linux may be obtained by pulling or building the Docker images and/or building the ISO installer images as described in the documentation. Unofficial ISO installer images for Malcolm and Hedgehog Linux are not hosted on GitHub, but may be downloaded from https://malcolm.fyi/.

Assets 6
Loading

Malcolm v23.03.0

08 Mar 20:38
@mmguero mmguero
v23.03.0
50cf2eb
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
Malcolm v23.03.0

Malcolm v23.03.0 is a release with enhancements, component version updates and bug fixes.

v23.02.0...v23.03.0

  • Enhancements

    • Replace Zeek's misc/scan.zeek with ncsa/bro-simple-scan
    • terminate start and restart scripts once Malcolm has started properly (#240 and #241, thanks @Njinx)
    • minor usability improvements for ISO-installed Malcolm and Hedgehog (idaholab#155)
      • Added a "Configure Malcolm" menu item (under the "Internet" GTK menu with the other Malcolm stuff) and launcher on the top panel of icons in Malcolm. This runs ./scripts/install.py --configure in full screen. May look at starting this automatically on first boot in the future. (Malcolm)
      • Added Malcolm shortcut to gtk-3.0/bookmarks so it shows up in Thunar sidebar (Malcolm)
      • Added /opt/sensor/sensor_ctl shortcut to gtk-3.0/bookmarks so it shows up in Thunar sidebar (Hedgehog)
      • Have tilix from launcher panel start in /opt/sensor/sensor_ctl (Hedgehog)
    • minor tweaks to defaults for install.py --configure (enable offline-capable file scanners by default)
    • interrupt NetBox startup import script when netbox-restore is run
    • added NetBox restore logic to reset_and_auto_populate.sh script (used mostly for demos and presentations)

  • Component version updates

  • Fixes

    • last few seconds' Zeek logs prior to log rotation may be lost (idaholab#151)
    • in ISO-packaged Malcolm installation scripts directory, symlink netbox-backup and netbox-restore to control.py
    • improve opensearchpy connect/health check logig in pcap_watcher.py in pcap-monitor container

Malcolm and Hedgehog Linux may be obtained by pulling or building the Docker images and/or building the ISO installer images as described in the documentation. Unofficial ISO installer images for Malcolm and Hedgehog Linux are not hosted on GitHub, but may be downloaded from https://malcolm.fyi/.

Contributors

Njinx
Assets 2
Loading

Malcolm v23.02.0

03 Feb 22:05
@mmguero mmguero
v23.02.0
360fbd6
Compare
Choose a tag to compare
Malcolm v23.02.0

Malcolm v23.02.0 is a feature release with new features and enhancements, component version updates and bug fixes.

v23.01.0...v23.02.0

  • New features

    • Compare and highlight discrepancies between NetBox inventory and observed network traffic (idaholab#133)
      • Added Zeek Known Summary and Asset Interaction Analysis dashboards which include visualizations about uninventoried devices and services
      • Added Uninventoried Internal Assets and Uninventoried Observed Services views to Arkime
      • Documentation updates related to NetBox
    • Added default device roles and service templates for initial NetBox population
    • Added netbox-backup/netbox-restore scripts to control.py for NetBox database and media
    • Added zeek_script_to_malcolm_boilerplate.py script for automating some of the tasks involved with adding new Zeek logs to Malcolm

  • Enhancements

    • configurable dark mode for OpenSearch Dashboards (idaholab#145)
    • added third-party OpenSearch Dashboards custom visualization component lguillaud/osd_transform_vis
    • modbus and modbus_detailed logs should be better normalized for event.action and event.result (idaholab#146)
    • Added -n argument to script/logs akin to tail -n (#234, thanks @Njinx)
    • Accounted for major additions to the OPCUA-Binary parser in both parsing and the corresponding dashboard
    • Set state:storeInSessionStorage to true for OpenSearch dashboards: this allows some complicated visualizations to be built with the Vega and Transform plugins, at the cost of having some URL bookmarks not contain every possible state the current dashboard has
    • Added related.device_name for normalization and pivoting
    • Removed related.segment in favor of ECS network.name
    • allow NetBox in Malcolm's "read-only" configuration

  • Component version updates

  • Fixes

    • failure to build logstash container due to illformed gem requirement (idaholab#144)
    • when running as UID/GID other than 1000, chown on dashboards and logstash containers takes a LONG time (idaholab#148)
    • Logs are being spammed with Suricata warnings pertaining to duplicate rules (#233)
    • Opensearch statistics are now parsed correctly when only a one node is present (#232, thanks @Njinx)
    • Explicitly check /usr/bin for docker-compose in case for some reason that's not in PATH (?) (#226)
    • Some refactoring of the Zeek pipeline in Logstash

Malcolm and Hedgehog Linux may be obtained by pulling or building the Docker images and/or building the ISO installer images as described in the documentation. Unofficial ISO installer images for Malcolm and Hedgehog Linux are not hosted on GitHub, but may be downloaded from https://malcolm.fyi/.

Contributors

Njinx
Assets 6
Loading

Malcolm v23.01.0

09 Jan 16:21
@mmguero mmguero
v23.01.0
3afb619
Compare
Choose a tag to compare
Malcolm v23.01.0

Malcolm v23.01.0 is a feature release with new features and enhancements, component version updates and bug fixes.

v6.4.3...v23.01.0

  • New features

  • Enhancements

  • Component version updates

  • Fixes

    • when using custom locations, pcap/upload and pcap/processed directories don't get created correctly after wipe (idaholab/Malcolm#140)
    • one Malcolm instance forwarding to another secondary tier Malcolm instance continually imports opensearch index templates (idaholab/Malcolm#142)
    • Updated source code copyright dates from 2022 to 2023

  • Deprecated

    • Removed host-map.txt and cidr-map.txt for host and subnet name assignment (use net-map.json file or NetBox now)
    • MAC address to host name mapping for host and subnet name assignment (MAC address is too inconsistent to use as an identifier for a host as network captures may not show the actual MAC address for a given host's communication)

Malcolm and Hedgehog Linux may be obtained by pulling or building the Docker images and/or building the ISO installer images as described in the documentation. Unofficial ISO installer images for Malcolm and Hedgehog Linux are not hosted on GitHub, but may be downloaded from https://malcolm.fyi/.

Assets 6
Loading

Malcolm v6.4.3

06 Dec 21:44
@mmguero mmguero
v6.4.3
082bae7
Compare
Choose a tag to compare
Malcolm v6.4.3

Malcolm v6.4.3 is a minor release containing enhancements, component version updates and bug fixes.

v6.4.2...v6.4.3

  • Enhancements

    • Import the NetBox Device Type Library on NetBox first run to populate manufacturers, device types, models and modules
    • idaholab/Malcolm#127 have install.py --configure ask about other storage locations for PCAP, Zeek logs and OpenSearch indices
    • idaholab/Malcolm#128 have install.py --configure prompt for Arkime to manage uploaded PCAP files or not

  • Component version updates

  • Fixes

    • Fix some bad links in the documentation and other minor documentation improvements
    • Fix idaholab/Malcolm#126, suricata logs show up in Arkime as "notip" for the protocol
    • Fix idaholab/Malcolm#129, filtering by rootId in Arkime returns no results
    • Fix Docker health checks for NetBox and supporting containers
    • Fix "read-only" version of nginx.conf
    • Tweaks to install.py memory recommendations

Malcolm and Hedgehog Linux may be obtained by pulling or building the Docker images and/or building the ISO installer images as described in the documentation. Unofficial ISO installer images for Malcolm and Hedgehog Linux are not hosted on GitHub, but may be downloaded from https://malcolm.fyi/.

Assets 6
Loading