Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 10 additions & 5 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

31 changes: 22 additions & 9 deletions docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,28 @@ Welcome to the Project Sysinspect!
==================================

.. note::
This documentation covers **Sysinspect** — the solution to
examine any system, based on its architecture Model Description.
This documentation covers **Sysinspect** — the multi-solution to many things.

Welcome to **Sysinspect**: originally conceived as an engine for anomaly detection and root cause analysis. The name
itself is a portmanteau of "system" and "inspection", which should give you a fair idea of its intent.

**Sysinspect** began life as a generic engine designed to monitor system consistency, using a formal Model Description
of the system's architecture. Over time, it has grown into a collection of tools and libraries for system introspection,
configuration management, anomaly detection, root cause analysis, and automated remediation. The project is shaped by
the needs and contributions of its users, and is intended to be both hackable and extensible.

With **Sysinspect**, you can:

- Examine and introspect arbitrary systems, provided you have a Model Description of their architecture.
- Detect anomalies and perform root cause analysis using a combination of rules, heuristics, and data-driven methods.
- Track configuration drift and check compliance against a known-good state.
- Automate remediation actions, either as one-offs or as part of a workflow.
- Extend the system with your own modules, handlers, and integrations.
- Run it on Android, embedded Linux environments with questionable tooling (or none at all), or traditional server environments.

If you enjoy tinkering with system internals, building automation, or just poking around to see how things work,
Sysinspect is meant to be a toolkit you can adapt and extend. Contributions, bug reports, and wild ideas are all
welcome—see the section on contributing for how to get involved.

.. toctree::
:maxdepth: 1
Expand All @@ -27,13 +47,6 @@ Welcome to the Project Sysinspect!
tutorial/module_management


Welcome to Sysinspect: an engine of Anomaly Detection and Root Cause Analysis.
The name consists of two words "system" and "inspection", resulting in "suspicion".

This engine is indented to perform anomaly detection and root cause analysis on
any system. It is using Model Description as a source of knowledge and a collection of
modules with telemetry data in order to perform various testing scenarios.

Licence
-------

Expand Down
164 changes: 128 additions & 36 deletions docs/modeldescr/actions.rst
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,7 @@ of an action as follows:
Below is the description of configuration sections:

``module: namespace``
^^^^^^^^^^^^^^^^^^^^^

This element assigns the content of an action to a specific module that will process it.
Example:
Expand All @@ -80,6 +81,7 @@ Below is the description of configuration sections:
module: sys.proc

``bind: [list]``
^^^^^^^^^^^^^^^^^

This element binds entities to the action. I.e. an action will process every
mentioned entity. Example:
Expand All @@ -91,6 +93,7 @@ Below is the description of configuration sections:
- journald

``state : [map]``
^^^^^^^^^^^^^^^^^

A configuration group for the particular state. It must be the same ID as state ID in the entities collection.
If actions processing the system in a serial fashion without knowing what it is even discovered, then how exactly
Expand All @@ -104,61 +107,144 @@ Below is the description of configuration sections:
the corresponding module aware of the currently processed state. Therefore, in case of the state is requested other
than it is currently detected on the device, the module should return **true**.

``opts|options: [list]``

Options element ``opts`` (or ``options``) specifies flags to the module, in case it is needed. For example, a module
called ``sys.proc`` might have different modes, such as checking if a process at all runs
and do nothing else, or return its PID or owner, even stop it, restart it etc — it depends on
a module. In any case, options would be statically passed in this action. Example:
``opts|options: [list]`` (optional)

.. code-block:: yaml
Options element ``opts`` (or ``options``) specifies flags to the module, in case it is needed. For example, a module
called ``sys.proc`` might have different modes, such as checking if a process at all runs
and do nothing else, or return its PID or owner, even stop it, restart it etc — it depends on
a module. In any case, options would be statically passed in this action. Example:

opts:
- info
.. code-block:: yaml

The example above is equivalent to a command line expression like this:
opts:
- info

``some-program --info``
The example above is equivalent to a command line expression like this:

``args|arguments: key/[list]``
``some-program --info``

The ``args`` (or ``arguments``) element specifies keywords to the module. One **distinct difference** from
a classic keywords is that this is a ``key/[list]`` *(of values)* rather then a ``key/value``.
Example:
``args|arguments: key/[list]`` (optional)

.. code-block:: yaml
The ``args`` (or ``arguments``) element specifies keywords to the module. One **distinct difference** from
a classic keywords is that this is a ``key/[list]`` *(of values)* rather then a ``key/value``.
Example:

args:
file:
- /var/log/messages
.. code-block:: yaml

The example above is equivalent to a command line expression like this:
args:
file:
- /var/log/messages

``some-program --file=/var/log/messages``
The example above is equivalent to a command line expression like this:

.. note::
``some-program --file=/var/log/messages``

Arguments and options are not directly one-to-one transpose of a CLI arguments.
They are just structures in JSON format, those still can be properly interpreted
by a module.
.. note::

As per note above, if a specific program requires multiple same arguments, this still
can be achieved by grouping them as a list under one argument. For example, if a CLI
equivalent is needed to this:
Arguments and options are not directly one-to-one transpose of a CLI arguments.
They are just structures in JSON format, those still can be properly interpreted
by a module.

``some-program --file=/var/log/messages --file=/var/log/dmesg``
As per note above, if a specific program requires multiple same arguments, this still
can be achieved by grouping them as a list under one argument. For example, if a CLI
equivalent is needed to this:

The form above still can be achieved in this form:
``some-program --file=/var/log/messages --file=/var/log/dmesg``

.. code-block:: yaml
The form above still can be achieved in this form:

.. code-block:: yaml

args:
file:
- /var/log/messages
- /var/log/dmesg

In this case a module will get a JSON data with ``file`` key and a list of paths,
that can be then translated by a module in whatever required format.

``context|ctx [map]`` (if defined)

Context variable definitions for **documentation** purposes, when they are required by a model description.
They are defined as key/value pairs, where key is the variable name and value is its description.
Example:

.. code-block:: yaml+jinja

context:
foo: Some value that will be used to run the module
bar: Some other flag or value for the same reason

# And then usage of context variables in args:
args:
{% if context.foo is defined %}
something: "context(foo)"
{% endif %}
{% if context.bar is defined %}
another: "context(bar)"
{% endif %}

Surely, ``context`` does not have to be defined, but then API will not reflect and introspect
the whole model properly, because SysInspect will first render and then examine the model. As it is seen
in the example above, context variables are used in Jinja2 templating. In this case ``{% if %}`` clause
will just cut out a chunk of Model description, rendering impossible to reflect state arguments to the
end user.

``conditions|conds: [map]`` (optional)

Conditions are additional constraints that setting up the environment for a module.
For example, a module might require to run as ``nobody`` user, or it might require
a specific working directory, or it might require a specific amount of memory
or disk space. These conditions are setting up the environment for a module.
Example:

.. code-block:: yaml

conditions: # or conds:
uid: 65432 # nobody user
gid: 65432 # nobody group
virtual-memory: 64Mb

# working directory can be set only if working-disk is defined
working-dir: /tmp
working-disk: 100Mb

This is important to understand that conditions are not using ``sudo`` mechanism.
Which means, conditions can only limit down the privileges of a module, but
cannot elevate them. For example, if a minion is running as ``nobody`` user,
a module cannot be elevated to ``root`` user. However, if a minion is running as
``root``, a module surely can be dropped down to ``nobody`` user.

.. note::

Default conditions are transparent, acquiring all privileges of the minion.
That is, ``uid`` and ``gid`` will be the same as the minion is running.
``working-dir`` will be any current one, ``virtual-memory`` and ``disk`` are
as limited as allowed to the minion.

Here is the list of available options:

``uid`` and ``gid``

Numeric values of the user and group respectively.

``virtual-memory``

Maximum amount of virtual memory a module can allocate.

``working-dir``

Working directory for a module.

``working-disk``

Amount of disk space a module can use.

``fsize-cap``

args:
file:
- /var/log/messages
- /var/log/dmesg
Maximum size of a file a module can create.

In this case a module will get a JSON data with ``file`` key and a list of paths,
that can be then translated by a module in whatever required format.


Examples of Actions
Expand Down Expand Up @@ -254,6 +340,12 @@ Another example, showing static data references. Consider the following configur
- syslogd
state:
$:
conditions:
uid: 0
gid: 0
virtual-memory: 64Mb
disk: 100Mb
working-dir: /tmp
args:
# Variable $(foo.bar) always refers to a full path from the document root.
- free-disk: "static(entities.syslogd.claims.storage.free)"
Expand Down
1 change: 1 addition & 0 deletions libsysinspect/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -52,3 +52,4 @@ jsonpath_lib = "0.3.0"
jsonpath-rust = "1.0.4"
humantime = "2.3.0"
humantime-serde = "1.1.1"
libc = "0.2.176"
17 changes: 17 additions & 0 deletions libsysinspect/src/intp/actions.rs
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,11 @@ pub struct ModArgs {
#[serde(alias = "args")]
arguments: Option<IndexMap<String, String>>,

#[serde(alias = "ctx")]
context: Option<IndexMap<String, String>>, // Context variables definition for Jinja templates. Used only for model documentation.

#[serde(alias = "conds")]
conditions: Option<IndexMap<String, Value>>, // Conditions to be met for this state
}

impl ModArgs {
Expand All @@ -46,6 +50,11 @@ impl ModArgs {
pub fn context(&self) -> IndexMap<String, String> {
self.context.to_owned().unwrap_or_default()
}

/// Get conditions
pub fn conditions(&self) -> IndexMap<String, Value> {
self.conditions.to_owned().unwrap_or_default()
}
}

#[derive(Debug, Serialize, Deserialize, Default, Clone)]
Expand Down Expand Up @@ -200,6 +209,7 @@ impl Action {
// Setup modcall
let mut modcall = ModCall::default().set_state(state).set_module(mpath).set_aid(self.id()).set_eid(eid.to_string()).set_constraints(cst);

// Set module launching arguments
for (kw, arg) in &mod_args.args() {
let mut arg = arg.to_owned();
if let Ok(Some(func)) = functions::is_function(&arg) {
Expand All @@ -222,9 +232,16 @@ impl Action {
modcall.add_kwargs(kw.to_owned(), arg);
}

// Set module launching options
for opt in &mod_args.opts() {
modcall.add_opt(opt.to_owned());
}

// Add module launching conditions
for (kw, cond) in mod_args.conditions() {
modcall.add_condition(kw, cond.clone());
}

self.call = Some(modcall);
} else {
return Err(SysinspectError::ModelDSLError(format!(
Expand Down
Loading
Loading