Skip to content

Pastelyzer 0.10-beta

Pre-release
Pre-release
Compare
Choose a tag to compare
@jdz jdz released this 15 Jun 13:34
· 37 commits to master since this release
rel-0.10-beta
cb9a0cc

  • discard filter action can be used to get rid of unwanted artefacts.

  • All non-discarded artefacts are stored in database.

  • Artefacts now have important flag which can be set with set-important
    filter action. Artefacts marked important are highlighted in dashboard and
    the flag is also stored in the database.

  • Artefacts now have note slot which can be set with set-note filter
    action. This note is stored in the database along with the artefact.

  • No more IMPORTANT-BANK-CARD class ("important" flag can be used with
    BANK-CARD-NUMBER artefact instead).

  • Introduction of user sets — IPv4 networks, super-domains (multi-level) and
    bank card bins. Multiple sets of the same or different kinds can be loaded
    (previously with command-line options only one set of each kind could be
    used).

  • Important: the following command line options have been made redundant
    with the introduction of user sets: --important-cc-bins,
    --interesting-tlds, --networks-file and --tlds-file. Running
    pastelyzer in server mode with any of these options provided will show you
    sample configuration fragments you should integrate into your configuration
    instead. The relevant documentation is section "3.2 User sets" (and also
    section "3.6 Example configuration").

  • Important: the default value of --resolve-domains option is no longer
    guessed (since --networks-file was removed). This means that domain
    resolution must be turned on explicitly.

  • Comparison filter functions (<, = and >) can also be used with strings.

  • mixed-case? filter predicate.

  • Additional extractors:

    • note
    • source-string
    • bytes
    • context-before
    • context-after
    • source-context

  • Discoveries are logged with stat instead of hit.

  • Added an ability to reprocess previously processed documents. Useful when
    upgrading to a new version of pastelyzer that changes (hopefully improves)
    the way documents are processed (like this version). The documentation
    mentions this, but so that users don't miss it:

    Important: if pastelyzer configuration contains rules to send emails or
    interact with other systems/programs then the reprocessing might trigger
    these rules again. It is therefore advisable to create a separate
    configuration file for reprocessing and either omit these rules or change
    them so that the information does not go to production systems.

  • An experimental API to query the database has been added (for cases when
    using curl and jq is preferable to SQL).

Assets 3
Loading