v5.7.0

New features:

• Log file configuration:
  • Add possibility to enable/disable main log file and to configure its location and daily roll-over parameters
  • Add possibility to log process chain output to separate log files
• Log file access:
  • Add HTTP endpoint to access process chain logs
  • Add UI page to display process chain log and to follow updates in real time
  • Display interactive log on process chain detail page (only visible if process chain logs are enabled)
  • Add button to download process chain logs
• VM setups:
  • Add possibility to attach additional volumes to VMs
  • Display additional VM volumes in UI
• Improved VM creation strategy:
  • Multiple VMs can be created in parallel. Use the maxCreateConcurrent parameter when specifying a VM setup.
  • The cloud manager will only create as many VMs in parallel as necessary for the running workflow(s)
• New runtime plugin interface, which allows the output of executables to be logged immediately. The old interface is still available but deprecated. It will be removed in Steep 6.0.0.
• Sort services in UI alphabetically
• Log caller location on retry

Maintenance:

• Update Gradle to 6.8.3
• Update Kotlin to 1.4.30
• Minor dependency updates
• Update testcontainers to fix CI build

Bug fixes:

• Do not retry an executable if the process chain has been cancelled
• Allow process chain to be cancelled even if agent currently waits for retry

v5.6.0

New features:

• Add possibility to deploy multiple agent instances (configuration item steep.agent.instances)
• Optimized scheduling algorithm:
  • Select best required capabilities by total count of remaining process chains. This makes sure process chains are distributed more evenly to agents supporting similar capabilities, which can reduce the overall runtime of workflows.
  • Cache agent responses during scheduling. This allows us to reduce the number of messages sent over the event bus by skipping agents that are known to be busy as well as those that do not support a given required capability set.
  • Continue scheduling even if an agent could not be allocated. This increases the scheduling throughput (i.e. the number of process chains scheduled in one step).
  • Do not unnecessarily run multiple scheduling lookups in parallel
  • Add MongoDB compound index to speed up fetching the next process chain
• New retry feature:
  • Specify a retry policy for executables in the workflow
  • Specify default retry policy in service metadata
  • Display service's retry policy in web UI
• Log the time each scheduling step took
• Delete orphaned VMs in parallel
• Do not include default value of store flag in serialized workflows
• Expose additional Prometheus metrics
  • Number of process chains executed by the scheduler
  • Number of retries performed by the local agent (per service ID)

Maintenance:

• Update UI dependencies
• Remove unnecessary log messages

Bug fixes:

• Pull alpine docker image before running unit tests

v5.5.0

New features:

• Remove legacy web UI
• Publish code coverage results to codecov (https://codecov.io/gh/steep-wms/steep)

Maintenance:

• Add unit tests for the progress estimator plugin type
• Bump bl from 4.0.2 to 4.0.3 in /ui
• Improve test coverage of LocalAgent and DockerRuntime
• Remove unused shell service
• Update UI dependencies

Bug fixes:

• Automatically resize workflow editor when browser window changes
• Also replace argument types in runtimeArgs
• Fix process chain cancellation

v5.4.0

New features:

• A rough estimation of the progress of process chains will automatically be calculated during execution. This estimation is based on the number of actions already executed divided by the total number of actions in the process chain. This information is available through the processchains endpoint (see in particular the process chain data model). It will also be displayed in the web UI.
• A new plugin type has been added to provide a more fine grained estimation of the progress of process chains: progress estimator plugins can analyse the log output of running processing services and look for hints (e.g. percentages or number of bytes processed).
• The executable data model has been extended to keep the service ID
• Provisioning templates can now use the new readFile function to read a local file and deploy it to a VM.
• A new page has been added to the UI that allows users to submit new workflows or to resubmit existing ones. A resubmit button has been added to the dropdown menu on the workflow detail page.

Maintenance:

• Improve UI responsiveness
• Update UI dependencies (in particular update Next.js to 9.5.1)
• Migrate UI to styled-jsx
• Update Vert.x to 3.9.2
• Update Kotlin to 1.4.0

Bug fixes:

• Fix an unnecessary page refresh in the web UI
• Render HTTP errors with correct content type (i.e. text/plain)

v5.3.0

New features:

• Add database migration to remove deprecated data model properties. This will rename the argument type argument in process chains to input and merge parameters of execute actions into their inputs.

Maintenance:

• Finally remove deprecated properties from internal data models. The API is not affected by this and remains backward compatible.
• Bump lodash from 4.17.15 to 4.17.19

v5.2.1

Bug fixes:

• Restore backward compatibility. Re-add deprecated properties to data models.

v5.2.0

New features:

• Allow separate SSH usernames per setup to be configured
• Allow additional environment variables to be configured that will be passed to containers created by the Docker runtime
• Add validator that checks for common mistakes in posted workflows. Invalid workflows will be rejected with HTTP status code 400.
• Use new web UI by default. The old UI is now deprecated and will be removed in one of the next versions.
• Deprecate action parameters and service arguments:
  • Parameters of execute actions are actually unnecessary because they have the same structure as inputs. We decided to deprecate parameters. They will be removed in Steep 6.0.0. In the meantime, parameters of an execute action in a posted workflow will automatically be merged into its inputs. A warning will be logged.
  • The same applies to service parameters with the type argument. They are unnecessary because they can be easily mapped to inputs. We deprecated the type argument and will remove it in Steep 6.0.0. For the time being, argument will automatically be replaced by input in your services.yaml file. A warning will be logged.

Bug fixes:

• Prevent duplicate setup IDs
• Do not throw exception in NotifyingSubmissionRegistry if VM does not exist
• Update websocket-extensions to 0.1.4
• Fix position of text in dropdown button

v5.2.0-beta.3

New features:

• Allow additional environment variables to be configured that will be passed to containers created by the Docker runtime

v5.2.0-beta.2

Bug fixes:

• Do not throw exception in NotifyingSubmissionRegistry if VM does not exist
• Update websocket-extensions to 0.1.4

v5.2.0-beta.1

New features:

• Allow separate SSH username per setup

Bug fixes:

• Prevent duplicate setup IDs