Meta values and runtime expressions

[hawkeyexl]

New features: Meta values and runtime expressions

Runtime expressions

Runtime expressions allow the use of meta values and operators within other values, evaluated at runtime. Runtime values support JSON and JSON pointer expressions. Environment variables within runtime expressions are loaded before the expression is evaluated.

Runtime expressions may include meta values, JSON, JSON Pointer, JSONPath, XPath, and jq expressions. Some fields, such as assertion statements, accept runtime expressions without additional markup, but ALL other fields MUST be able to identify and evaluate runtime expressions contained between {} characters. JSON, JSON Pointer, JSONPath, XPath, and jq expressions need to be identified programmatically and individually resolved before evaluating operators. JSONPath, XPath, and jq expressions MUST NOT be nested.

Runtime expressions MUST need to accept particular JSONPath or XPath versions to evaluate. Current supported JSONPath versions are draft-goessner-dispatch-jsonpath-00. Current supported XPath versions are xpath-30, xpath-20, or xpath-10.

Operators

Operator	Description
<	Less than
<=	Less than or equal
>	Greater than
>=	Greater than or equal
==	Equal
!=	Not equal
!	Not
&&	And
||	Or
()	Logical Grouping
[]	Index (0-based)
.	Property de-reference
contains	value is an array that contains comparison value OR value is an object that contains at least the comperison object
oneOf	value is present in the comparison array
matches	value has a match to the specified regular expression

Usage examples

• $$statusCode == 200
• $$response.body[?count(@.pets) > 0]
• $$element.text matches ^Hello World.*$

•  {
     "url": "{$$response.body#/url}"
   }

Meta values

Meta values provide access to runtime information during test execution. Access them using the $$ prefix. For example: $$timestamp or $$tests.myTest.outputs.

Global Scope

Available everywhere in the test specification.

System Information

• $$timestamp - Current timestamp in ISO format
• $$system.os - Operating system name
• $$system.platform - Platform details (e.g., 'linux', 'mac', 'windows')
• $$system.arch - System architecture
• $$system.version - OS version
• $$system.hostname - System hostname
• $$system.tmpdir - System temp directory
• $$system.applications - Array of recognized applications Doc Detective can interact with that are available (e.g., 'chrome', 'firefox', 'edge', 'chromedriver')

Environment

• $$env.{{variable}} - Access to environment variables
• $$config - Current Doc Detective configuration
• $$workingDirectory - Current working directory

Test Run Information

• $$runId - Unique identifier for the current test run
• $$startTime - Test run start time
• $$duration - Test run duration in milliseconds
• $$specs - Object containing all specs in the run
• $$specs.{{id}} - Access specific spec (and everything in the spec scope) by ID

Specification Scope

Available within a test specification.

• $$spec - Specification object
• $$spec.output - User-specified output values
• $$spec.file - Specification file path
• $$spec.directory - Directory containing the spec

Test Management

• $$tests - Object containing all tests in the specification
• $$tests.{{id}} - Access specific test by ID
• $$tests.{{id}}.steps - Steps within a specific test
• $$tests.{{id}}.outputs - User-specified output values
• $$tests.{{id}}.status - Test status (pass/fail)
• $$tests.{{id}}.duration - Test duration
• $$tests.{{id}}.errors - Array of test errors
• $$tests.total - Total number of tests
• $$tests.passed - Number of passed tests
• $$tests.failed - Number of failed tests

Test Scope

Available within a specific test.

Step Management

• $$steps - Object containing all steps in the current test
• $$steps.{{id}} - Access specific step by ID
• $$steps.{{id}}.outputs - Step outputs
• $$steps.{{id}}.status - Step status
• $$steps.{{id}}.duration - Step duration
• $$steps.{{id}}.errors - Array of step errors
• $$steps.total - Total number of steps
• $$steps.passed - Number of passed steps
• $$steps.failed - Number of failed steps

Browser Context (when applicable)

• $$browser.name - Current browser name
• $$browser.version - Browser version
• $$browser.userAgent - Browser user agent
• $$url - Current page URL
• $$title - Current page title

Source Tracking

• $$source - Source file generating the test (if applicable)
• $$source.file - Source file path
• $$source.line - Line number in source
• $$source.description - Source description (e.g., from Arazzo)

Step Scope

Available within a specific step.

HTTP Requests (for httpRequest action)

• $$request.url - Request URL
• $$request.method - Request method
• $$request.headers - Request headers
• $$request.data - Request body data
• $$response.status - Response status code
• $$response.headers - Response headers
• $$response.data - Response body data

Browser Elements (for find action)

• $$element - Current element reference
• $$element.text - Element text content
• $$element.html - Element HTML
• $$element.attributes - Element attributes
• $$element.value - Element value (for form inputs)
• $$element.isVisible - Element visibility state
• $$element.location - Element coordinates

Shell Commands (for runShell action)

• $$command - Executed command
• $$stdout - Command standard output
• $$stderr - Command standard error
• $$exitCode - Command exit code

File Operations

• $$file.path - Current file path
• $$file.content - File content (if applicable)
• $$file.size - File size
• $$file.modified - Last modified timestamp

Screenshots and Recordings

• $$screenshot.path - Screenshot file path
• $$screenshot.timestamp - Screenshot capture time
• $$recording.path - Recording file path
• $$recording.duration - Recording duration

Usage Examples

{
  "steps": [
    {
      "action": "httpRequest",
      "url": "https://api.example.com",
      "validateFn": "$$response.status === 200 && $$response.data.success === true"
    },
    {
      "action": "find",
      "selector": "#submit-button",
      "validateFn": "$$element.isVisible && $$element.text.includes('Submit')"
    },
    {
      "action": "runShell",
      "command": "echo 'Hello'",
      "validateFn": "$$stdout.trim() === 'Hello' && $$exitCode === 0"
    }
  ]
}

Best Practices

1. Use the most specific scope needed for your validation
2. Check meta value existence before use
3. Consider using multiple meta values for robust validation
4. Document which meta values your tests depend on