Add query engine support (JMESPath + Simple Pick)

[amondnet]

Problem Statement

The current output module only supports flat field filtering (e.g., "number,title,state"), which limits users to selecting top-level fields. This prevents:

• Accessing nested properties (user.name, metadata.created)
• Filtering arrays based on conditions (issues[?state=='OPEN'])
• Reshaping data structures (projections, multi-select hashes)
• Performing complex queries with conditional logic

Proposed Solution

Add query engine support with both JMESPath (for complex queries) and enhanced simple pick (for basic nested field selection) using separate functions for clarity.

See ADR 0001: Query Engine Support for detailed architectural decisions.

API Design

JMESPath Functions (Complex Queries)

import { outputJsonWithJMESPath, outputToonWithJMESPath } from '@pleaseai/cli-toolkit/output'

// Array filtering
outputJsonWithJMESPath(issues, 'items[?state==`OPEN`].{number: number, title: title}')

// Nested field access with projection
outputToonWithJMESPath(data, 'users[*].{name: name, email: email, org: organization.name}')

// Complex conditional queries
outputJsonWithJMESPath(data, 'issues[?priority>2 && state==`OPEN`]')

Simple Pick Functions (Basic Nested Access)

import { outputJsonWithPick, outputToonWithPick } from '@pleaseai/cli-toolkit/output'

// Dot notation for nested properties
outputJsonWithPick(data, ['user.name', 'metadata.created', 'assignee.login'])

// Array indexing
outputToonWithPick(data, ['items[0].id', 'items[0].title'])

Implementation Checklist

Core Implementation

• Add jmespath and @types/jmespath dependencies
• Create src/output/query/ module structure
  • src/output/query/types.ts - Type definitions
  • src/output/query/jmespath.ts - JMESPath implementation
  • src/output/query/pick.ts - Enhanced pick implementation
  • src/output/query/index.ts - Public API exports

Output Functions

• Add outputJsonWithJMESPath(data, query, fields?)
• Add outputJsonWithPick(data, paths, fields?)
• Add outputToonWithJMESPath(data, query, fields?)
• Add outputToonWithPick(data, paths, fields?)
• Update src/output/index.ts exports

Testing

• Create test/output/query/jmespath.test.ts
  • Simple field selection
  • Array filtering with conditions
  • Projections and multi-select hashes
  • Nested property access
  • Error handling for invalid queries
• Create test/output/query/pick.test.ts
  • Dot notation for nested properties
  • Array indexing (items[0].id)
  • Multiple paths
  • Error handling for invalid paths
• Create test/output/json-query.test.ts - Integration tests with JSON output
• Create test/output/toon-query.test.ts - Integration tests with TOON output
• Verify test coverage ≥ 90%

Documentation

• Update README.md with query engine usage examples
• Add JSDoc comments to all new functions
• Create migration guide from fields to queries
• Add examples for common query patterns

Package Configuration

• Update package.json exports to include query submodule:

"./output/query": {
  "types": "./dist/output/query/index.d.ts",
  "default": "./dist/output/query/index.js"
}

Success Criteria

1. ✅ JMESPath queries work correctly for field selection, array filtering, projections, and nested access
2. ✅ Simple pick works correctly for dot notation, array indexing, and multiple paths
3. ✅ Both engines work with JSON and TOON output formats
4. ✅ Backward compatibility maintained (existing API unchanged)
5. ✅ Test coverage ≥ 90% for query module
6. ✅ Clear error messages for invalid queries
7. ✅ Documentation includes examples and migration guide

Technical Details

Query Pipeline:

Raw Data → Query Engine → Field Filter → Format → Console Output

Processing Order:

1. Apply query transformation (JMESPath or Pick) if provided
2. Apply field filtering (existing fields parameter) for backward compatibility
3. Format output as JSON or TOON

Backward Compatibility:
All existing functions (outputData, outputJson, outputToon) remain unchanged. New query functions are additive only.

References

• ADR 0001: Query Engine Support
• JMESPath Specification
• JMESPath Tutorial
• AWS CLI Query Documentation

Estimated Effort

• Core implementation: ~2-3 hours
• Comprehensive testing: ~1-2 hours
• Documentation: ~1 hour
• Total: 4-6 hours