Skip to content

Commit 1add7b0

Browse files
committed
Add session database alias and COBOL copybook symbols
1 parent 68cedd8 commit 1add7b0

12 files changed

Lines changed: 303 additions & 100 deletions

‎CHANGELOG.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@ All notable changes to Token-Goat are documented in this file. Format follows Ke
77
### Changed
88

99
- The canonical skill now treats whole-file/range denials and the >80%-indexed warning as a per-file circuit breaker, directing reads and edits to surgical token-goat commands.
10+
- `sqlite-query session` resolves the active Copilot session database; COBOL outlines expose nested data items and level-88 conditions.
1011

1112
## [2.9.34] - 2026-11-02
1213

‎docs/cli.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -71,7 +71,7 @@ token-goat pdf-extract manual.pdf --pages 12-15 --layout --head 120
7171
| `token-goat openapi-op <spec> <operation>` | Full detail (parameters, request body schema, response schemas, description) for exactly one OpenAPI operation instead of a raw Read. `operation` may be an operationId (exact match) or a `"METHOD path"` spec, e.g. `"GET /users/{id}"`. |
7272
| `token-goat sqlite-tables <db> [--json]` | Ultra-compact overview of tables, views, row counts, and column counts in a SQLite database instead of a raw Read. |
7373
| `token-goat sqlite-schema <db>` | Tables/views, columns, indexes, foreign keys, and row counts of a SQLite database instead of a raw Read. |
74-
| `token-goat sqlite-query <db> "<SELECT ...>"` | Run a read-only `SELECT` against a SQLite database instead of a raw Read or shelling out to `sqlite3` — rejects any non-`SELECT` statement. |
74+
| `token-goat sqlite-query <db> "<SELECT ...>"` | Run a read-only `SELECT` against a SQLite database instead of a raw Read or shelling out to `sqlite3` — rejects any non-`SELECT` statement. Use `session` for the active Copilot session database. |
7575
| `token-goat session-schema [table] [--json]` | Authoritative schema catalog for `session_store_sql` (DuckDB/SQLite) and session SQLite tables (`todos`, `todo_deps`), complete with column types, caveats, and common column misnomer mappings (`title` → `summary`, `role` → `agent_name`), eliminating trial-and-error `SELECT *` schema exploration (85–95% smaller). |
7676
| `token-goat describe <target> [table] [--json]` | Unified database schema inspection helper: inspects any known session store table or SQLite `.db` file on disk, displaying table structure and column definitions instead of exploratory queries. |
7777
| `token-goat imports "file"` | Show the import graph for a file one level deep. Accepts a comma-separated file list (`"a,b,c"`) to cover several files in one call, one clearly-headed block per file; extra space-separated file arguments are reported in a note naming that comma form instead of being silently dropped. `--grep <pattern>` only shows imports whose MODULE SPECIFIER matches this regex (literal substring if it is not valid regex), applied before `--json`'s truncation; when it matches nothing among real imports, the output names the active filter instead of reading like the file has no imports at all. |

‎src/canonical_skill.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -61,6 +61,8 @@ Failure shapes to catch yourself in, and the command that replaces each:
6161
- persisting a small cross-call fact in conversation context → `note set/get/unset/list`
6262
Commands: `symbol NAME`, `read "file" "symbol"`, `brief "file::symbol"`, `section "file" "Heading"`, `semantic "description"`, `outline file`/`skeleton file`, `map --compact`, `refs "file::symbol" --callers`, `changed --symbol`, `config-get file KEY`, `json-query file 'a.b.c'`/`yaml-query file 'a.b.c'`/`xml-query file 'a.b.c'`, `json-outline file`/`yaml-outline file`/`xml-outline file`, `bash-output`/`web-output`/`mcp-output`, `gdrive-sections <file-id>`, `image-meta file`/`image-text file`, `pdf-meta`/`pdf-outline`/`pdf-locate`/`pdf-extract`, `docx-outline`/`docx-tables`/`docx-text`, `pptx-outline`/`pptx-slide`/`pptx-notes`/`pptx-text`, `xlsx-sheets`/`xlsx-columns`/`xlsx-head`/`xlsx-range`/`xlsx-query`, `skill-section`/`skill-body`, `session-outline`/`session-slice`, `recall`, `note set/get/unset/list`.
6363

64+
`sqlite-query session "<SELECT ...>"` queries the active Copilot session database. COBOL `.cbl`/`.cpy` files expose procedure sections/paragraphs and named level 01–49/77 items plus 88 conditions to `outline`, `symbol`, and `read`.
65+
6466
Sub-agent briefs must carry this gate verbatim: a sub-agent inherits none of this context and its reads spend the same token budget.
6567

6668
`token-goat stats` — self-check. Flat counts during code work mean the gate is being skipped.

‎src/canonical_skill.ts‎

Lines changed: 73 additions & 72 deletions
Large diffs are not rendered by default.

‎src/cli_cmd_formats.ts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -617,7 +617,7 @@ export function registerFormatCommands(program: Command, guard: GuardFn): void {
617617

618618
program
619619
.command('sqlite-query <file> <sql>')
620-
.description('run a read-only SELECT against a SQLite database instead of a raw Read or shelling out to sqlite3 -- rejects any non-SELECT statement')
620+
.description('run a read-only SELECT against a SQLite database; rejects non-SELECT statements. Use "session" for the active Copilot session database.')
621621
.option('--head <n>', 'limit to the first N returned rows')
622622
.option('--json', 'emit rows as a JSON array of objects instead of a table')
623623
.action(guard(cmdSqliteQuery))

‎src/copilot_waste.ts‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -247,6 +247,19 @@ export function findLatestCopilotSession(projectRootOrOpts?: string | FindCopilo
247247
return bestActive ?? (opts.onlyActive === true ? null : bestInactive)
248248
}
249249

250+
/** Return the database file alongside the currently active Copilot session log. */
251+
export function findActiveCopilotSessionDatabasePath(): string | null {
252+
const eventsPath = findActiveCopilotSession()
253+
if (eventsPath === null) return null
254+
255+
const databasePath = path.join(path.dirname(eventsPath), 'session.db')
256+
try {
257+
return fs.statSync(databasePath).isFile() ? databasePath : null
258+
} catch {
259+
return null
260+
}
261+
}
262+
250263
export interface DetectedSession {
251264
path: string
252265
kind: 'copilot' | 'claude'

‎src/languages/cobol.ts‎

Lines changed: 45 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
/** COBOL adapter: programs, procedure-division sections and paragraphs, level-01/77 data items and FD/SD file descriptions, leading comments, `COPY` imports, and `PERFORM`/`GO TO`/`CALL`/embedded-SQL `CALL` references. Reference format (IBM Enterprise COBOL for z/OS, https://www.ibm.com/docs/en/cobol-zos/6.4.0?topic=structure-reference-format): columns 1-6 are the sequence number area, column 7 the indicator area, 8-11 Area A, 12-72 Area B, and 73-80 are ignored. Indicator `*` or `/` makes a comment line, `D`/`d` a debugging line, `-` a continuation line (https://www.ibm.com/docs/en/cobol-zos/6.4.0?topic=format-indicator-area); `*>` starts a floating comment. Free format (`>>SOURCE FORMAT IS FREE`, the ISO 2002 directive) has no columns: text starts at column 1 and only `*>` comments. */
1+
/** COBOL adapter: programs, procedure-division sections and paragraphs, level-01/49/77 data items, level-88 conditions, FD/SD file descriptions, leading comments, `COPY` imports, and `PERFORM`/`GO TO`/`CALL`/embedded-SQL `CALL` references. Reference format (IBM Enterprise COBOL for z/OS, https://www.ibm.com/docs/en/cobol-zos/6.4.0?topic=structure-reference-format): columns 1-6 are the sequence number area, column 7 the indicator area, 8-11 Area A, 12-72 Area B, and 73-80 are ignored. Indicator `*` or `/` makes a comment line, `D`/`d` a debugging line, `-` a continuation line (https://www.ibm.com/docs/en/cobol-zos/6.4.0?topic=format-indicator-area); `*>` starts a floating comment. Free format (`>>SOURCE FORMAT IS FREE`, the ISO 2002 directive) has no columns: text starts at column 1 and only `*>` comments. */
22

33
import type { RefEntry, SymbolEntry } from '../parser_types.js'
44
import type { AdapterImport } from './common.js'
@@ -188,21 +188,17 @@ export function extractCobol(
188188
let section: number | undefined
189189
let paragraph: number | undefined
190190
let fileDescription: number | undefined
191-
let record: number | undefined
191+
const dataItems: Array<{ level: number; index: number | undefined }> = []
192+
let condition: number | undefined
192193
let division = ''
193194
let lastLine = 0
194195
let idDivisionLine: number | undefined
195196
let idDivisionDocstring: string | undefined
196197

197-
// A copybook with no division header is a fragment: data items when it opens with a level number, procedure text otherwise; its record level is the lowest level it uses.
198-
let recordLevel = 1
198+
// A divisionless copybook is a fragment; ignore leading comments and blank lines when classifying it.
199199
if (isCopybook && !lines.some((l) => DIVISION_RE.test(l.code))) {
200-
const levels = lines.map((l) => LEVEL_RE.exec(l.code)).filter((m) => m !== null).map((m) => Number(m[1]))
201-
const first = lines[0]
200+
const first = lines.find((l) => l.code.trim() !== '')
202201
division = first !== undefined && LEVEL_RE.test(first.code) ? 'DATA' : 'PROCEDURE'
203-
const structural = levels.filter((n) => n >= 1 && n <= 49)
204-
// Reduced, not spread: `structural` holds one entry per matching line with no cap (MAX_SYMBOLS gates `emit`, which runs later), so a large copybook spread into Math.min exceeds the engine's argument limit and throws out of the extractor.
205-
if (structural.length > 0) recordLevel = structural.reduce((a, b) => (b < a ? b : a), structural[0]!)
206202
}
207203

208204
const emit = (name: string, kind: string, line: number, parent: string, docstring = ''): number | undefined => {
@@ -225,9 +221,25 @@ export function extractCobol(
225221
close(section, end)
226222
section = undefined
227223
}
224+
const closeDataItems = (end: number, minLevel: number): void => {
225+
while (dataItems.length > 0) {
226+
const item = dataItems[dataItems.length - 1]!
227+
if (item.level < minLevel) break
228+
dataItems.pop()
229+
close(item.index, end)
230+
}
231+
}
232+
const dataParent = (): string => {
233+
for (let i = dataItems.length - 1; i >= 0; i--) {
234+
const index = dataItems[i]!.index
235+
if (index !== undefined) return symbols[index]!.name
236+
}
237+
return fileDescription !== undefined ? symbols[fileDescription]!.name : unitParent()
238+
}
228239
const closeData = (end: number): void => {
229-
close(record, end)
230-
record = undefined
240+
close(condition, end)
241+
condition = undefined
242+
closeDataItems(end, 0)
231243
close(fileDescription, end)
232244
fileDescription = undefined
233245
}
@@ -302,16 +314,28 @@ export function extractCobol(
302314
} else {
303315
const m = LEVEL_RE.exec(code)
304316
const level = m ? Number(m[1]) : -1
305-
if (m && (level === recordLevel || level === 77)) {
306-
close(record, lastLine)
307-
record = undefined
308-
if (level === 77) {
309-
close(fileDescription, lastLine)
310-
fileDescription = undefined
311-
}
312-
if (m[2]!.toUpperCase() !== 'FILLER') {
313-
const owner = fileDescription !== undefined ? symbols[fileDescription]!.name : unitParent()
314-
record = emit(m[2]!, 'variable', l.line, owner, l.leadingComments)
317+
if (m) {
318+
close(condition, lastLine)
319+
condition = undefined
320+
}
321+
if (m && (level === 88 || level === 77 || (level >= 1 && level <= 49))) {
322+
const name = m[2]!
323+
if (level === 88) {
324+
if (name.toUpperCase() !== 'FILLER') {
325+
condition = emit(name, 'condition', l.line, dataParent(), l.leadingComments)
326+
}
327+
} else {
328+
if (level === 77) {
329+
closeDataItems(lastLine, 0)
330+
close(fileDescription, lastLine)
331+
fileDescription = undefined
332+
} else {
333+
closeDataItems(lastLine, level)
334+
}
335+
const index = name.toUpperCase() !== 'FILLER'
336+
? emit(name, 'variable', l.line, dataParent(), l.leadingComments)
337+
: undefined
338+
dataItems.push({ level, index })
315339
}
316340
}
317341
}

‎src/read_inspect.ts‎

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,7 @@ import {
1616
} from './coverage_query.js'
1717
import { extractExportNames, extractImports, importsExtensionFor } from './import_export_extract.js'
1818
import { querySymbols } from './index_reader.js'
19+
import { findActiveCopilotSessionDatabasePath } from './copilot_waste.js'
1920
import { getNote, isNoteStale, listNotes, WHOLE_FILE_NOTE_SYMBOL } from './notes.js'
2021
import type { SymbolEntry } from './parser_types.js'
2122
import { displaySafeJson, displaySafeText, toDisplayPath } from './paths.js'
@@ -237,7 +238,14 @@ export function runSqliteQuery(opts: SqliteQueryCliOptions): number {
237238
}
238239

239240
try {
240-
const result = runReadOnlySqliteQuery(opts.file, opts.sql)
241+
const databasePath = opts.file.toLowerCase() === 'session'
242+
? findActiveCopilotSessionDatabasePath()
243+
: opts.file
244+
if (databasePath === null) {
245+
throw new Error('No active Copilot session SQLite database was found')
246+
}
247+
248+
const result = runReadOnlySqliteQuery(databasePath, opts.sql)
241249
const totalCount = result.rows.length
242250
const headTruncated = head !== undefined && result.rows.length > head
243251
const rows = head !== undefined ? result.rows.slice(0, head) : result.rows

‎tests/cobol_natural_adapters.test.ts‎

Lines changed: 39 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@ function shape(r: Extracted): string[] {
2020

2121
describe('COBOL adapter', () => {
2222
it('reads a very large copybook instead of throwing out of the extractor', () => {
23-
// HAND-DERIVED: a copybook with no division header is a fragment, so the adapter takes the lowest level number it uses. That list holds one entry per data line and has no cap of its own, and spreading a list this size into Math.min exceeds the engine's argument limit -- which threw, so the file indexed to nothing at all rather than to fewer symbols.
23+
// HAND-DERIVED: a copybook with no division header is a fragment; scanning its levels must not materialize an uncapped list before the symbol limit is applied.
2424
const lines: string[] = []
2525
for (let i = 0; i < 150_000; i++) lines.push(` 05 FIELD-${i} PIC X.`)
2626
const r = extractCobol(lines.join('\n'), 'BIG.cpy')
@@ -38,6 +38,8 @@ describe('COBOL adapter', () => {
3838
'program PAYROLL 2-27',
3939
'file EMP-FILE 7-10 PAYROLL',
4040
'variable EMP-REC 8-10 EMP-FILE',
41+
'variable EMP-ID 9-9 EMP-REC',
42+
'variable EMP-PAY 10-10 EMP-REC',
4143
'variable WS-TOTAL 12-12 PAYROLL',
4244
'variable WS-COUNT 13-13 PAYROLL',
4345
'paragraph MAIN-PARA 17-21 PAYROLL',
@@ -103,9 +105,41 @@ describe('COBOL adapter', () => {
103105
expect(r.refs.map((x) => x.name)).toEqual(['inner-para', 'SUBPGM'])
104106
})
105107

106-
it('reads a copybook with no division as record-level data items', () => {
107-
const r = extractCobol([' 01 CUST-REC.', ' 05 CUST-ID PIC 9(5).', ' 01 CUST-ADDR.', ' 05 STREET PIC X(30).'].join('\n'), 'CUST.cpy')
108-
expect(shape(r)).toEqual(['variable CUST-REC 1-2', 'variable CUST-ADDR 3-4'])
108+
it('reads nested data items and 88-level conditions after leading comments in a copybook', () => {
109+
const r = extractCobol([
110+
'*> Customer copybook',
111+
'',
112+
'01 CUST-REC.',
113+
' 05 CUST-ID PIC 9(5).',
114+
' 05 CUST-STATUS PIC X.',
115+
' 88 CUST-ACTIVE VALUE "A".',
116+
' 88 CUST-INACTIVE VALUE "I".',
117+
'01 CUST-ADDR.',
118+
' 05 STREET PIC X(30).',
119+
].join('\n'), 'CUST.cpy')
120+
expect(shape(r)).toEqual([
121+
'variable CUST-REC 3-7',
122+
'variable CUST-ID 4-4 CUST-REC',
123+
'variable CUST-STATUS 5-7 CUST-REC',
124+
'condition CUST-ACTIVE 6-6 CUST-STATUS',
125+
'condition CUST-INACTIVE 7-7 CUST-STATUS',
126+
'variable CUST-ADDR 8-9',
127+
'variable STREET 9-9 CUST-ADDR',
128+
])
129+
})
130+
131+
it('closes 88-level conditions before level-66 and level-78 entries', () => {
132+
const r = extractCobol([
133+
'*> Data item boundary test',
134+
'01 CUST-REC.',
135+
' 05 CUST-STATUS PIC X.',
136+
' 88 CUST-ACTIVE VALUE "A".',
137+
' 66 CUST-STATUS-ALIAS RENAMES CUST-STATUS.',
138+
' 78 CUST-UNKNOWN VALUE 1.',
139+
' 05 CUST-NEXT PIC X.',
140+
].join('\n'), 'CUST.cpy')
141+
142+
expect(shape(r)).toContain('condition CUST-ACTIVE 4-4 CUST-STATUS')
109143
})
110144

111145
it('attaches leading comments to programs, sections, paragraphs and copybook records', () => {
@@ -137,6 +171,7 @@ describe('COBOL adapter', () => {
137171
const copybook = extractCobol(['*> Customer copybook', '01 CUSTOMER-REC.', ' 05 CUSTOMER-ID PIC X(8).'].join('\n'), `CUST.${extension}`)
138172
expect(copybook.symbols.map(({ name, docstring }) => [name, docstring])).toEqual([
139173
['CUSTOMER-REC', 'Customer copybook'],
174+
['CUSTOMER-ID', ''],
140175
])
141176
}
142177
})

‎tests/copilot_waste.test.ts‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@ import { join } from 'node:path'
66
import {
77
buildCopilotWasteReport,
88
findActiveCopilotSession,
9+
findActiveCopilotSessionDatabasePath,
910
findLatestCopilotSession,
1011
findProjectSession,
1112
isCopilotSessionActive,
@@ -261,6 +262,34 @@ describe('findLatestCopilotSession', () => {
261262
expect(findActiveCopilotSession(project)).toBe(olderActive)
262263
})
263264

265+
it('finds the active session database beside its event log', () => {
266+
const activeId = 'tg-active-db'
267+
const inactiveId = 'tg-inactive-db'
268+
const activeDir = join(root, 'session-state', activeId)
269+
const inactiveDir = join(root, 'session-state', inactiveId)
270+
session(activeId, 1_000_000)
271+
session(inactiveId, 2_000_000)
272+
const activeDb = join(activeDir, 'session.db')
273+
writeFileSync(activeDb, '')
274+
writeFileSync(join(activeDir, `inuse.${process.pid}.lock`), String(process.pid), 'utf-8')
275+
writeFileSync(join(inactiveDir, 'session.db'), '')
276+
277+
expect(findActiveCopilotSessionDatabasePath()).toBe(activeDb)
278+
})
279+
280+
it('does not fall back to an inactive session database when the active session has none', () => {
281+
const activeId = 'tg-active-without-db'
282+
const inactiveId = 'tg-inactive-with-db'
283+
const activeDir = join(root, 'session-state', activeId)
284+
const inactiveDir = join(root, 'session-state', inactiveId)
285+
session(activeId, 1_000_000)
286+
session(inactiveId, 2_000_000)
287+
writeFileSync(join(activeDir, `inuse.${process.pid}.lock`), String(process.pid), 'utf-8')
288+
writeFileSync(join(inactiveDir, 'session.db'), '')
289+
290+
expect(findActiveCopilotSessionDatabasePath()).toBeNull()
291+
})
292+
264293
it('honors COPILOT_AGENT_SESSION_ID when projectRoot matches', () => {
265294
const project = join(root, 'env-proj')
266295
mkdirSync(project, { recursive: true })

0 commit comments

Comments
 (0)