Repository navigation
Phase 0: establish foundational research contract - #1
Conversation
Reviewer's GuideThis PR establishes Phase 0 as a machine-checkable, falsifiable research contract: it separates hypotheses and motivation from evidence, defines methodology and safeguards for future experiments, documents five initial research tracks and the full roadmap, and enforces contract completeness through dependency-free tests and GitHub Actions CI. Flow diagram for the Phase 0 research contractflowchart TD
A[Human motivation] --> B[ falsifiable hypothesis ]
B --> C[Freeze task contract]
C --> D[Run controlled trials]
D --> E[Independent verification]
E --> F[Retain successes and failures]
F --> G[Bounded conclusion]
G --> H[Supported, weakened, or falsified]
A -. not evidence .-> H
Flow diagram for the CONSTRAINT-SHIFT research roadmapflowchart LR
P0[Phase 0: Research contract] --> P1[Phase 1: Experiment schema]
P1 --> P2[Phase 2: Language harness]
P2 --> P3[Phase 3: Compiler feedback]
P3 --> P4[Phase 4: Legacy preservation]
P4 --> P5[Phase 5: Implementation fungibility]
P5 --> P6[Phase 6: Modding and recombination]
P6 --> P7[Phase 7: Tool legitimacy]
P7 --> P8[Phase 8: Language fitness model]
P8 --> P9[Phase 9: Replication and sensitivity]
P9 --> P10[Phase 10: Archival release]
File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
There was a problem hiding this comment.
Hey - I've found 4 issues
Prompt for AI Agents
Please address the comments from this code review:
## Individual Comments
### Comment 1
<location path="research/tool-legitimacy-gap.md" line_range="12" />
<code_context>
+## Core design principle
+When testing disclosure, the artifact must remain constant.
+
+A minimal randomized design could expose participants to an identical artifact with descriptions such as independent human creator, independent creator using AI-assisted development tools, or primarily AI-generated under human direction.
+
+## Candidate outcomes
</code_context>
<issue_to_address>
**Disclosure effect cannot be isolated**
When the proposed descriptions are used to evaluate H6 or estimate `D_AI`, the conditions vary both disclosure and the stated degree of AI authorship, and none provides an undisclosed baseline. Their ratings therefore cannot isolate the disclosure effect or estimate the stated penalty.
Use conditions that vary disclosure while holding the artifact and stated authorship constant, and include an undisclosed condition for the `D_AI` contrast.
Also at `research/tool-legitimacy-gap.md:25-27`, `research/tool-legitimacy-gap.md:32`.
</issue_to_address>
### Comment 2
<location path="research/language-selection.md" line_range="18" />
<code_context>
+Phase 0 rejects a premature universal score. A language/toolchain observation should initially preserve separate measured dimensions. Definitions and measurement procedures must precede aggregation.
+
+## Candidate experiment
+Give one agent/model an equivalent task contract across several languages.
+
+Record first-generation validity, compile attempts, diagnostic cycles, test failures, repair iterations, human interventions, completion outcome, static/formal check outcomes, and time/token/cost data where reliable.
</code_context>
<issue_to_address>
**Language comparison cannot isolate diagnostics**
When cross-language outcomes from the proposed experiment are used to support or weaken H2, the proposed comparison varies languages and toolchains while measuring outcomes, but does not independently vary diagnostic quality or specify how to control the listed confounds. Differences in outcomes therefore cannot establish that diagnostics or constraints caused an advantage, which is the claim in H2.
Specify a design that isolates diagnostic and constraint effects, such as controlled conditions within toolchains, and state how the listed confounds will be matched or controlled.
</issue_to_address>
### Comment 3
<location path="scripts/validate_phase0.py" line_range="34" />
<code_context>
+ "H6 — Tool Legitimacy Gap",
+)
+
+TERMS = (
+ "Constraint Shift",
+ "Implementation Fungibility",
</code_context>
<issue_to_address>
**Missing terms pass validation**
When a terminology definition omitted from `TERMS` is removed, `validate_repo()` checks only the headings listed in `TERMS`, so removing Specification Primacy, Machine Verifiability, Diagnostic Feedback Quality, or Research Contract still passes validation and CI reports a valid contract.
Add every contractually required terminology heading to `TERMS`.
Also at `scripts/validate_phase0.py:35-42`, `scripts/validate_phase0.py:74-76`.
</issue_to_address>
### Comment 4
<location path="scripts/validate_phase0.py" line_range="70-72" />
<code_context>
+ if errors:
+ return errors
+
+ for identifier in HYPOTHESES:
+ if identifier not in texts["HYPOTHESES.md"]:
+ errors.append(f"missing hypothesis contract: {identifier}")
+
+ for term in TERMS:
</code_context>
<issue_to_address>
**Duplicate hypothesis headings pass**
When a hypothesis heading is duplicated while all six expected identifiers remain present, `validate_repo()` checks only whether each identifier appears somewhere, and the uniqueness test checks the constant tuple rather than the Markdown headings. The duplicate therefore passes CI and leaves an ambiguous research contract.
Parse the hypothesis headings from `HYPOTHESES.md` and validate their expected contents and uniqueness.
Also at `tests/test_phase0.py:18-19`, `scripts/validate_phase0.py:35-42`.
</issue_to_address>Sourcery assessment
Approval pending. 4 findings to address first.
Blocking findings: research/tool-legitimacy-gap.md:12, research/language-selection.md:18, scripts/validate_phase0.py:34, scripts/validate_phase0.py:72
|
@codex Please review this exact SHA |
|
@sourcery-ai review Please review this exact SHA |
|
@codex review Please review exact SHA |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 9ea5418c26
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
@codex review Please review exact SHA |
|
@sourcery-ai review Please review exact SHA |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 57e8517092
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
@codex review Please review exact SHA |
|
@sourcery-ai review Please review exact SHA |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 8fd0699335
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Sourcery withdrew this approval because it has stopped reviewing this pull request.
|
Sourcery has withdrawn its approval of this pull request. It auto-reviews a pull request 5 times, and this push is past that limit, so the approval no longer reflects code Sourcery has read. Comment |
|
@codex review Please review exact SHA |
|
@sourcery-ai review Please review exact SHA |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 1e1fa9cefa
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
@codex review Please review exact SHA |
|
@codex Please Review this exact SHA b0e1e79 against the existing contract. Report only actionable correctness defects, with a minimal reproduction, expected versus actual behavior, and affected lines. State whether each reproduction was executed or statically inferred. Don’t repeat fixed findings without a new failing case. Keep architectural suggestions separate and non-blocking. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b0e1e79666
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
@codex Please Review this exact SHA cb405dd against the existing contract. Report only actionable correctness defects, with a minimal reproduction, expected versus actual behavior, and affected lines. State whether each reproduction was executed or statically inferred. Don’t repeat fixed findings without a new failing case. Keep architectural suggestions separate and non-blocking. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: cb405ddd22
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
@codex Please Review this exact SHA b8aacc5 against the existing contract. Report only actionable correctness defects, with a minimal reproduction, expected versus actual behavior, and affected lines. State whether each reproduction was executed or statically inferred. Don’t repeat fixed findings without a new failing case. Keep architectural suggestions separate and non-blocking. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b8aacc51b7
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
@codex Please Review this exact SHA 3ea0ba8 against the existing contract. Report only actionable correctness defects, with a minimal reproduction, expected versus actual behavior, and affected lines. State whether each reproduction was executed or statically inferred. Don’t repeat fixed findings without a new failing case. Keep architectural suggestions separate and non-blocking. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 3ea0ba8edb
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
@codex Please Review this exact SHA 75c74db against the existing contract. Report only actionable correctness defects, with a minimal reproduction, expected versus actual behavior, and affected lines. State whether each reproduction was executed or statically inferred. Don’t repeat fixed findings without a new failing case. Keep architectural suggestions separate and non-blocking. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 75c74db17a
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
@codex Please Review this exact SHA 778105b against the existing contract. Report only actionable correctness defects, with a minimal reproduction, expected versus actual behavior, and affected lines. State whether each reproduction was executed or statically inferred. Don’t repeat fixed findings without a new failing case. Keep architectural suggestions separate and non-blocking. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 778105b309
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
@codex Please Review this exact SHA f5b9b2f against the existing contract. Report only actionable correctness defects, with a minimal reproduction, expected versus actual behavior, and affected lines. State whether each reproduction was executed or statically inferred. Don’t repeat fixed findings without a new failing case. Keep architectural suggestions separate and non-blocking. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f5b9b2f47c
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
@codex Please Review this exact SHA 44c9e0c against the existing contract. Report only actionable correctness defects, with a minimal reproduction, expected versus actual behavior, and affected lines. State whether each reproduction was executed or statically inferred. Don’t repeat fixed findings without a new failing case. Keep architectural suggestions separate and non-blocking. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 44c9e0cf91
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| visible_line, next_comment_state = _mask_html_comments( | ||
| code_safe_line, in_html_comment | ||
| ) | ||
| if in_html_comment or next_comment_state: | ||
| in_html_comment = next_comment_state | ||
| paragraph_open = False |
There was a problem hiding this comment.
Executed reproduction: after removing the roadmap's original required phrase, append [visible][machine-checkable Phase 0 validator]\n\n> <!--\n[machine-checkable Phase 0 validator]: /url; validate_texts() returns []. Expected the missing-validator error because the unclosed comment belongs to the block quote, ends when the definition line exits that quote, and therefore does not prevent the top-level definition from resolving the reference. Actual reference discovery carries in_html_comment across the container exit and skips the definition; this is a new failing path in the reference-label scanner, separate from the visible-line comment handling.
Useful? React with 👍 / 👎.
| def _normalize_rendered_visible_text(text: str) -> str: | ||
| text = html.unescape(text) |
There was a problem hiding this comment.
Executed reproduction: replace the roadmap's required phrase with machine-checkable Phase 0 validator; validate_texts() returns []. Expected roadmap does not require Phase 0 validator because the numeric character reference lacks its required semicolon, so Markdown renders it literally and the contiguous required phrase is absent. Actual html.unescape() applies permissive HTML parsing and converts - to -, manufacturing the phrase.
Useful? React with 👍 / 👎.
| text = re.sub( | ||
| r"(?<!\\)(\*\*|__)(?=\S)(.+?)(?<=\S)\1", | ||
| r"\2", |
There was a problem hiding this comment.
Executed reproduction: replace the roadmap's required phrase with machine__-checkable__ Phase 0 validator; validate_texts() returns []. Expected the missing-validator error because the first intraword underscore run cannot open emphasis under Markdown's underscore-flanking rules, leaving the underscores rendered and the required phrase noncontiguous. Actual normalization removes any paired __ around non-whitespace, producing machine-checkable Phase 0 validator regardless of delimiter eligibility.
Useful? React with 👍 / 👎.
| comment_allows_lazy_continuation = ( | ||
| paragraph_open | ||
| or bool(visible_comment_prefix.strip(" \t")) | ||
| ) |
There was a problem hiding this comment.
Executed reproduction: after removing the roadmap's original required phrase, append > ## heading <!--\nmachine-checkable Phase 0 validator -->; validate_texts() reports roadmap does not require Phase 0 validator. Expected no error because an ATX heading is a single-line leaf block, so the unquoted second line cannot lazily continue the quoted heading and its required phrase remains visible. Actual comment eligibility treats any nonempty prefix—including ## heading—as a paragraph that permits lazy continuation, so the second line is incorrectly masked.
Useful? React with 👍 / 👎.
Derive headings, references, ownership and prose from one token stream. Retain upstream rules with explicit reference-length and whitespace corrections. Add review regressions, official corpus projections, CLI and performance checks. Pin and hash dependencies; validate on Python 3.12 and 3.14.
|
@codex Please Review this exact SHA c0946e4 against the existing contract. Report only actionable correctness defects, with a minimal reproduction, expected versus actual behavior, and affected lines. State whether each reproduction was executed or statically inferred. Don’t repeat fixed findings without a new failing case. Keep architectural suggestions separate and non-blocking. |
|
Codex Review: Didn't find any major issues. More of your lovely PRs please. Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
1 participant
Summary
Establishes Phase 0 of CONSTRAINT-SHIFT as a falsifiable, machine-checkable research contract.
Research foundation
Validation
Headings, prose, resolved references, and quote/list ownership now come from one CommonMark parse per required file. This removes the separate handwritten visibility, reference, code-span, HTML, and prose scanners that produced recurring inconsistent results.
For example, a reference definition after an exited quote-scoped comment now resolves correctly; malformed numeric entities and intraword underscores cannot manufacture required prose. A quoted ATX heading cannot extend its inline comment into an unquoted paragraph.
markdown-it-py==4.2.0andmdurl==0.1.2.Validation
Clean installations on Python 3.12.14 and 3.14.7: repository validator passes, all 215 tests pass, and all 18 separately supplied review tests pass.
git diff --checkpasses.See validation policy for behavior changes, parser limitations, and corpus attribution. The official corpus comparison normalizes only newline formatting in empty blockquotes; full CommonMark conformance is not claimed.
Motivation is not evidence: none of the six hypotheses is considered supported merely because it is documented here.