datasets: add 149 contrastive Turkish prompt-injection cases

[3nesdeniz]

Summary

• adds 149 Turkish-first, standalone prompt-injection and data-extraction regression cases
• keeps each adversarial prompt with a vocabulary- and context-matched benign control so evaluators can measure both attack containment and false-positive behavior
• includes a strict extension schema, deterministic builder, SHA-256 manifest, pinned provenance, and an executable validator
• preserves the source release text byte-for-byte; one malformed reversed-payload pair (pair_0140 / TC-0440) is transparently excluded rather than silently corrected

Source and licensing

The cases are adapted from the Turkish Conversation Prompt-Injection Dataset v1.0.2, also published on Hugging Face with DOI 10.5281/zenodo.21379389.

Source attack/control text remains under CC BY 4.0. New mappings, expectations, metadata, scripts, and collection arrangement are contributed under this repository's CC BY-SA 4.0 license. Attribution, source revision, release tag, per-file hashes, and adaptation changes are recorded in the suite README, manifest, and case provenance.

Coverage

• 149 paired cases across 10 attack families (nine families × 15; obfuscation/code-switching × 14 after the documented exclusion)
• 85 data-extraction cases and 64 prompt-injection-control cases
• Turkish-first coverage: 147 Turkish-primary and 2 English-primary attacks; both English-primary cases retain Turkish controls
• DSGAI mappings across DSGAI01, 02, 04, 05, 06, 09, 11, 13, 15, and 17
• all systems, credentials, tools, and protected records are synthetic or mocked

Validation

• python3 build_cases.py --source <pinned-v1.0.2-checkout>: 149 cases; two consecutive builds produced identical hashes for all 149 cases plus the manifest
• python3 validate.py --source <pinned-v1.0.2-checkout>: 149/149 schema, provenance, source-text, manifest, mapping, safety, and integrity checks passed
• source dataset validator: 750/750 rows passed; published JSONL and Parquet checksums verified
• upstream test suite: 27 passed, 1 skipped
• exact attack/control collisions: 0
• exact overlap with existing repository prompts: 0
• sensitive-data, live-secret, and named-vendor findings: 0
• maximum unrelated lexical similarity remained below the 0.92 failure threshold

Both commits are SSH-signed and verified by GitHub. The contribution is ready for maintainer review.

[Copilot]

Pull request overview

This PR expands the prompt-injection/data-extraction regression dataset by adding a new Turkish-first contrastive suite derived from the Turkish Conversation Prompt-Injection Dataset v1.0.2, pairing each adversarial prompt with a vocabulary-matched benign control and including provenance + integrity metadata for reproducible evaluation.

Changes:

• Added new standalone JSON test cases for the tr_turkish_conversation_prompt_injection_pairs suite, including contrastive controls, provenance, and SHA-256 integrity fields.
• Updated the top-level dataset README to reference the canonical schema and enumerate current suites.

Reviewed changes

Copilot reviewed 155 out of 155 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
datasets/promptinj_dataextraction_testcases/tr_turkish_conversation_prompt_injection_pairs/cases/TC-0449.json	Adds a new contrastive obfuscation/code-switching control testcase (TC-0449).
datasets/promptinj_dataextraction_testcases/tr_turkish_conversation_prompt_injection_pairs/cases/TC-0447.json	Adds a new contrastive obfuscation/code-switching control testcase (TC-0447).
datasets/promptinj_dataextraction_testcases/tr_turkish_conversation_prompt_injection_pairs/cases/TC-0444.json	Adds a new data-extraction testcase with matched benign control (TC-0444).
datasets/promptinj_dataextraction_testcases/tr_turkish_conversation_prompt_injection_pairs/cases/TC-0435.json	Adds a new memory-context poisoning control testcase (TC-0435).
datasets/promptinj_dataextraction_testcases/tr_turkish_conversation_prompt_injection_pairs/cases/TC-0432.json	Adds a new memory-context poisoning control testcase (TC-0432).
datasets/promptinj_dataextraction_testcases/tr_turkish_conversation_prompt_injection_pairs/cases/TC-0428.json	Adds a new memory-context poisoning control testcase (TC-0428).
datasets/promptinj_dataextraction_testcases/tr_turkish_conversation_prompt_injection_pairs/cases/TC-0426.json	Adds a new memory-context poisoning control testcase (TC-0426).
datasets/promptinj_dataextraction_testcases/tr_turkish_conversation_prompt_injection_pairs/cases/TC-0425.json	Adds a new memory-context poisoning control testcase (TC-0425).
datasets/promptinj_dataextraction_testcases/tr_turkish_conversation_prompt_injection_pairs/cases/TC-0424.json	Adds a new memory-context poisoning control testcase (TC-0424).
datasets/promptinj_dataextraction_testcases/tr_turkish_conversation_prompt_injection_pairs/cases/TC-0423.json	Adds a new memory-context poisoning data-extraction testcase (TC-0423).
datasets/promptinj_dataextraction_testcases/tr_turkish_conversation_prompt_injection_pairs/cases/TC-0390.json	Adds a new tool-action abuse control testcase (TC-0390).
datasets/promptinj_dataextraction_testcases/tr_turkish_conversation_prompt_injection_pairs/cases/TC-0389.json	Adds a new tool-action abuse control testcase (TC-0389).
datasets/promptinj_dataextraction_testcases/tr_turkish_conversation_prompt_injection_pairs/cases/TC-0388.json	Adds a new tool-action abuse control testcase (TC-0388).
datasets/promptinj_dataextraction_testcases/tr_turkish_conversation_prompt_injection_pairs/cases/TC-0387.json	Adds a new tool-action abuse control testcase (TC-0387).
datasets/promptinj_dataextraction_testcases/tr_turkish_conversation_prompt_injection_pairs/cases/TC-0384.json	Adds a new tool-action abuse control testcase (TC-0384).
datasets/promptinj_dataextraction_testcases/tr_turkish_conversation_prompt_injection_pairs/cases/TC-0380.json	Adds a new tool-action abuse control testcase (TC-0380).
datasets/promptinj_dataextraction_testcases/tr_turkish_conversation_prompt_injection_pairs/cases/TC-0376.json	Adds a new tool-action abuse control testcase (TC-0376).
datasets/promptinj_dataextraction_testcases/tr_turkish_conversation_prompt_injection_pairs/cases/TC-0375.json	Adds a new sensitive-data exfiltration testcase (TC-0375).
datasets/promptinj_dataextraction_testcases/tr_turkish_conversation_prompt_injection_pairs/cases/TC-0374.json	Adds a new sensitive-data exfiltration testcase (TC-0374).
datasets/promptinj_dataextraction_testcases/tr_turkish_conversation_prompt_injection_pairs/cases/TC-0373.json	Adds a new sensitive-data exfiltration testcase (TC-0373).
datasets/promptinj_dataextraction_testcases/tr_turkish_conversation_prompt_injection_pairs/cases/TC-0370.json	Adds a new sensitive-data exfiltration testcase (TC-0370).
datasets/promptinj_dataextraction_testcases/tr_turkish_conversation_prompt_injection_pairs/cases/TC-0369.json	Adds a new sensitive-data exfiltration testcase (TC-0369).
datasets/promptinj_dataextraction_testcases/tr_turkish_conversation_prompt_injection_pairs/cases/TC-0367.json	Adds a new sensitive-data exfiltration testcase (TC-0367).
datasets/promptinj_dataextraction_testcases/tr_turkish_conversation_prompt_injection_pairs/cases/TC-0364.json	Adds a new sensitive-data exfiltration testcase (TC-0364).
datasets/promptinj_dataextraction_testcases/tr_turkish_conversation_prompt_injection_pairs/cases/TC-0363.json	Adds a new sensitive-data exfiltration testcase (TC-0363).
datasets/promptinj_dataextraction_testcases/tr_turkish_conversation_prompt_injection_pairs/cases/TC-0362.json	Adds a new sensitive-data exfiltration testcase (TC-0362).
datasets/promptinj_dataextraction_testcases/tr_turkish_conversation_prompt_injection_pairs/cases/TC-0361.json	Adds a new sensitive-data exfiltration testcase (TC-0361).
datasets/promptinj_dataextraction_testcases/tr_turkish_conversation_prompt_injection_pairs/cases/TC-0356.json	Adds a new authority-claim bypass control testcase (TC-0356).
datasets/promptinj_dataextraction_testcases/tr_turkish_conversation_prompt_injection_pairs/cases/TC-0355.json	Adds a new authority-claim bypass data-extraction testcase (TC-0355).
datasets/promptinj_dataextraction_testcases/tr_turkish_conversation_prompt_injection_pairs/cases/TC-0354.json	Adds a new authority-claim bypass data-extraction testcase (TC-0354).
datasets/promptinj_dataextraction_testcases/tr_turkish_conversation_prompt_injection_pairs/cases/TC-0350.json	Adds a new authority-claim bypass control testcase (TC-0350).
datasets/promptinj_dataextraction_testcases/tr_turkish_conversation_prompt_injection_pairs/cases/TC-0349.json	Adds a new authority-claim bypass data-extraction testcase (TC-0349).
datasets/promptinj_dataextraction_testcases/tr_turkish_conversation_prompt_injection_pairs/cases/TC-0347.json	Adds a new authority-claim bypass data-extraction testcase (TC-0347).
datasets/promptinj_dataextraction_testcases/tr_turkish_conversation_prompt_injection_pairs/cases/TC-0345.json	Adds a new roleplay jailbreak data-extraction testcase (TC-0345).
datasets/promptinj_dataextraction_testcases/tr_turkish_conversation_prompt_injection_pairs/cases/TC-0344.json	Adds a new roleplay jailbreak control testcase (TC-0344).
datasets/promptinj_dataextraction_testcases/tr_turkish_conversation_prompt_injection_pairs/cases/TC-0343.json	Adds a new roleplay jailbreak data-extraction testcase (TC-0343).
datasets/promptinj_dataextraction_testcases/tr_turkish_conversation_prompt_injection_pairs/cases/TC-0342.json	Adds a new roleplay jailbreak control testcase (TC-0342).
datasets/promptinj_dataextraction_testcases/tr_turkish_conversation_prompt_injection_pairs/cases/TC-0340.json	Adds a new roleplay jailbreak control testcase (TC-0340).
datasets/promptinj_dataextraction_testcases/tr_turkish_conversation_prompt_injection_pairs/cases/TC-0336.json	Adds a new roleplay jailbreak data-extraction testcase (TC-0336).
datasets/promptinj_dataextraction_testcases/tr_turkish_conversation_prompt_injection_pairs/cases/TC-0335.json	Adds a new roleplay jailbreak data-extraction testcase (TC-0335).
datasets/promptinj_dataextraction_testcases/tr_turkish_conversation_prompt_injection_pairs/cases/TC-0334.json	Adds a new roleplay jailbreak control testcase (TC-0334).
datasets/promptinj_dataextraction_testcases/tr_turkish_conversation_prompt_injection_pairs/cases/TC-0331.json	Adds a new roleplay jailbreak control testcase (TC-0331).
datasets/promptinj_dataextraction_testcases/tr_turkish_conversation_prompt_injection_pairs/cases/TC-0330.json	Adds a new system-prompt extraction testcase (TC-0330).
datasets/promptinj_dataextraction_testcases/tr_turkish_conversation_prompt_injection_pairs/cases/TC-0329.json	Adds a new system-prompt extraction testcase (TC-0329).
datasets/promptinj_dataextraction_testcases/tr_turkish_conversation_prompt_injection_pairs/cases/TC-0328.json	Adds a new system-prompt extraction testcase (TC-0328).
datasets/promptinj_dataextraction_testcases/tr_turkish_conversation_prompt_injection_pairs/cases/TC-0327.json	Adds a new system-prompt extraction testcase (TC-0327).
datasets/promptinj_dataextraction_testcases/tr_turkish_conversation_prompt_injection_pairs/cases/TC-0325.json	Adds a new system-prompt extraction testcase (TC-0325).
datasets/promptinj_dataextraction_testcases/tr_turkish_conversation_prompt_injection_pairs/cases/TC-0324.json	Adds a new system-prompt extraction testcase (TC-0324).
datasets/promptinj_dataextraction_testcases/tr_turkish_conversation_prompt_injection_pairs/cases/TC-0323.json	Adds a new system-prompt extraction testcase (TC-0323).
datasets/promptinj_dataextraction_testcases/tr_turkish_conversation_prompt_injection_pairs/cases/TC-0322.json	Adds a new system-prompt extraction testcase (TC-0322).
datasets/promptinj_dataextraction_testcases/tr_turkish_conversation_prompt_injection_pairs/cases/TC-0321.json	Adds a new system-prompt extraction testcase (TC-0321).
datasets/promptinj_dataextraction_testcases/tr_turkish_conversation_prompt_injection_pairs/cases/TC-0320.json	Adds a new system-prompt extraction testcase (TC-0320).
datasets/promptinj_dataextraction_testcases/tr_turkish_conversation_prompt_injection_pairs/cases/TC-0319.json	Adds a new system-prompt extraction testcase (TC-0319).
datasets/promptinj_dataextraction_testcases/tr_turkish_conversation_prompt_injection_pairs/cases/TC-0318.json	Adds a new system-prompt extraction testcase (TC-0318).
datasets/promptinj_dataextraction_testcases/tr_turkish_conversation_prompt_injection_pairs/cases/TC-0317.json	Adds a new system-prompt extraction testcase (TC-0317).
datasets/promptinj_dataextraction_testcases/tr_turkish_conversation_prompt_injection_pairs/cases/TC-0316.json	Adds a new system-prompt extraction testcase (TC-0316).
datasets/promptinj_dataextraction_testcases/tr_turkish_conversation_prompt_injection_pairs/cases/TC-0315.json	Adds a new direct-instruction override control testcase (TC-0315).
datasets/promptinj_dataextraction_testcases/tr_turkish_conversation_prompt_injection_pairs/cases/TC-0314.json	Adds a new direct-instruction override control testcase (TC-0314).
datasets/promptinj_dataextraction_testcases/tr_turkish_conversation_prompt_injection_pairs/cases/TC-0311.json	Adds a new direct-instruction override control testcase (TC-0311).
datasets/promptinj_dataextraction_testcases/tr_turkish_conversation_prompt_injection_pairs/cases/TC-0309.json	Adds a new direct-instruction override control testcase (TC-0309).
datasets/promptinj_dataextraction_testcases/tr_turkish_conversation_prompt_injection_pairs/cases/TC-0308.json	Adds a new direct-instruction override control testcase (TC-0308).
datasets/promptinj_dataextraction_testcases/tr_turkish_conversation_prompt_injection_pairs/cases/TC-0307.json	Adds a new direct-instruction override control testcase (TC-0307).
datasets/promptinj_dataextraction_testcases/tr_turkish_conversation_prompt_injection_pairs/cases/TC-0306.json	Adds a new direct-instruction override control testcase (TC-0306).
datasets/promptinj_dataextraction_testcases/tr_turkish_conversation_prompt_injection_pairs/cases/TC-0305.json	Adds a new direct-instruction override control testcase (TC-0305).
datasets/promptinj_dataextraction_testcases/tr_turkish_conversation_prompt_injection_pairs/cases/TC-0304.json	Adds a new direct-instruction override control testcase (TC-0304).
datasets/promptinj_dataextraction_testcases/tr_turkish_conversation_prompt_injection_pairs/cases/TC-0303.json	Adds a new direct-instruction override control testcase (TC-0303).
datasets/promptinj_dataextraction_testcases/README.md	Documents the canonical schema and lists available suites.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[3nesdeniz]

The branch is now synchronized with current main, the schema/category guidance thread is addressed and resolved, and the hosted Dependency Review check is green. Fresh local validation on head 1f246a2: 149/149 cases passed the canonical and extension schemas plus pinned source/provenance/integrity checks; the deterministic rebuild produced no diff; all 6 data-validation unit tests and Ruff checks passed. Ready for maintainer review.

[3nesdeniz]

Hi @emmanuelgjr — could you take a look at this dataset contribution when convenient? The validated head above remains current and mergeable, and the hosted Dependency Review is green. I’m happy to address any dataset-level feedback.

[3nesdeniz]

Hi @emmanuelgjr — following up once more on this dataset contribution. The validated head is still current and mergeable with the hosted Dependency Review green. If there's anything blocking review, I'm happy to address it.

[emmanuelgjr]

Thank you for this contribution. The cases are high quality: all 149 validate, the attack/control pairing is well done, and the IDs don't collide with anything on main. Before we can merge, we need changes under this repository's vendor-neutrality policy.

Policy: this is a global, vendor-neutral OWASP repository. Merged files may credit a contributor by name only, and license-required attribution is kept to the minimum: author name, title, license and one canonical identifier (the DOI). Merged files may not carry company affiliations or repeated links to a contributor's own repositories or distributions.

Requested changes

1. Remove the affiliation. Delete the | Affiliation | AltaySec | row from the suite README, and remove the validate.py check that fails when that row is missing.
2. Reduce per-case source links to one citation. Every case currently carries source_repository and source_distribution_url pointing at 3nesdeniz/..., plus the URL again inside source (about 450 links in total). Please replace them with a single suite-level citation in the README: Enes Deniz, "Turkish Conversation Prompt-Injection Dataset", v1.0.2, CC BY 4.0, doi:10.5281/zenodo.21379389. In each case, keep only the pair ID, release version and license needed for provenance. The const URL checks in the extension schema and validate.py would go too.
3. Remove the copyright-string assertions (expected = "| Attribution | Enes Deniz, Copyright © 2026 |", readme.count("Enes Deniz") != 1). The CC BY attribution line above is enough, and a dataset validator shouldn't enforce authorship text.
4. Add a .gitattributes that forces LF line endings for this folder. validate.py's hash checks currently fail on Windows checkouts because of CRLF conversion.
5. Optional, but it would help reviewers: consider moving build_cases.py/validate.py out of the dataset folder, or dropping them. No other dataset suite commits builder code, and the pinned-source rebuild can live in your upstream repo.

Two smaller notes on the data: DSGAI05 is applied to 89 cases as a catch-all for instruction-override attacks, and 64 cases use category: "other". Please tighten those where a more specific mapping fits.

Happy to re-review once this is updated.

[emmanuelgjr]

Rather than wait on the change-request round trip, we've applied the requested changes on this branch (maintainer edits), since they were mechanical and your builder made them safe to do:

• Attribution reduced to the license minimum, stated once in the suite README: author name, title, release, license, and the canonical DOI (10.5281/zenodo.21379389). The ~450 repository/distribution links, the affiliation row, and the copyright-string validator checks are gone; the per-case source is now "… pair pair_NNNN; CC BY 4.0" and per-case provenance keeps the DOI, pinned revision, and all content hashes — nothing about reproducibility is lost.
• .gitattributes forcing LF added, so validate.py's hash checks now hold on Windows checkouts (they failed with core.autocrlf on before).
• All 149 cases and the manifest were regenerated through your own build_cases.py against the pinned v1.0.2 source (9a2163b). Attack and control text are byte-for-byte unchanged; the only per-case diff is the source string and the two removed URL fields.

Re-verified after the edits: validate.py --source <pinned checkout> passes 149/149 (source text, pair identity, split, hashes, similarity, sensitive-data scans); a second build is byte-identical; every case passes both the base promptinj_testcase schema and the updated extension schema; the repo's data_validation tests pass 6/6.

Left as follow-ups for you, no urgency: DSGAI05 is used as a catch-all on 89 cases and category: "other" on 64 — tightening those where a more specific mapping fits would raise the suite's value; and the builder/validator scripts living in the dataset folder is a pattern we may consolidate later.

This is a strong contribution — the contrastive control design and the pinned, hash-verified provenance are exactly what this dataset track wants. Merging.

[emmanuelgjr]

Vendor-neutrality changes applied and re-verified on this branch: attribution reduced to the license minimum with the DOI as the single canonical identifier, affiliation/copyright enforcement removed, LF enforced for hash stability, and all 149 cases regenerated through the suite's own deterministic builder against the pinned source with content byte-identical. 149/149 validate; both schemas pass; repo tests 6/6.