Add practical web exploit examples for unsafe AI outputs

[huasohacker]

Summary

Adds practical examples of web exploits enabled by unsafe AI outputs, addressing Issue #28.

Changes

• New file: Web_Exploits_Enabled_by_AI_Outputs.md
• 14 detailed exploit scenarios across 6 vulnerability categories:
  • XSS (Stored, DOM-based, SVG/Markdown)
  • SQL Injection (Direct, Dynamic, NoSQL)
  • Command Injection (Shell, Code exec, Filenames)
  • SSRF (URL fetch, Webhooks)
  • Path Traversal (File read, Upload paths)
  • SSTI (Jinja2 templates)
• Testing methodology with context-specific payloads
• Mitigation strategies and code review checklist

Related Issue

Closes #28

[MatOwasp]

Hi,
please read here:
https://github.com/OWASP/www-project-ai-testing-guide/blob/main/Document/content/1.3_Objectives_of_AI_Testing_Guide.md

This guide does not attempt to replace or duplicate existing foundational security testing methodologies. Instead, it complements them by focusing on AI-specific threats. We can add it to: https://github.com/OWASP/www-project-ai-testing-guide/blob/main/Document/content/tests/AITG-APP-05_Testing_for_Unsafe_Outputs.md but we need to merge your new content.

Thanks,
Mat

[SuneetMalhotra]

Thanks @huasohacker for this contribution. Six months in the review queue is a long time for a substantive PR; offering a first-pass technical review in the hope it helps move it forward.

Overall

The technical content is solid — 14 exploit scenarios spanning six major web-vulnerability classes (XSS, SQLi, Command Injection, SSRF, Path Traversal, SSTI) with clean vulnerable-vs-secure code pairings and a context-aware encoding table at the end. This addresses Issue #28 substantively and is the kind of practitioner-oriented reference that fills a real gap in AI-output security guidance.

Some things that I think need to be addressed before merge:

Structural / integration issues (the big ones)

1. File placement doesn't follow the guide's taxonomy. The file is at Document/content/Web_Exploits_Enabled_by_AI_Outputs.md, but every existing test case in the guide lives under Document/content/tests/ with the naming convention AITG-{CATEGORY}-{NN}_{Description}.md. As currently placed, the content is orphaned from the taxonomy and will not be discoverable through the guide's TOC or framework mapping. Two reasonable resolutions:

• (a) Convert to AITG-APP-XX test cases — split the six exploit families into separate AITG-APP test files (e.g. AITG-APP-15 for AI-Generated XSS, AITG-APP-16 for AI-Generated SQL Injection, etc.) or fold into an extension of the existing AITG-APP-05 (Testing for Unsafe Outputs).
• (b) Keep as a stand-alone appendix — rename to Appendix_X_Web_Exploits_from_AI_Outputs.md following the pattern that other appendices (F/JEF in PR Add Appendix F: Jailbreak Evaluation Framework (JEF) #69) are using, and add a TOC entry pointing to it.

2. No TOC update in Document/README.md or in 3.0_...Framework.md / 3.1_AI_Application_Testing.md. Whichever integration path is chosen, the file needs to be indexed or nobody browsing the guide will find it.

3. Cross-reference inconsistency with AITG-APP-05. The header says "AITG Reference: AITG-APP-05 (Testing for Insecure Output Handling)", but the current guide has AITG-APP-05 as "Testing for Unsafe Outputs" — the name in this doc appears to reference the OWASP LLM Top 10 (LLM02: Insecure Output Handling) rather than the AITG-APP catalogue. Recommend clarifying the naming or fixing the cross-reference so future readers know exactly what to consult.

4. The Contributors section still contains a placeholder — literal [Tu nombre aquí] (Spanish for "Your name here") at the bottom. Blocker for a merge.

Technical content — specific suggestions

5. Missing exploit categories. Three notable gaps relative to the AI-output threat surface:

• Insecure deserialization — AI-generated JSON structures used with pickle (Python), Java Object Serialization, or YAML safe_load-vs-load confusion. Very common in Python/Java web stacks and directly enabled by AI-generated code.
• Regex Denial of Service (ReDoS) — AI-generated regex patterns with catastrophic backtracking. Widely underrated failure mode; well-documented in OWASP and USTS.
• Prototype pollution — AI-generated Node.js object-merge / assign patterns that write to __proto__ or constructor.prototype. Very common when AI generates deep-merge utilities.

6. SSRF example is dated. The http://169.254.169.254/latest/meta-data/ example targets AWS IMDSv1, which most production environments have deprecated in favour of IMDSv2 (PUT /latest/api/token first, then token-authenticated GET). The exploit as shown will fail on any AWS environment where the operator has forced IMDSv2. Recommend either adding an IMDSv2 flow example, or covering multi-cloud (GCP metadata.google.internal, Azure IMDS, Kubernetes 169.254.169.254 in-cluster) as a family of link-local metadata endpoint targets.

7. Testing methodology (Section 7) is thin. The payload contexts are useful but there is no systematic evaluation harness or metric. Consider adding: (a) a per-scenario expected-pass condition, (b) a corpus-level Attack Success Rate (ASR) definition, and (c) a suggested minimum sample size for defensible measurement. PR #69 (JEF) in this repo has some framing that could be borrowed for cross-consistency.

8. Mitigation table (Section 8.1) has a minor error. For the SQL row, the recommendation is "Parameterized queries (not encoding)" — correct — but it would be clearer to explicitly state "Never use string interpolation for SQL; always use parameter binding provided by the DB driver." The current phrasing risks being read as "encoding is optional for SQL," which is stronger than intended.

9. Defense-in-depth diagram (Section 8.2) is a serviceable ASCII flow. If the guide accepts SVG/PNG figures, this would render better as an image — the guide already uses figures elsewhere (e.g. AITG-APP-11 references hallucination.png).

Positives worth calling out

• Vulnerable-vs-secure code pairing is the right pedagogical pattern for this content. Reviewers or readers can copy the "Secure Alternative" block directly.
• Command Injection filename example (Section 3.3) is particularly valuable — this is a real production incident pattern that developers frequently miss.
• CSP + validation + encoding + sandboxing as the four-layer mitigation stack is the correct framing.

Happy to help further once you and the maintainers land on an integration direction. Thanks for the substantial contribution here.

---

Suneet Malhotra
IEEE Senior Member · ICSE 2027 SEIP Program Committee · ASE 2026 Artifact Evaluation Committee · NeurIPS 2026 Ethics Reviewer
Independent practitioner-researcher — LLM-augmented software testing
suneetmalhotra.com · github.com/SuneetMalhotra · ORCID 0009-0003-8707-9590