Skip to content

feat(exploitation): add Recommendation Memory Poisoning exploit (#25) - #81

Open
MRX-72 wants to merge 1 commit into
GenAI-Security-Project:mainfrom
MRX-72:feat/recommendation-poisoning
Open

MRX-72 wants to merge 1 commit into
GenAI-Security-Project:mainfrom
MRX-72:feat/recommendation-poisoning

Conversation

@MRX-72

@MRX-72 MRX-72 commented Sep 14, 2026

Copy link
Copy Markdown

Summary

Implements #25 (Recommendation Memory Poisoning). Adds a new exploitation
example, exploitation/recommendation_poisoning/, against the existing
llm_memory_local sandbox (#22, #60).

The angle that makes this distinct from the sibling memory_poisoning exploit
is the vector. There, the attacker sends a "remember that ..." chat
message in their own session. Here the attacker never talks to the assistant:
they publish a web widget — a "Share this deal" button — whose visible label is
innocent but which carries a hidden instruction (aria-label + a
display:none span). When the application forwards that page content to the
assistant, the hidden text rides along and is persisted into the unscoped
memory store, steering a later, unrelated user's product recommendation toward
the attacker's brand. This is the indirect prompt-injection scenario described
in the issue and the Microsoft "AI recommendation poisoning" writeup it links.

What's included

  • webwidget.py — extracts hidden instructions (aria-label, title,
    hidden / display:none / visibility:hidden text) from attacker markup.
  • attack.py — live attack against the running sandbox: harvest the hidden
    instruction, submit it, then confirm the attacker's brand leaks into an
    unrelated victim's recommendation.
  • recommendation_demo.py — offline, deterministic companion (no container,
    model, or network). Three phases: neutral baseline → poisoning → leak. It
    drives the sandbox's real app/memory.py (build_memory_context /
    extract_and_store_facts) with a small deterministic recommender stand-in,
    and exits non-zero if the steering is not observed, so it doubles as a
    regression check.
  • config/config.toml, Makefile (setup / attack / demo / stop /
    all), and a README with an attack diagram and OWASP mapping.

How to test

Offline, no setup required:

cd exploitation/recommendation_poisoning
python recommendation_demo.py

Expected tail:

STEERING CONFIRMED: the assistant gave a neutral recommendation before
poisoning, then recommended the attacker's brand to an unrelated victim after a
hidden web-widget instruction was laundered through trusted memory.

Live, against the sandbox:

make setup
make attack   # -> [!] VULNERABLE: the poisoned brand 'QuantumGlow' leaked ...

Notes

  • Reuses sandboxes/llm_memory_local unchanged; no sandbox edits.
  • Payloads are harmless, clearly-marked test values: a fictional brand
    (QuantumGlow) and a non-resolving .test store.
  • Formatted with black and isort (--profile black); mypy clean.

…I-Security-Project#25)

Adds exploitation/recommendation_poisoning targeting the llm_memory_local
sandbox. Unlike the direct memory_poisoning exploit, the attacker plants a
hidden instruction in a web widget (indirect prompt injection) that the app
forwards into unscoped memory, steering a later unrelated user's product
recommendation to the attacker's brand.

- webwidget.py: extract hidden instructions (aria-label, hidden/display:none)
- attack.py: live attack against the running sandbox
- recommendation_demo.py: offline deterministic steering demo + regression check
  driving the sandbox's real app/memory.py
- config, Makefile, README with attack diagram and OWASP mapping

Formatted with black/isort; mypy clean.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant