Prompt Injection

Prompt injection attempts to redirect a model through supplied instructions. Perez and Ribeiro distinguish goal hijacking from prompt leaking. Greshake and colleagues extend the concern to instructions embedded in retrieved material.^[1][2]

A document is not a supervisor

Imagine asking an assistant to summarize a museum webpage. The page contains a note telling the assistant to abandon the summary and recommend an unrelated shop. In this original hypothetical example, the page should be evidence about the museum. Following its instruction would give the document authority over the task.

Think of a librarian reading a book that says “rearrange the library.” Reading the sentence does not authorize the librarian to follow it. The useful question is which text supplies information and which text has authority to assign work.

A direct attack arrives through the attacker’s own interaction. An indirect one reaches an assistant through material it processes, potentially during another person’s legitimate task. This distinction concerns the delivery route; it does not establish the attacker’s capabilities or resulting harm in every setting.^[2]

Questions for an incident report

Ask what content the model read, who controlled it, what actions were available, what it actually did and how the outcome was recorded. The incidents guide separates controlled demonstrations from production incidents. Monitoring research raises further questions about shared vulnerabilities; multiple opinions are useful only to the extent that the relevant failure is independently caught.

Follow the defense question

Instruction Hierarchy separates the authority of text from its wording. Read the training experiment, then the architectural position paper. Their claims require different evidence; neither establishes that every possible attack is prevented.