Tests Find Forged Email Headers Can Mislead AI Summaries Without Explicit Instructions
Summary
Tests of six email variants found fabricated dates and totals in every test summary. Explicit instructions consistently removed true facts, but hidden text was not required. The results were limited to one email client and viewport.
Key points
- Across 10 trials for each of six test emails, summaries included fabricated dates and invoice totals in every trial.
- Emails with explicit instructions to the summarizer caused all four pre-registered true facts to disappear in every trial.
- Without explicit instructions, the plain-view and invisible-text variants retained all four true facts; the below-the-fold variant retained only two.
- The no-instruction samples used forged second email headers and message content, showing that instruction-wording detection would not identify every manipulation.
- The researchers conclude that hidden-text detection alone is insufficient; instruction detection may block some attacks but can miss injections that introduce false information.
- The test used one email client and viewport with temperature set to zero; the authors say the findings may not generalize and identify higher temperatures and other setups as unanswered questions.
Article Details
- Attack Vectors
- Indirect prompt injection in email content: forged second header blocks and message-history structure caused the summarizer to present fabricated meeting details and invoice totals as facts.
- Explicit instructions embedded in email content caused the summarizer to remove all four pre-registered true facts in the tested trials.
- Email messages padded with 30 blank lines caused the summarizer to omit two true facts in all ten tested trials, even without an explicit instruction.
- The tests found fabricated information could appear in summaries without hidden text or explicit instructions; the authors note that plain-view samples were not realistic attack scenarios on their own.
- Defensive Notes
- Hidden-text detection alone is insufficient: plain-view instructions also affected summaries, and fabricated information appeared in samples without hidden text.
- Instruction-text detection may block some attacks, but the authors found it did not prevent fabricated information from being inserted in summaries without an explicit instruction.
- The findings are limited to one email client and one viewport; the authors do not claim they generalize to other clients.
- The tests used temperature 0 and do not establish how the behavior changes at higher temperatures.