Phishing in the LLM Era: Challenges and Opportunities
Seminar Dagstuhl Seminars: Security and Privacy of Large Language Models
Dagstuhl, Germany
Oneliner: The best thing about phishing, research-wise, is that everyone can relate to it.
(TBD)
Abstract: Every day, our inboxes are flooded with unsolicited emails, ranging between annoying spam to more subtle phishing scams. Unfortunately, despite abundant prior research efforts proposing solutions achieving near-perfect accuracy, and despite numerous organizations employing phishing education campaigns, the reality is that countering malicious emails still remains an unsolved dilemma—which has been exacerbated by the advent of LLM.
This talk will cover two recent publications (AsiaCCS’25, and AISec’25) which present novel results featuring the application of LLMs in the phishing-email context. First, I will show the effectiveness that LLM-generated emails, crafted with naive OSINT techniques, can have against ~18k employees of three different organizations. Then, I will focus on the research side, and elucidate various “open problems” that affect academic literature on phishing-email detection—such as the excessive reliance on datasets containing emails exchanged up to 20 years ago. Finally, I will present how LLMs can be used defensively, i.e., to generate large-scale datasets of phishing emails which could resemble those sent by real attackers, and which could be used to benchmark existing (and future) detection methods—including, of course, those based on LLMs!