“Do Users fall for Real Adversarial Phishing?” Investigating the Human Response to Evasive Webpages
Draganovic, A., Dambra, S., Aldana Iuit, J., Roundy, K., Apruzzese, G., APWG Symposium on Electronic Crime Research [Runner-up for BEST PAPER AWARD], 2023 Conference
Oneliner: The first user-study assessing the human capabilities to recognize real "adversarial" phishing webpages that evaded a real phishing detection system based on deep learning
Abstract. Phishing websites are everywhere, and countermeasures based on static blocklists cannot cope with such a threat. To address this problem, state-of-the-art solutions entail the application of machine learning (ML) to detect phishing websites by comparing their visual similarity with the legitimate webpages of well-known brands. These techniques have achieved promising results in research and, consequently, some security companies began to deploy them also in their phishing detection systems (PDS). However, ML techniques are not perfect and some samples are bound to bypass even production-grade PDS.
In this paper, we scrutinize whether genuine phishing websites that evade commercial ML-based PDS represent a problem “in reality”. Although nobody likes landing on a phishing webpage, a false negative may not lead to serious consequences if the users (i.e., the actual target of phishing) can recognize that “something is phishy”. Practically, we carry out the first user-study (N=126) wherein we assess whether unsuspecting users (having diverse backgrounds) are deceived by “adversarial” phishing webpages that evaded a real PDS. We found that some well-crafted adversarial webpages can trick most participants (even IT experts), albeit others are easily recognized by most users. Our study is relevant for practitioners, since it allows prioritizing phishing webpages that simultaneously fool (i) machines and (ii) humans—i.e., their intended targets.