With cyber security on the minds of many large and small organizations, phishing, a type of social engineering attack, poses an increasingly common threat to every organization's information technology (IT) enterprise and therefore to the organization's ability to perform successfully. Phishing attacks target the weakest link in the information security chain—the individual end users. For example, one phishing attack tried to defraud users into resetting their DoD Common Access Card (CAC) Personal Identification Numbers (PINs) via an external website. Some organizations have attempted to protect themselves by engaging their workforce in phishing attack exercises. Frequently, these training exercises are announced beforehand and do not include remediation—these two factors may impede any organization's ability to improve user behavior and to attain required IT security outcomes in an actual work environment.

This paper describes the methodology, results, and lessons learned from a blind study on the effectiveness of pre-incident training to improve performance against phishing attacks (N = 467). During the study, each of the five treatment and control groups received a different type of training before exposure to an unannounced phishing attack. The study then measured the effectiveness of combining sustained, unannounced, phishing exercises with remedial training. The results show that an approach employing sustained training and exercises can significantly improve learning transfer and on-the-job performance as opposed to traditional training approaches, which had no positive impact on performance. Additionally, the response metrics and feedback from the treatment groups offer key insights into how phishing awareness training and exercises should be implemented for a workforce. Also, a real-world phishing attack during the study provided supporting evidence to the efficacy of the sustained training and exercises approach.