The first thing I recommend, is reading Phishing Dark Waters by Christopher Hadnagy, Michele Fincher, and Robin Dreeke. They have a lot of great insights on phishing and how to build a program and I used the book as a guide to build my own. One of the ideas in the book that really helped give me direction for building the program were the metrics. The book broke metrics down into four categories:

• Clicked and Reported

• Clicked and Didn’t Report

• Didn’t Click and Reported

• Didn’t Click and Didn’t Report

The idea of a phishing program is to reduce click rates and increase reporting rates. These metrics helped establish goals and strategies for building and running a successful phishing program. Using these metrics as a guide we were able to reduce click rates and improve reporting rates by over 50% at a company with over 6000 employees. Below we’ll get into getting started, the mindset to have, how to mature the program, and metrics and reporting.

Getting Started

Leadership buy-in

The first thing needed is leadership buy-in. The higher up the leadership buy-in the more effective the program. If buy-in isn’t at the highest level don’t fret. Once the program is started leadership will start to buy-in once they see the metrics. Metrics have a way of providing valuable insight into the risk associated with phishing attacks for the company.

Who to tell

Before sending a phish you need to inform the people that will help keep the phish from becoming a full blown incident. This can vary depending on the organization. Some will want very few people to be told. Others will want legal and HR input. The essential people that need to be involved is the person you report to and the Security Operations Center (SOC) and help desk managers.

The SOC and help desk managers will need to determine if their people need to be told. The SOC and help desk should be included in the phishing simulation, other times it might be more beneficial to let them to know. Often, they managers will want to see how their directs respond to a phishing email report. For larger phishes it’s a good idea to inform the help desk but for more targeted phishes they may not need to be told. There’s also always the option of making them a targeted phishing group.

Automation

Sending out phishes will increase the workload on other departments like the help desk, the SOC, and anyone monitoring the security inbox, if that’s not already the SOC. Automation is a friend here. Setup automated responses wherever a phishing email may be reported.

We didn’t do this for our first phish of the company and had over 500 people report the email. I responded to every single one of them because it was my miss and I wanted to acknowledge and show people appreciation for reporting a phish. If they’re not acknowledged and thanked they’ll be less likely to send in a phishing email in the future.

Recognize people who report phishing emails

To make an effective phishing program people need to be recognized and thanked for taking the time to identify and report a phishing email. If there’s a platform where employees can send other employees praise or recognition I would load anyone who reports a phish in there. People need positive feedback to continue the positive behavior.

Also, it’s okay if people tell each other about the simulated phish. We want others getting into the habit of giving their peers and co-workers a heads up that they have a phishing email in their inbox. Simulated phish or real phish people giving each others a heads up is a good thing.

Create your first phish

To start pick something super dumb that has a lot of indicators that easily identify it as a phishing email. This will provide a baseline for the overall click rate of the organization. It will help build the roadmap for future phishes. Establishing the baseline sets the starting point. As click rates go down the difficulty of the phishes can be increased and reported on. This will help show a reduction in risk to leadership.

The thing to remember about click rate and phishing emails is that there a lot of factors that go into clicking on an email. The time of day, the stress levels of people, what’s going on at work and at home, and luck. Who get’s sent a phish, time of day, and the type of phish are the only things in our control. Click rate is volatile. I’ve seen a monthly phish get a 2% click rate. I’ve also seen a monthly phish get a 14% rate. Pay attention to the time of year and what might be going on inside and outside the organization.

Deciding on whether to blast out the email or schedule it over a period of time is going to be very important. For larger groups you want to schedule the phish over a period of time. I would phish the entire company monthly. They’d get the phish at random times throughout the month. For smaller groups I had the option of sending them the phish all at once. Sending out a phish to several thousand emails in one day that will not make you any friends with the SOC or help desk, especially if automation is not set up.

What’s off limits

Even if your CEO gives you free reign, like I’ve had in the past, you do not have free reign. GoDaddy got in trouble for a phish in 2020 that the security team sent. The lure was a $650 holiday bonus. After people clicked they instead got told they were assigned extra security awareness training. While the bad guys may use this type of technique or other types of phishing emails we as the good guys should not stoop so low. That type of phish is getting people’s hopes up and then bringing it back down. This will result in an angry reaction.

Anything dealing with financial, family members, politics, religion, or sex are off limits. These topics create an extra strong emotional reaction from people. I also wouldn’t mess with anything related to marketing or other departments needing to get employees engaged. Any of these will be sure to get you in political hot water. Even if you get backed up by the CEO that group may have to accept it, but they won’t like it and will look to sabotage the program.

The phishing program is something people in the organization should understand is here to help. It’s already hard enough to get people to buy-in and feel good about security. Pissing them off won’t help the program and may even result in it being hamstrung. That’s why it’s important to remember that a phishing program is practicing for the real thing. It’s not the game of “Gotcha!” it’s practice.

It’s about practice

The phishing program is about practicing the activity of receiving and responding to a phishing email. Getting people to get them doesn’t help and can put the phishing program in choppy political waters. That’s why the program needs to tie back to something real world.

Dig into your email gateway and look for phishes that are being caught in there. Check the security inbox to see what actual phishing emails are being reported there. Look for ones that are of a general nature for the entire company. Pay attention to the news and what are some of the latest phishing emails being sent to people. Think about the time of year. Packages are flying around during November and December. The phishing platforms do a good job of adding new templates with the latest phishing emails they’re seeing. Make it relevant.

Targeted phishes

Targeted phishes are phishes that are sent to a targeted group. The purpose should be specific to the department or group of people and related to techniques attackers may use to try and get into an organization. Again, look in security tooling to understand what certain groups are being targeted with and research phishes in the news that relate to the company’s industry.

Depending on your organization you can go outside of the parameters of making it related to outside news events. In the past I’ve seen phishes using Game of Thrones and the latest Avengers movies as lures. These were sent to groups who were aware of the phishing program and did a better job of identifying phishes. For targeted phishes like this make sure to host training afterwards to discuss and reiterate the practice aspect of the phish.

One of the most successful phishes I ever did was part of a lunch and learn session. The phish got a 50% click rate and it wasn’t even my idea. As part of the session I asked the people in attendance for ideas for a phish to send to IT. We had a praise platform that you could use to send people praise. So we decided to do a phish that used one of the notification emails for getting praise. Then we made it look like it was from the CEO. We did add several indicators that it was a phishing email such as giving them a nonexistent praise and an obvious link if you hover over it. We got clicks almost immediately during the session.

Later that day I was visited by a couple of directors in the IT department who said they had never fallen for an internal phish before at any organization. I avoided severe political backlash in this situation because they were in a group with a low click rate and they had access to the lunch and learn where we did the phish. In another organization this could have caused a lot of issues.

Despite conducting phishes as a way to gather information and reduce risk in the organization we are still going to bruise some people’s ego. Which is why we need to be thoughtful and careful about the phishes we send.

Increase the difficulty

As the click rate goes down, increase the difficulty. Determining if you can increase the difficulty should be from a reduced click rate from a period of over three or more months. Month-to-month click rate can be volatile. To increase difficulty reduce the number of indicators in a phishing email. If you started with five indicators reduce it to four. This allows the phishing emails to have levels of difficulty that can be reported on.

Indicators can be anything from reducing misspellings to making domains look a lot more legitimate. We’ve used domains that were bought to protect the company from typosquatting attacks. We loaded those into the platform and used them when we needed to increase the difficulty of phishing emails.

Reporting, Metrics, and extra training

As mentioned above, I like to use click rate and report rate. Other statistics don’t provide as much insight. The phishing platform may not have those statistics as default which means some excel jujitsu will be needed to get the metrics worth reporting up.

I never liked calling out individuals unless they were flagged multiple times as repeat clickers and put the company at a significant risk. In that case a conversation with their manager and HR is useful. One of the things I find useful was to group click rate and report rate by department. Grouping departments gives people an out but still allows large groups of people to be reported up if they’re having trouble with phishing emails. Leadership liked this grouping as it provided them with good insight into which departments were struggling with phishing emails. This also motivates departments do better because they don’t want to be in the top 10 click rate and want to be in the top 10 for report rate.

As far as training, I didn’t like assigning extra training from the phishing platform unless there was buy-in from the top and could be tied to something performance wise from an HR standpoint. If I assigned training without any sort of outcome, people could ignore the training and not have any repercussions. I do still think training is important and preferred in-person training because it allowed me to walk them through the phish and allow them to ask questions. I found that the groups I got to work with in these training sessions did a much better job with phishing emails. Those sessions can also be recorded and put into a LMS platform.

Summary

A phishing program can be a powerful security awareness tool for an organization. It should look to decrease click rate and improve report rate. The first phish should set a baseline. Increase the difficulty as click rates go down and report rates go up. Try to tie phishes to relevant phishes that is being seen in the company’s security tooling. Even with free reign certain phishes are off-limits. The CEO might be okay with it but everyone else will start to harbor bad feelings towards the phishing program and security and will look to undermine it when possible.

Identify what metrics are important and put those together to be reported up. Creating top 10 lists for departments is a great way to gamify the reporting and get people to more actively participate. Finally. remember this is about practice. Anyone can fall for a phish if the right factors line up. Taking an empathetic approach will help with making the program more engaging and effective.

Drop any questions you may have in the comment section below or reach out via the contact form.

This post first appeared on Exploring Information Security.