Red Team: Pwning the Hearts and Minds one Ticket at a Time

One of the foundational cornerstones of DevSecOps is red teaming. Looking at your infrastructure and code from the viewpoint of an attacker allows for a better security understanding of the weaknesses and strengths of an application, service, datacenter and cloud.

Having worked in a security context for a majority of my career I know that scan reports suck and are completely broken. This might not be a popular opinion but has anyone here actually gone through a 500+ page scan report page by page and understood the real impact of the vulnerabilities that were found? Have you ever seen any progress being made off a 500+ page report? This antiquated approach to security is flawed and actually puts organizations in danger. So how can we fix this broken model?

Recently I have had the privilege of participating in and assisting in the implementation of a red team at a large corporation. Having previously done penetration testing it was quite the adjustment in mindset. One of the major differences between red teaming and penetration testing is scope. Red teams do not have a set scope. This is how it should be. A real world attacker could care less about scope. Attackers don’t discriminate between production and QA and test so neither should we. What it really comes down to is our red team has 1 major rule:

Don’t take down production.

Other than that everything is fair game. We perform social engineering, phishing and hacking attacks on a regular basis. Everyone from executives to interns are a target. Initially, I felt that having this aggressive tactic was bad. I thought that we would receive major push back development and devops teams. However, contrary to my initial thoughts, the results we have seen because of being aggressive have been able to drive a substantial change in the security culture of our organization. I personally have never seen such a positive response to security in all my years working in information security. Essentially, what it comes down to is that no one wants to be a target and no one wants to be pwned by the red team and everyone wants to be the one to catch us.

What it really comes down to is actively demonstrating weaknesses. When the red team finds and exploits an issue the once theoretical weakness that was buried in a scan report is now a proven issue. It’s really hard to argue with a shell on a remote server. The presence of active exploitation also is a great way to motivate devops to prioritize to fix an issue rather than focusing on the next feature. This approach is especially powerful when leadership begins to see how exploited vulnerabilities can directly lead to access to internal data.

The DevSecOps mindset thinks of security issues as bug fixes. Security vulnerabilities are essentially bugs that can lead to unwanted outcomes in a product. By working with and discussing the issue as a bug it is easier for a developer to understand the root case of the vulnerability.

As much fun as the hacking aspect of red teaming is the most important part of a red team engagement is remediation. Unlike pentesting we don't write reports. Pentesting reports usually only get read by management and very much like a vulnerability report usually just get filed and not much gets done. Instead of the report approach we use ticketing as a way to track remediation. Ticketing gives us a common language to work with developers. As stated previously we consider security vulnerabilities a bug in the project we found them in. By creating a bug fix ticket we can add steps for the developers to reproduce the issue as well as provide clear and crisp remediation steps for resolution. This approach has been one of the best ways I have ever seen to discuss vulnerabilities with developers and devops. I have seen complete remediations of findings in less than 8 hours which is unheard of in many large organizations. Another devops aspect we leverage are SLAs. Our developers have very specific SLAs in place to track critical bugs. We use these SLAs as part of our findings. This has really helped us show the urgency of some of our findings. It also lets developers know the severity of the findings in a common language instead of security speak.

Finally, it's important to get the developers in on red teaming. By working beside them and having them participate in red team activities it is possible to discover weaknesses and security vulnerabilities that are hard to find. It can also foster some healthy competition. We have had devops teams challenge us to attack their applications. We have also had devops come to us with security concerns that they have and need some help with getting fixed. Since we started our red team activities there has been a huge interest by the developers to participate and overall it has been a positive and rewarding experience.

So get out there hack yourself, find bugs, create tickets, track remediation and make your organization more rugged in the process.

Questions, Comments or Concerns? Let's start a conversation. Leave a comment, let me know directly at ian@devsecops.org or hit me up on Twitter @iallison.