One of the ways they've improved the way our organization functions is that when we first signed up with Threat Stack, we were just using password authentication. Managing 70 servers with passwords is terrible. One of the first things they noticed and that we collaborated on was that we needed to start automating some of these logins and actually knowing who was on the box, so it wouldn't always show up as the same user. We needed to disable password authentication, which makes it a lot easier to deal with search certificates. It was basic stuff that I already knew. 

Then they pushed us in the direction of automating that, removing the human element from it. We've been moving forward with Chef to automate all of that so I can get rid of the password authentication on all of our servers. That's just one code-push and all the servers update and I don't have to worry about it.

If I lose an employee, I remove their key from all the servers via Chef and they can't get into any of our boxes. If I need to redeploy an SSL certificate to our boxes, it's all done through Chef. That's been the biggest thing, their partnering with us and helping us identify ways to automate and make that better. 

They've also helped with a couple of security audits that we've had and that has been very helpful.

The other positive changes are better insight into security, so when something happens it's not just, "Oh we had a security incident," or, "I've seen a tech blog that says there's a vulnerability in this version of Nginx. Which of my servers have that?" They take care of all that for me. 

Also, getting the alerts in your face - we've integrated with Slack - so the dev team gets a notification every time a vulnerability is found or something is off. We can then check if someone really did that in our AWS account: Did somebody really mean to do that on our server? And then we can address the issue that way.

Rules give us more visibility and control over what's being triggered and that's been super helpful. I don't have the time to go in there and create those rules. So instead, if we do something that's out of the norm - something we're allowing security-wise that we probably shouldn't, but we're going to address it in the future - they'll contact us, they'll reach out to us as soon as they see something as an anomaly and say, "Hey, did you mean to do this?" We can then say, "Yeah, we did," and then they'll help us configure those rules to suppress them for a limited amount of time until we can resolve the issue, so we're not inundated by non-useful alerts.

In terms of cloud infrastructure, the biggest thing is the fact that they do connect with our AWS account and they let us know which boxes are and are not running the agent. They give us details on that. That's the biggest insight they've given there. That's allowing me to see which servers I have my agent on and which ones I don't. I can get a quick glance at my weak points and servers that I need to either migrate over or get rid of.

We have also seen a measurable decrease in the meantime to remediation in the sense that before, we wouldn't have even been able to detect and then get to the remediation. The remediation wasn't even happening. Now, we're actually alerted to and can start working the security issues. Before, we never would have known, so that's quite the improvement. It's really hard to quantify because we didn't have a good process. We were oblivious to vulnerabilities. 

It has absolutely cut down the time to investigate potential attacks because it tells us immediately via Slack. We have a link, we click the link, we open Threat Stack, and it takes us right to the events we need to know about. That's been just awesome. In terms of time saved, to go in and dig through the servers and find all the logs, it probably saves 45 minutes to two hours per incident, depending on how impacting it is. We get a handful of alerts a week that we have to deal with, so we're saving a couple of hours a couple of times a week. Obviously, partnering with Threat Stack and implementing Chef makes all of that a lot faster. If you take into account all of that, we're saving oodles of time. If we actually had to go patch every box manually without Chef - which we got because of Threat Stack... That's saving a boatload of time, because of their recommendation and going through the security measurements.

The capacity to respond to evidence requests from the SOX auditors has significantly improved because of this tool.

It has also provided us with the ability to gain actionable insight into our cloud infrastructure. We have a long list. The vulnerability and patch-management components allow us to see what our most severe and actionable items are for platform OS, our EC2 instances, our golden images. We're able to see what instances have the greatest need for assessment and remediation and we move down the list on those. Over time, that's going to substantially improve our overall security structure.

We're also seeing the ability to respond to things in real time, particularly Sev 1 Alerts. We don't have any delay. We get the alert, we can immediately jump in. We use Threat Stack to do some forensics on it, figure out what's actually going on, and resolve the situation very quickly. Fortunately, we've not had any true penetrations, but we've had things that have happened and we've been able to alert on those and make adjustments.

It's given us another 50 percent in terms of the time it takes for us to be aware of something. Threat Stack is a great tool for that because it makes you aware more quickly, as opposed to CloudWatch or CloudTrail. The time-to-awareness is significantly decreased because it's an alerting platform. By comparison, it's arduous to write rules that really apply well in CloudTrail or CloudWatch.

In terms of the time needed to investigate potential attacks, the data that's available in the single pane of glass probably knocks half the time off because we don't have to jump over to AWS. We've got it all there.

We have about 210 microservices that make up our product. There are over 140 developers who have access to production, and they can troubleshoot but they're not allowed to make changes. We have to give them enough access to do their troubleshooting while ensuring that they aren't making any changes to the production system. The only way to do that is to monitor every command that they run in production and alert on those commands that are suspicious. It's working to ensure our developers are doing the right thing.

It can also provide a warning that someone from the outside may have compromised a machine. If somebody runs a suspicious command, like whoami or netcat or curl, or any of those kinds of commands that you don't expect, we're immediately alerted. It's a really great tool for that, and we can specify really granular rules.

The things that our developers are normally allowed to do and that we expect to happen, those aren't going to alert somebody. But the things that we don't expect to see in a production environment, those will alert somebody, and we'll move very quickly on them.

The way Threat Stack has improved our organization is directly related to how our production environment works and how we monitor it. The improvement is that we are able to get PCI certification. We use Threat Stack as a compensating control for PCI. We do have developers who have access to our production environment, so we don't have the traditional separation of duties that PCI would like, where the developers who write the code don't have access to production. But we're able to show the compensating control, that we monitor everything that happens in production and that there are no changes made in production by these developers. Threat Stack gives us that ability to implement a compensating control and show it. We were able to get PCI with this control.

The rules definitely give us more visibility and control over what's being triggered. We are able to monitor our environment and see what is normal. When we first installed Threat Stack, we obviously had a lot of alerts. Over time we have been able to monitor and see which of those things is normal. For example, which alerts happen because of automation, automated things that are happening in the environment and that trigger expected alerts? We don't need to see these as alerts. These are expected actions, they're authorized and not caused by users. They wouldn't be caused by a bad actor. They're just simply automation. We are able to write very granular alerts that look for that automation and no longer alert us on it, so we're able to cut down the alerts to a manageable level.

In terms of our cloud infrastructure, one of the things that we get from it is that we now have a baseline of normal. What do we expect to see? What are normal operations? From a security standpoint, what's going on that is the average, that we expect, and what is an outlier? This is one of the tools that allows us to say, "Okay, this is our normal baseline, these things are outliers." And even if they don't reach the alert level of a Sev 1, they're still outliers that we're logging as Sev 2 and Sev 3, and we're still looking at those every day just to see what patterns are changing.

In addition, we use Threat Stack for SOC 2 auditing and it saves us time for the same reason I noted about the separation of duties. It's a tool that we use in the SOC products to show how we're monitoring what happens in our production environment. We use it as a compensating control for the lack of a separation of duties.

Finally, Threat Stack has cut down on the time needed to investigate potential attacks by about 75 percent.  It's much faster now.

It provides the security team with visibility into parts of the organization that were otherwise difficult to see into. By installing the agent we can get visibility into parts of our infrastructure that we otherwise didn't have access to and couldn't see.

The solution provides us with the ability to gain actionable insights into our cloud infrastructure. It gives us a lot of visibility into what's happening in our AWS accounts. The security team can monitor and provide oversight to the cloud operations team. For example, when new security groups are being created, or ingress and egress points are being created at the network layer, we can ensure that they've been documented, tested, approved, and that they have gone through change-control management; things of this nature which are required for, say, compliance purposes. We can detect and then ensure the controls are in place to close the whole loop of the change-control management process.

We develop a SecOps program around this solution. We're using this application to establish some of the controls as part of a SOC 2 audit, as part of a control environment, as well as PCI.

It is a fantastic tool that gives us a level of comfort knowing that there is not only something that's watching, something that can alert and detect, but also knowing that there's an outsourced operation center that can be an auxiliary part of our security team. That is super-helpful. Having their experience in the Amazon Web Services environment is really great because most of our operations are in Amazon Web Services.

The most important example of how it has improved our organization is that we had a security incident that I can't give you a lot of details around. But about two months, ago an attacker compromised an internet facing system. We were able to detect, analyze, and remediate that in less than 60 minutes, on a Saturday.

When an attack compromises a system it changes the configuration of that system. Being able to detect that immediately and take action on it in an extremely short period of time is unbelievably valuable and pretty much mandatory.

The rules definitely give us more visibility and control over what's being triggered. We demo a lot of different security tools, especially cloud-specific security tools. So far, Threat Stack is the only one that we have found that ties all the relevant pieces together, so that we can take action in a meaningful way. Every other security tool we've looked at is good at containers, or at Kubernetes, is good at AWS, or at instance monitoring. But nobody is good at tying all of those things together, and that's really where Threat Stack shines. They take endpoint security and these new technologies very seriously. That alone differentiates them from just about every other competitor in the market right now.

It has absolutely provided us with the ability to gain actionable insights into our cloud infrastructure. We use it as a configuration monitoring and alerting tool. The fact that we can tie 20 AWS accounts into a single view, or a single pane of glass, and monitor the security configurations of those 20 accounts in one setting, is just huge.

We have also used this solution as part of a SOC 2 audit, two years in a row, and it has saved us drastic amounts of time. Before Threat Stack, collecting endpoint evidence in, for instance, AWS configuration evidence, would take a team of three people about a month, in terms of total duration, not total time. Now, we're able to provide that evidence within an hour.

There has been a measurable decrease in the meantime to remediation, by 95 percent. It's a ridiculous level of change, I can't speak highly enough about it. When we had security incidents before, if we detected it - and that's "if" because we didn't have the same level of visibility - the remediation cycles could last weeks. The reason for that was trying to understand what the blast-radius of an attack was. It took a long time to figure that out because we were correlating information from multiple tools, trying to link data, and it turned into a big data problem that we had to solve very quickly. Each incident was different so the data sets were different. It was really hard to set up playbooks to do that quickly. But with Threat Stack, because we have so many different tech verticals already collated in one place, our ability to respond is drastically different than it used to be. It has also cut down the time to investigate potential attacks by the same amount, 95 percent.

Threat Stack allows us to quickly identify public AWS buckets across a large number of accounts, so we can validate what is within those public buckets and should be publicly accessible. That no buckets are being created incorrectly is probably a safe thing.

The ability to reconfigure alert rules allows us to ensure that what we are alerted on is a priority for us.

It provided valuable data in our recent SOC 2 type II audit, where it saved us time.

Threat Stack is pretty easy tool because their integration with AWS instances and everything, that's easy. So you build up a Threat Stack server, if you go to their AWS instances one at a time and then later on, if a new instance gets added or removed, it will keep an eye on that. It acts as a traditional IPS, so whatever, when it is introduced at the first time, that data is the normal state.

In addition, you can do all the integration, and the ticketing becomes very easy, because command is a secure orchestration tool.