The ability to review and close out tickets or alerts through our mobile phone and being able to interact with engineers on their side via the app are the most valuable features. That's been one of the more beneficial components.

So far, the mobile app has been great. We've been able to reply and interact with them through it. The collaboration is very cohesive.

Nothing will help me solve every alert by any means. We don't do a lot of remediation through CRITICALSTART. We're doing more detection because we do Splunk integration. Whereas, if we were doing the endpoint integration like Microsoft Silence or SentinelOne they would have the ability to lock down a computer-based on that and probably get more insight than what they get right now. The trusted behavior registry does give us the ability based on the alert logging we have with Splunk to dig in a little bit deeper and to even know that something that was an anomaly even occurred. Whereas, before we didn't have that dataset.

In terms of how many escalated alerts we receive in a week or a month, we would always get them during tuning. I would say that we would probably get about a couple of hundred alerts during any normal month, if not a week. It just depends. However, when we moved to CRITICALSTART, we found that we could turn anything on and give them a little bit more information. Of course, until that gets tuned down and we find out what's normal versus not normal in our environment, it is a little chatty. For example, we turned on a certain logging type for our command line and our alerts increased by around double, if not two and a half times but it ended up being a false positive. We just had to go in and ended up tweaking it or filtering it out. It helps decide those alerts but for the most part, it's dropped our alerts down by around 50-75% and we're able to focus on the more important things.

We have decreased a lot of these other alerts that we're able to filter out through CRITICALSTART. With their integration into Splunk, they've been able to add new alerts that we never had set up prior, so it's increased. Whereas, one area might have decreased and another area has increased. Now, we have the visibility of seeing when people don't change, they're having a hard time changing their password or if someone's being added to a local administrator group, things like that. We're getting more visibility than we had before. For those types of things, there's nothing CRITICALSTART can do. So we have those sent right over to us. And we'll work it out on our end and investigate it because they don't know who was added to what. It makes no sense for them to be able to try to work those.

For the most part, using the mobile app to talk to service providers has been pretty responsive, they usually respond within a couple of hours. I would say that before they respond, they typically will do their own homework, which is why it probably takes that long to get their response to investigate. If we escalate it back to them to do filtering, they're pretty quick about getting it.

It definitely alleviates workload because now if we filter out something, for example, if we find that a CID is based on a security group for something that's allowed to be put anywhere in our environment, like an elevated group or a privileged group, then we don't ever need to see that. It doesn't need to come to us so we filter it out. Now, if we've been getting 10 or 15 tickets because of that, we filter it out and we don't see it anymore. Or if there's some change in the way that Microsoft operating system works and it initiates a lot of command-line processes that are alerting because of the way that they're being handled by the operating system, but we know it's not a true positive, but a false positive, we'll filter that out and that'll drop our alerts dramatically.

In terms of the intuitiveness, they are still working out the bugs in the new version because we're testing that out. When I say bugs, I mean that there is still a little bit of slowness. But because they're still working on it, I'm giving a little grace in that aspect. Overall, intuitiveness is great. I've noticed that over the last week or so the response time has actually increased, so that's good. They're still working on it. It's not primetime but the intuitiveness is pretty slick. Responsiveness is a work in progress. 

The updated UI allows us to respond to escalations. It's able to close out tickets or review them a lot quicker because it's all within one interface. If we received the email alert, we already know exactly that this ticket just needs to be closed out because we've already investigated it. Before, we'd have to go click in it, go into that specific alert, and close it out. There were three or four steps. They've removed those two or three extra steps and made it to where you can do all of that from the initial page.

We've been able to integrate everything that we need to integrate. 

The Trusted Behavior Registry helps resolve alerts in the sense that CRITICALSTART is doing a lot of that initial triage for me. Out of a given 500,000 events and alerts, for example, that come through, they're taking out 495,000 of them. That only leaves me with a subset of that to actually have to triage, and that's where it benefits us. They take care of Tier-1 and Tier-2 triage.

And the new mobile app is awesome. It is one of the best I've ever seen. It's much better than its predecessor. It's more intuitive, a whole lot easier to navigate and get where you need to go. It's less repetitive and just generally easier to use. It allows me to not have to be sitting at my computer all the time. I can be on my phone or tablet or wherever I'm at. It makes it a lot easier to answer tickets and do that kind of thing.

Also, the intuitiveness of the updated user interface for the service is spot-on. It is much easier to navigate, and know where to navigate, in the newer interface. I've never had an issue with responsiveness. It's very quick and doesn't sit there and chug on anything. It's fast, it's efficient. It has enabled our SecOps team to take action faster because if you have multiple ways of connecting to it and actually getting your alerts answered and taking care of things fast, it is extremely helpful.

All the information that you need to make a determination is usually in the alert itself that comes through the Zero-Trust Analytics Platform (ZTAP). I don't find myself going back to the app itself very often. That still happens, but not as often. The ability to flow the information forward, from the alert standpoint, helps me because it saves me from running back to get the information. It's improved my efficiency.

Finally, there haven't been any data sources that the service wasn't able to integrate with.

Their Zero Trust Analytics Platform (ZTAP) engine, which is kind of their correlation engine, is by far and away one of the best in the business. We can filter and utilize different lists to build out different alerts, such as, what to alert on and when not to alert. This engine helps reduce our number of alerts and false positives.

The service's Trusted Behavior Registry helps the provider solve every alert. The way that they have it built out is very intelligent. The way every alert comes in, it gets triaged one direction or another. If it is already a false positive, then it is still getting addressed and reviewed on a regular cadence. Also, true positive alerts get escalated to the appropriate personnel.

Its mobile app is great. The ability just to be able to quick reference and see what's coming in when you're on the move or go. You don't always need to have your computer or laptop handy, because you can operate it just from the mobile app. It can communicate with analysts, which is great.

The mobile app is great at affecting the efficiency of our security operations. Those guys are using it throughout the day, whether that be at the office, home, or off hours. Typically, they triage from the mobile app. Then, if an escalation needs to be done on a computer, they will pull out a computer.

We were on the original UI for a few years, so the updated UI has been a refreshing change. It has significantly more ability to filter and translate data, then load that data. It is rather intuitive to click through for some of our junior analysts or interns, especially as we are starting to onboard and teach them different aspects of the security operations team.

The quick interaction between the agents is the most valuable feature. If we have questions, they're quick to answer. If we make a change to our system, they quickly make the changes that are necessary to filter the logs correctly.

They do trusted behavior registry. They filter out the unnecessary stuff and present us with the things that are interesting and let us determine the validity of that type of action in our environment.

We get probably 10 or 12 escalated alerts a week, and there are hundreds or thousands of transactions that would need to be filtered otherwise.

The mobile app is a nice way to get quick access to something when I don't have access to the full system. It's a good way of accessing all the data that I would need when I'm remote. The mobile app gives me more comfort in that I will be alerted if there is something going on, even when I'm remote.

CRITICALSTART makes us much more comfortable with knowing someone else is watching our data and our systems and knowing that professional security people are taking a look at any issues that do arise.

The new UI seems a little slower but some of the functionality is a little bit quicker to get to things in terms of navigation. It has made it easier to respond to escalations. The alerts are displayed in a way that makes it simpler to respond. The response dialogue is right on the screen.

In terms of transparency, it seems like all the data is available to us. It affects our security by allowing us to see what they are doing in terms of filtering and making sure that we agree with all the filters that they're adding.

CRITICALSTART has increased our analyst's efficiency to the point that they can focus on other areas of business. We implemented some of these tools at the same time we started with CRITICALSTART. Some of that wasn't being done before, but now it is being done and we still have the time to do other things.

It also takes care of the tier one and tier two triage. It saves my team around 10 hours a week. 

I think that the provider contractually committed to paying a penalty if it misses a one hour SLA to resolve an escalated alert. But it wasn't a huge deal for us. It wasn't a critical thing that we looked at. So far, they haven't missed such SLAs, as far as I know. It has yet to miss an attack. 

We chose not to integrate data sources due to the cost of our firewall logs. They would have been able to ingest them through a SIEM had we wanted to.

We benefit from alert reduction and the ability to cross-correlate multiple logs to achieve a more secure environment. CRITICALSTART consolidates alerts, creates reporting, and gives us a more holistic view of our security landscape.

They start with a Zero-Trust model and build from Zero up to Trusted. We found this to be extremely effective in filtering out alerts. So, we started from Zero-Trust, then we built the trust from there. This has became extremely effective for us in our environment. 

We have over 99 percent filter rate for the service’s Trusted Behavior Registry.

It is extremely effective for our team's utilization of the service. It is easy to maneuver and understand. If it ever requires any additional information or a deep dive, we reach out to CRITICALSTART to help understand an alert, why we're getting an alert that we think we shouldn't be getting, or fine tuning an alert.

It has enabled our SecOps and internal SOC managers to take action faster and respond to escalations more easily. Because the front-end is easily maneuverable, we have the ability to work through it, get into it, and understand it. This allows us to pivot back and forth between logs, log sources, and understand the alerting. It's not a convoluted front-end.

The 24/7 SOC security: There is a team of people who monitor our traffic and processes 24/7, so if anything raises a flag or alert, it will escalate back to me right away. That's the most incredible part: Humans working behind the scenes 24/7 to monitor our networks.

The intuitiveness and responsiveness of the updated service's user interface is pretty cool, especially the dark theme, which I like. It is easy on the eyes. It's not like a traditional portal. It looks very futuristic, but I think it's more accessible and less crowded. The new interface is definitely an improvement. 

I am a one-man team. Everything is done by just me. I did find that it is easy to find things on the UI. I think it's an improvement from the one we had when I started.

There are two parts of CRITICALSTART's services that are most valuable to us

• The MDR solution where they monitor our computers, laptops, and users across the board. 
• Their knowledge of Palo Alto firewalls.

And their mobile app is actually our preferred method of interacting with them. We get notifications and can reply to tickets on-the-go. I don't think there's any other solution that offers such a thing. It's super-useful. Everybody's got a web portal, but this mobile app is quite something. It's pretty cool.

The mobile app is self-explanatory. You have a ticket or you get a notification and you can chat or submit information. You can talk to their team on-the-go. It's very convenient. If you go farther, you can look up tickets and you can look at the assigned statuses. There's more to it; it's a full-blown app. Maybe there are a couple of features that are easier to use in a web browser with a larger window, but I think it's pretty full-featured. You can change tickets, you can assign the queues, you can post a reply. You can look at the details. The whole thing is there. For us, the main thing is that when there is an alert we can act on it right then.

We also talk with CRITICALSTART analysts, two folks in particular. Their response time is very quick. If they cannot talk to us, we get a reply from them anyway. We don't have to wait around. The response time is very good in comparison to larger companies. CRITICALSTART is fairly large, but there are larger companies where you send a ticket, request support, and you're not sure who's going to get the ticket, who's going to respond; you're not sure when that is going to happen. It's always a waiting game. With CRITICALSTART, it doesn't look that way. They give you a personal approach. Their folks are always available. That makes us more likely to do business with them.

When it comes to the transparency of data in the platform, everything is there if we want to look at it. We really don't get too much into it, but if you want to look at it, it's all available. They show the details; they show how they do it. If you want to know if they're lying to you or not, you can look at the details and the facts they base their decisions on when blocking certain things or monitoring certain stuff. It's pretty transparent. It's very trustworthy. It gives us confidence in the decision-making process, because we see how things are done. It gives us peace of mind.

The most valuable part of the service is that they are 100 percent taking care of all first-line alerts. With eyes on glass, fingers on keyboard, they're doing the work. If they have a question, or they haven't seen something in our environment before, then they will escalate it to me. The service takes care of Tier-1 and Tier-2 triage. They actually provide a report that gives details on how much that saves us. I looked at it when we first started, and it was multiple FTEs, on an annual basis, that they're saving us.

I also use their mobile app. It's very easy to use and very convenient to be able to respond to alerts wherever you are. I love the app. You can respond and communicate, per ticket, with their SOC in near real-time. The response is very quick. I can close tickets, I can escalate them. I have very close to all of the capabilities that I have on my desktop. All the things that I need to do in a ticket, I can typically do them from the app. I am a one-man show. I'm the only security analyst for our organization. I couldn't really do my job without the app. I can't sit in front of a computer all the time, so it's critical for us.

I communicate with CRITICALSTART's security analysts. I haven't spoken with them over the phone, except for one time, in a year-and-a-half, but their accessibility is very high. I always receive quick responses to my escalated tickets. When I'm commenting, they're following up, and they're very fast.

I feel I have full transparency to their SOC. Anything I want to go look at, I can do so. I can see all of the comments and discussions that the SOC team has on behalf of us. I have full transparency.

In terms of CRITICALSTART contractually committing to paying a penalty if it misses a one-hour SLA to resolve an escalated alert, I honestly haven't looked at the contract in a year and a half, so I don't remember if it's monetary. I believe that it is. They're very proud of their SLA and not missing it, so I've not ever had an issue or concern or had to think about it. This high commitment to SLAs was our CIO's primary concern when we were looking at CRITICALSTART. After seeing their record, 18 months ago, of not missing a single SLA, it became a moot point. It was a concern at the time but they satisfied that concern.

Outside of using the platform to manage alerts, the feature of the service that we get the most value from is being able to reach out to them and say, "Hey, we might go buy a SIEM," for example. They give us their overview of what's out there, what they've dealt with, what they integrate with, and what that looks like. That's been pretty powerful over the years for us.

And when it comes to the alerts, they get the number of them down and only alert us about what we really need to know about. We get about a dozen or so things escalated in a day. Most of those are low alerts.

We chat with CRITICALSTART's analysts back and forth with comments or when we escalate things back to them. Occasionally we'll open a support request for a feature or we'll have a question about something and we may converse with them over that. Their availability has always been pretty good, especially when it comes to escalating to the SOC directly. We get responses pretty quickly.

I've used the updated user interface about a half-a-dozen times. I felt like it was going to take a little bit of getting used to it, but it did seem like it was pretty quick. It had more of the data right in front of me that I usually want, as opposed to clicking around to go find it. So far I have nothing but positive things to say about it.

The most valuable feature of their service is their tuning. All the service really does is get things to the point where we get fewer alerts sent to us. If we were getting 1,000 alerts a day without them, they tune it until they know what to do for 999 of them, and one will make it through to us per day. That tuning is the most valuable part of their solution.

When we had Carbon Black, we were getting at least one escalated alert a day, maybe more, because it wasn't able to be tuned the same way that other services can be, or maybe Carbon Black itself alerts that much more. With Cortex XDR, we're only getting about one escalated alert a week, or one a month. It's much less.