They could dig a little bit deeper into the Splunk alerts when they feel like they need to be escalated to us. For example, if a locked account shows up, they could do a little extra digging to verify that the locked account was due to a bad password on the local system. They could just do a little extra digging within the Splunk environment instead of pushing it onto us to go do that extra little digging. We actually created dashboards for our help desk group to be able to hunt down locked down accounts. We've asked CRITICALSTART to start using that as a means of validating the lockdown accounts before they just start escalating them to us.

If we go down the endpoint protection route, then I could probably have other input after I've used that for a while.

The only thing I can think of that I would like to see, and I'm sure they could work this into a service pretty easily, is not only alerts on issues that are affecting my company, but some threat intelligence of a general nature on what's out there in the environment. That might be a nice add-in.

The biggest room for improvement is not necessarily in their service or offering, but in the products that they support. I would like them to further their knowledge and ability to integrate with those tools. They have base integrations with everything, and we haven't come across anything. They should just continue to build on that API interface between their applications and other third-party consoles.

The UI has become slower but it's not something I would call them out on. 

Our analysts do like getting into the console more than they like getting into the mobile app.

We have questioned them on the level of an alert and why alerts have come in lower than we would had anticipated them, e.g., it was maybe a medium instead of a high or medium instead of a critical.

We have a lot of homegrown applications, and we don't push a lot of those data sources to them. We are kind of a unique outfit in that way. So, there are some data sources that the service wasn’t able to integrate with. We're working on having the service be able to ingest them through a SIEM and provide us access right now. They will be storing some of those logs for us.

From a project management standpoint, better communication was needed with the customer during the setup/project phase. I have expressed that, and they have understood this. They have tried to make corrective actions.

Our infrastructure is very simple. The service covers almost all the endpoints, except that a service we use doesn't have a function that can control portable storage. It does scan everything, including whatever you have on a USB plugged into your computer. My suspicion is it will get there, but not right away. It doesn't have a special function to control the portable devices, and that's one thing I see lacking because sometimes we do have users who need this.

In terms of responsiveness, when I open up an alert, sometimes it takes a bit of time to load. However, it only happened once or twice. Most of time, it take just one click, then I'm there.

The dark theme might not be everybody's favorite. When I built the app for our users with a dark theme, everybody kept complaining. However, it's perfect for me and I like it a lot.

There is room for improvement with the new UI, and that's about it. I would like to see a more intuitive design.

The updated UI is actually pretty bad. Regarding the intuitiveness, it is fairly easy to use, but the responsiveness, on a scale of one to 10, is a one. It's really poor performance.

I have shared this next point with them already, but I would like to see a monthly report to talk about advancements or new alerts, anything to do with what we call IOCs — indicators of compromise. When there is anything that they have changed on behalf of their customers on the backend, they should say, "Hey, we have made these modifications. We're now looking at these types of alerts." It would give the customer a sense that they're actively looking for new IOCs. So I would like a monthly recap of what they have done, not specifically for me, but what they've done for all of their customers. That would be good.

We've had a little bit of frustration with some of the alerts that we receive because they're not as high-priority for our type of organization, as we are very engineering-heavy. But I can understand from their perspective, if a bank were a customer, or some other organization that doesn't have a lot of heavy engineering folks who are in a command-line and running all kinds of tools, the service would be much more valuable to them. But that's one of the main frustrations we've had: Trying to find ways to tune that out so that we can say, "Look, for this group it's normal for them to run a ping or Nmap or the like, but if accounting does it that's a problem.

Also, it has frustrated us that they don't have a native Slack integration, because most things do now. That's something we've asked for, for years, and it just doesn't really seem like it's a priority. The workaround is that we just have it sent to an email and you can email into Slack. Of course, email through Slack is not very good, but that's our workaround. We set that up ourselves.

Where CRITICALSTART could potentially grow is on its internal compliance, and maybe how they disclose how they secure data. All of that could be a little stronger. I pushed them on that early on, and they did provide some information, but like I'm doing with us — we're ramping up our compliance efforts too — that's where I'm likely going to have to push them in the future to make sure that they're at least meeting the minimums that we have, because they are seeing data from our employees.

They just did a user interface overhaul to the website portal that you use for troubleshooting tickets. The old one was fine. The new one is not intuitive and I hate it.

It's an information overload issue. When you go there, there is a bunch of stuff to look at. I had to get a walkthrough last week because I didn't know how to get to the one screen that I'm looking for when I use it, the one that shows the tickets that I have and the tickets that I don't have. I couldn't figure out how to get to that. In the middle of the main screen there's a little button that'll take you there. And at the top there's a search bar and a filter that helps you find tickets that are assigned to your organization or their organization, tickets that are open, tickets that are closed. But it's not intuitive.