What is our primary use case?
We have a couple of servers on-premises to gather the logs from our devices. We have a lot of devices including vendor-agnostic collectors that will, for example, collect syslogs from our Linux host. The logs are then sent to the Devo Relay, which encrypts the data and sends it to the Devo Cloud.
What we send to Devo includes all of our Unix-based logs. These are the host logs, as well as logs from a lot of the network devices such as Cisco switches. Currently, we are working with Devo to set up a new agent infrastructure, and the agents will collect Windows event logs.
We were using a beta product that Devo provided for us, which was based on an open-source platform called Osquery. That did not quite work for the volume of logs that we have. It didn't seem to be able to keep up with a large number of servers, or the large amount of Windows event log volume that we have in our environment. We're currently working with them to transition to an Xlog and use their agents, which work really well to forward the logs to Devo.
We also send cloud logs to Devo, and they have their own collector that handles a lot of that. It basically pulls the logs out of our cloud environment. We are sending Office365 management logs, as well as a lot of Azure PaaS service logs. We're sending those through an event hub to Devo. We are currently working on onboarding some AWS logs as well.
We have several corporate locations, with the main location in the US. That is where the majority of our resources are, but we do also have Devo relays stood up in Canadian, Australia, and India. These locations operate in a way that is similar to what is described above, although on a smaller scale. They're sending all of their Unix devices and syslogs to the relay, and then I believe only Australia at the moment is using agents to pull from Windows logs. Canada is using a different SIEM at the moment, although that contract is about to expire, so then we'll onboard their Windows event logs as well. India does not have any Windows servers that need to have an agent for collecting logs, so just send the Linux and Unix logs over the relay to Devo.
Our main use case and customer base are our security operations center analysts. A lot of our process was built up and carried over from our previous SIEM, LogRhythm. We have an alerting structure built out that initiates a standard analyst workflow.
It starts when you get an alert. You drill down in the logs and investigate to see if it's a false positive or not.
We are in the process of onboarding our internal networking team into Devo, and we are gathering a lot of network logs. This means that they can monitor the health of our networking infrastructure, and at some point, maybe set up health alerts for whatever they are looking for.
We have another team that is using Devo, which is our internal fraud team. They're very similar to stock analysts, where they just look for suspicious events. They are especially interested in tax filing and e-filing. We gather logs for that, and they go through a really deep investigative workflow.
How has it helped my organization?
One of the immediate improvements that come to mind is the amount of hot, searchable data. In the SIEM we had before, we were only able to search back 90 days of hot, searchable data, whereas here we have 400 days worth. That definitely has improved our threat hunting capabilities.
We're also able to ingest quite a bit more data than we were before. We're able to ingest a lot of our net flow data, which if we had sent that to our previous SIEM would have brought it to its knees. So the amount of data that the analysts are able to see and investigate has been a really big beneficial use case. I'd say that's the biggest benefit that it's provided.
I myself do not leverage the fact that Devo keeps 400 days of hot data to look at historical patterns or analyze trends. A lot of times I will look at that to see the log volumes, the traffic, make sure there are no bottlenecks as far as how log sources are sending to Devo. I would say that the analysts definitely for certain cases will go back and try to retroactively view where a user was logging in, for example. At the moment, we haven't really had a use case to push the limit of that 400 days so to speak, and really go really far back. We definitely use the past couple of months of data for a lot of the analyst cases.
This is an important feature for our company especially with the recent SolarWinds attack, which was a big deal. We did not have Devo available, but because that happened so far in the past, it was a struggle to pull that data for it to look for those IOCs. That was definitely a really big selling point for this platform with our company.
Devo definitely provides us with more clarity when it comes to network endpoint or cloud visibility. We're able to onboard a lot of our net flow logs. We are able to drill down on what the network traffic looks like in our environment. For the cloud visibility, we're still working on trying to conceptualize that data and really get a grasp around it to make sure that we understand what those logs mean and what resources they're looking at. Also, there's a company push to make sure that everything in the cloud is actually logging to Devo. As far as cloud visibility, we as a company need to analyze it and conceptualize it a little bit more. For network visibility, I would say that Devo's definitely helped with that.
The fact that Devo stores the data raw and doesn't perform any transformation on it really gives us confidence when we know that what we are looking at is accurate. It hasn't been transformed in any way. I'd definitely say that the ability to send a bunch of data to Devo without worrying about if the infrastructure can handle it definitely allows us to have a bigger and better view of our environment, so when we make decisions, we can really address all the different tendencies. We're collecting a lot more types of log sources than we were before. So we can really see all sides of the issue; the vast amount of data and the ability to really take our decision and back it up with the data, and not just random data but we can use a query and display the data in a way that backs up the decision that we're making.
Devo helps to release the full potential of all our data. The active boards like the interactive dashboards that Devo provides really help us to filter our data, to have a workflow. There are a lot of different widgets that are available for us to visualize the data in different ways. The active boards can be a little slow at times, a little bit difficult to load, and a little bit heavy on the browser. So sometimes the speed of that visualization is not quite as fast as I would like but it's balanced by the vast amount of options that we have.
That's one of the big things that like all security companies, security departments really purported having that single pane of glass. The Devo active boards really allow us to have that single pane of glass. That part is really important to us as a company to be able to really visualize the data. I haven't found the loading speeds have become a significant roadblock for any of our workflows or anything, it's an enhancement and a nice to have.
We all want everything faster, so it's definitely not a roadblock but the ability to represent the data in that visualized format is very important to us. It's been really helpful, especially because we have a couple of IT managers, non-technical people that I am onboarding into the platform because they just want to see an overall high-level view, like how many users are added to a specific group, or how many users have logged in X amount of days. The ability to provide them not only with that high-level view, but allow them to drill down and be interactive with it has really been super helpful for us as a company.
Devo has definitely saved us time. The SIEM that we were on before was completely on-prem, so there were a lot of admin activities that I would have to do as an engineer that would take away from my time of contextualizing the data, parsing out the data, or fulfilling analysts requests and making enhancements. The fact that it is a stock platform has saved me a ton of time, taking away all those SIF admin activities.
I wouldn't say that it really increased the speed of investigations, but it definitely didn't slow it down either. They can do a lot more analysis on their own, so that really takes away from the time that it takes to reach out to other people. If you went back 90 days, you had to go through a time-consuming process of restoring some archives. The analysts don't have to do that anymore, so that also cuts off several days' worth of waiting. We had to wait for that archive restoration process to complete. Now it's just you pull it back and it's searchable. It's right there. Overall, I would say Devo has definitely saved us a lot of time. For the engineering space, I would say it saves on average about one business day worth of time every two weeks because a lot of times with on-prem infrastructure, there would be some instances where it would go down where I'd have to stay up half the night, the whole night to get it back up. I haven't had to do that with the Devo platform because I'm not managing that infrastructure.
What is most valuable?
We are using some of the other components, such as Relay, which is used to help us ship logs to Devo.
The most valuable feature is definitely the ability that Devo has to ingest data. From the previous SIEM that I came from and helped my company administer, it really was the type of system where data was parsed on ingest. This meant that if you didn't build the parser efficiently or correctly, sometimes that would bring the system to its knees. You'd have a backlog of processing the logs as it was ingesting them.
One thing that I love about Devo is that you can accept the data in a raw format. It's not going to try to parse it until you query it. This makes it really flexible for us because if the analysts come to us and explain that they need a specific log source, we can just work on the whole transportation system, insofar as how to get it to Devo. We don't have to worry about parsing it out until later. We can actually see the data in the platform and then we can use the queries to perform contextualization on it, parsing out whatever metadata we need.
I really like the flexibility that the queries offer to parse out the data. Parsing out JSON logs, for example, is very easy. You don't have to mess with regex. It's literally just a point-and-click interface. So that has been incredible. I would say overall in a nutshell, one of my favorite parts is that they really have captured the essence of sending us all your data. You don't have to worry about how to parse it. You can get the data onboard and then you can perform transformations on it later. And the transformations that you can perform on it are super flexible.
Devo definitely provides high-speed search capabilities and real-time analytics. The search can be a little bit slow at times. But for the amount of data that we're pulling back relatively speaking, I would say that the speed is very nice. The ability to pull back large amounts of data, also the amount of data that they keep hot and searchable for us is incredible. I would definitely say that they provide real-time analytics and searching.
I have heard from other customers that the multi-tenancy capabilities are pretty good, but I don't have much experience with that in the HR Block though.
What needs improvement?
When it comes to the ease of use for analysts, that's an area that they may need to work on a little bit. Devo offers its version of a case management platform called Devo SecOps. They did offer it to us. It's part of our contract with them. The analysts have found that the workflow isn't very intuitive. There are a couple of bugs within the platform, and so we are actually sticking with our old case management platform right now and trying to work with Devo to help iron out the roadblocks that the analysts are facing. Mostly it seems like they have trouble figuring out where the actual case is. A lot of the search features that are in the main Devo UI don't translate over into their SecOps module. They seem separate and disjointed. So the core of the platform where we have all of the data isn't integrated as well as we would like with their case management system. There's a lot of pivoting back and forth and the analysts can't really stay in the SecOps platform which adds some bumps to their workflow.
The SecOps module also needs improvement. It should be more closely integrated with the original platform that they had. The data search abilities in the SecOps platform should be made more like the data search abilities in the administrator's side of the platform.
From our experience, the Devo agent needs some work. They built it on top of OS Query's open-source framework. It seems like it wasn't tuned properly to handle a large volume of Windows event logs. In our experience, there would definitely be some room for improvement. A lot of SIEMs on the market have their own agent infrastructure. I think Devo's working towards that, but I think that it needs some improvement as far as keeping up with high-volume environments.
For how long have I used the solution?
We implemented Devo as a PoC last year but have only just started using it officially a few months ago.
What do I think about the stability of the solution?
It's a Devo-managed SaaS cloud platform. It does seem like lately that they've been having trouble keeping up with the large volume of events. It's maybe due to other customers besides H&R Block. I have shared in their cloud infrastructure and we have noticed some slowness and some downtime. I would say it's definitely not more than a maximum of three hours every two weeks. It's usually not a lot of downtime or slowness, at least not to the point where we cannot work within the platform, but it does seem to have been picking up a little bit more lately. That's why I average out around three hours every two weeks. But I would say as far as overall stability, the uptime has been really great. If it's "down", it's really more just that search is run really slow, but you can still get into the platform. It's not really that everything is down where you can't look at alerts. That rarely ever happens. I would say overall, it's pretty stable and it allows our analysts to stay on the platform.
What do I think about the scalability of the solution?
In terms of scalability, we are able to ingest as much or as little data as we want, so that is really awesome. I've been pretty amazed at how much we're able to throw at it. We can expand as much as we want to suit our needs, obviously within the confines of the subscription agreement. There is a data cap, but within that limit, we can really go crazy. The scalability is awesome. It's very scalable.
There are about 50 or so users on the platform right now. We have our SOC analysts at different levels that just perform investigative activities. The majority of our clients on the platform are our security operations center analysts. They have different privileges based on their roles. We give them the ability to create test alerts if they're trying something out.
We have various other team members throughout our corporation using it, only two or three here and there. We have three individuals from our networking team, a couple of individuals from IT support that often utilize the platform to investigate user lockout and stuff like that. Of course, we have the engineers in the platform which have been five or six individuals. The main user base is our SOC analysts.
We do maintenance for our servers and such. We don't really have them on the platform at the moment. They have their own kind of tools. They utilize their graphs to monitor the health of our infrastructure, but that may be something at some point in the future that we may be pursuing. The more teams that we get into our SIEM, the better because it really justifies the usage of the tool. Right now, as far as from a maintenance perspective, the only IT staff that would be using it for that sort of thing would be our networking team. And we have about three individuals that we just recently onboarded, so they're just getting used to the platform.
Devo is mostly being used for security logs. There's a push to start using this not only for security monitoring but for infrastructure health monitoring as well. So we're starting with the networking team. We really are still in phase one of really fleshing Devo out, adding more enhancements and alerts. My primary role is to support the security operations center, to support the security aspect of things but I haven't heard if it's set in stone yet. I would say that we are definitely going to try to push to utilize Devo more throughout our organization for health monitoring and for the networking team to use. Perhaps at some point in the future, we would expand our usage. It's not set in stone yet, but I could definitely see that happening.
How are customer service and technical support?
We have a couple of tickets open with support but mostly for platform health monitoring questions. We do have some in regards to alert logic, but nothing that was super important that we actually had to call Devo tech support.
Their professional services team works really hard on building out active boards for us and helping us make sure that we are monitoring the health of our platform. Overall, I'd say they're definitely really collaborative. They want to hear what's going well for us and what's not.
Sometimes they're not quite as responsive as I would like, but I think that is also due to the transition process because that was recent. We are giving them some time to get adjusted to our account and get things all set. I would say from a support standpoint, they have definitely been very responsive to our cases that we have open, so that's been really helpful. I've had instances in the past with vendors where it takes several weeks to hear anything back on a case.
Which solution did I use previously and why did I switch?
Prior to Devo, we used the LogRhythm SIEM.
We switched mainly because of the ability to ingest more data. In certain instances, we had to say no to onboarding certain log sources because of the amount of value it offered, the cost-benefit didn't weigh out. LogRhythm put the point where if you added too much data, if you had too much volume being ingested, it would start breaking. It would start complaining and things would just go bad. The amount of downtime we had with LogRhythm was really the main metric driver to get us to transition to Devo. Then what really appealed to us about Devo versus other SIEMs was their "give us all your data" model. That was something we were really struggling with and that was something that we really wanted from a SIEM. We wanted to correlate between as many data sources as possible. They offered us that capability that LogRhythm really did not.
How was the initial setup?
The initial setup was fairly straightforward. I don't know if this falls into the whole analysis of the Devo UI itself. I'm not sure if Devo considers their agent infrastructure as part of this offering because it was beta, but that was rather complex. We didn't get a lot of details as to the specs needed for when we set up the agent manager infrastructure in our environment, so that was pretty complicated. But as far as just onboarding the data, really offloading all of the alerts that we had out of our old SIEM, it was pretty straightforward.
I would definitely say one thing that may have made it easier would be for Devo to have some more out-of-box alerts. The previous SIEM that we had, had a lot of alerts that they offered us from their research and from their labs and we really built on top of those. We had to build a lot of our alerts from scratch to transition them from our old SIEM to Devo. If Devo had their own alert library or a more fully fleshed-out alert library, that would have made that aspect of it a little less time-consuming. Otherwise, as far as the data onboarding and the data ingestion, that was very straightforward as far as SIEM transitions go. The relay they provided us gave us a single point to send everything to.
We really shifted completely from our old SIEM to Devo in about three to four months, which by industry standards was very quick. It was a combination of a lot of hard work and teamwork on our side, but again, the data ingestion, the ease of getting that data into Devo took off a lot of that time. We had a single point to send it to which helped with the transition.
In terms of our implementation strategy, our initial goal was to get everything out of our old SIEM. So first we made sure that all of the log sources that were there would be redirected to Devo. And so we set up the appropriate components to forward it to Devo, and then once all the data was in there we started working on transitioning the alert and building out the alert logic, and making sure that the alert logic matched what we had in our old SIEM. After that was onboarding all the users, making sure that the RBAC controls were in place.
What about the implementation team?
We did use a consultant. We worked with Novacoast. We had a relationship with them in the past. They're an MSSP. They really helped us with building the alert logic mainly.
What's my experience with pricing, setup cost, and licensing?
You definitely get what you pay for. Devo has offered a lot of extra features to justify the price. Devo worries about managing the infrastructure and how it's going to handle that volume, how it's going to store it, and all those things. It allows us to not require as many engineers and not require as many engineering hours. We can devote that time to other things. That's the biggest benefit for the cost.
I have seen in the Devo documentation that for certain aggregation tasks that you have running in the background, you could be charged extra for those. I've been meaning to get clarification on that.
Which other solutions did I evaluate?
We were considering Azure Sentinel, mainly because we're an Azure shop. That was mainly the only other solution we looked at. We chose which SIEM we wanted to have a POC with. Azure Sentinel didn't seem to offer as many features as Devo did, which is why we chose them for our POC. I think that was it. We sent it out to a couple of vendors but none of them fulfilled our needs as much as those two and then it was really just between Azure Sentinel and Devo. And Devo gave us a lot more features than Azure Sentinel did.
As far as from an analyst perspective, something that was unique about Devo versus other SIEMs was the immense contextualization capabilities they have because the platform is really based on that you query the data and you perform all these contextualizations in the query.
Another thing that we were thinking about was the learning curve with querying and using the UI. Devo's response in that learning curve was they definitely provided a lot of training for us. That really helped the analysts.
As far as our previous solution, it definitely allows us to ingest more data than we were able to with LogRhythm. I don't think that the others had 400 days of hot, searchable data. They did not have that much time available as hot and searchable. We definitely have the ability to ingest and store for longer periods of time and to ingest more data. That's really the big thing that is the next-gen capability of Devo that we were looking for was really unlike other SIEMs that I've seen or administered where you don't parse on ingest, you parse after you get the raw data in the platform. That really removes that roadblock.
What other advice do I have?
I have been with the company for approximately three years and in the engineering space for about two.
If the more data the better is the goal for your organization, then Devo is really the way to go for that. But if you're looking more for a super robust analyst interface, next-gen analyst workflow, I don't think Devo is at that point yet. They're more at the point where you can ingest a lot of data and perform visualizations on it really well.
One of the things that I really like about Devo is the ability to parse the data, and not just the ability to parse the data after you ingest it. There are so many different ways to do it.
I would definitely explore trying to parse that out yourself because, for me, the first couple of times it was a little bit difficult to get used to the query language and everything. But now, when someone asks for something to be parsed out in a certain way, it's super easy. Explore the ability to use the queries to parse out data to give you that independence and ability to represent data however you want to represent it.
Devo definitely has all the next-gen concepts that I haven't really seen in any other SIEM, but I do think that they definitely have some more room for improvement. A lot of SIEMs offer their own agent and Devo does not at the moment. I would rate Devo a seven out of ten.
Most of the stuff that we saw in our POC with them was the "wow" moment. This platform can address anything. All of the features met my expectations from the POC. As far as the onboarding and integration, it's definitely improved our workflow but the "wow" moment was when we had our proof of concept with them and saw what the platform initially could do, and then it really lived up to that.
Which deployment model are you using for this solution?
Which version of this solution are you currently using?