Please share with the community what you think needs improvement with Vectra AI.
What are its weaknesses? What would you like to see changed in a future version?
The solution's ability to reduce false positives wasn't very good, initially, because it was picking up so much information. It took the investment of some time and effort on our part to get the triage filters in place in such a fashion that it was filtering out the noise. Once we got to that point, then there was definitely value in time-savings and in percolating up the high-risk events that we need to be paying attention to. I'd like to be able to get granular reports and to be able to output them into formats that are customizable and more useful. The reporting GUI is lacking.
We would like to see more information with the syslogs. The syslogs that they send to our SIEM are a bit short compared to what you can see. It would be helpful if they send us more data that we can incorporate into our SIEM, then can correlate with other events. We have mentioned this to Vectra. It does some things that I find strange, which might be the artificial intelligence. E.g., sometimes you have a username for a device, then it makes another. It detects the same device with another name, and that's strange behavior. This is one of the things that we have with Vectra support at the moment, because the solution is seeing the device twice.
Integration with other security components needs improvement. It should have true integration as opposed to just being a separate pane of glass.
The solution’s ability to reduce false positives and help you focus on the highest-risk threats is mostly good. It is still a bit of work in process, but I can give feedback to the company from the help desk. There is follow-up from the Vectra team who follows it closely. We can also give a lot of inputs to make it still a better product. It's already a very good product, but in comparison with a lot of systems I used in the past, the false positives are really a burden because they are taking a lot of time at this moment. The Office 365 integration is still a pretty new feature. I also have seen some improvements, and they email us with every step in the improvement process. I think that this integration will grow. Every area has room from improvement. Security is an ongoing process. It is important for Vectra to keep updating their system based on new behaviors. We would like to see the combination of the cloud with on-premise, e.g., what's happening in the cloud versus what's happening in the on-premise situation. If there is a phishing mail in the cloud, then the phishing mail comes in and a colleague clicks on that mail. Normally, it would be blocked by the system. However, when it's not blocked, then there can be malware on the system locally. We think it's important to get the integration of what's happening on Office 365 with phishing mails. Sometimes, it is a bit noisy on the dashboard because all the systems are on one field. On the dashboard, we have a complete overview of high, medium, and low risks. However, it would be more interesting for us if they could split that dashboard into high, medium, and low devices. For example, there is a dashboard on a device with a complete overview specifically for high-risk.
Vectra is still limited to packet management. It's only monitoring packet exchanges. While it can see a lot of things, it can't see everything, depending on where it's deployed. It has its limits and that's why I still have my SIEM. I am in contact with the Vectra team, if not weekly then on a monthly basis, to propose improvements. For the time being, the main improvement I can see would be to integrate with more external solutions. Since Vectra provides an API, that should be quite easy to handle. For example, we're using an open source ticketing system within our team and I want to have it handled properly by Vectra. We'll go forward on that with the API. Another area for improvement that I have pinpointed is that the Office 365 solution and the Detect solution cannot match the same users. That means we have two "different worlds" currently, the world from Office 365, which is bringing alerts based on users' emails and email addresses. And we have the network world, which is bringing an Active Directory view. On the one hand we are seeing emails or email addresses, and on the other hand we are seeing things like logons on to the domain controller. From time to time, it does not match and the tool cannot currently cross-check this info and consolidate everything. I would like to be able to see that detection related to one workstation and covering a user: what he is using, what services he is using, and what he did with his Office 365 and configuration. That would help. Another major feature would be to have all logs pushed to Cognito Detect, and all these logs should be also pushed to Recall. Currently, within Recall, I can't call up the Office 365 detections and I would love to do so. The last point would be an automated IoT threat feed consumption by the tool.
The false positives and the tuning side of it are some things that could use improvement but that could be from our side. I don't want to criticize the product for performance with our role out of it. It does what it says it's going to do very well. We've got issues with the way we've deployed it in some places, but the support we've had in that is very good as well, so I'm very happy with the support we get.
I would like to see a bit more strategic metrics instead of technical data. Information that I could show to my executive management team or board would be valuable. I would like to see some improvements on the integration aspects of it. They are getting better in this. However, most organizations have a plethora of cybersecurity solutions that they run, and I think that there is a bit more that could be done on the integration side.
It does a little bit of packet capture on alert so you can look at the packet capture activity going on, but it doesn't collect a whole lot of data. Sometimes it's only one or two frames, sometimes it does collect more. That's why they have the addition of their Recall platform, because that really does help expand the capability. I would also like to see more documentation or user guides about using the product.
Some of their integrations with other sources of data, like external threat feeds, took a bit more work than I had hoped to get integrated. I think the company has been very responsive, willing to take our feedback, and look at addressing our concerns. I have asked that they give direct packets capabilities.
Some of the customization could be improved. Everything is provided for you as an easy solution to use, but working with it and doing specific development could be worked on a bit more in the scope of an incident response team. In my opinion, it's built as a solution for everything, instead of it being part of a bunch of other tools. For example, we have a source solution which will orchestrate the ability for us to use a host EDR and the ability for us to use Vectra. We see Vectra from a purely network standpoint. Therefore, we don't want it to be the incident manager where we have to fill in specific things to be fixed. We think the integration with source solutions could be better. It tries to treat itself as an incident resolution platform.
One thing which I have found where there could be improvement is with regard to the architecture, a little bit: how the brains and sensors function. It needs more flexibility with regard to the brain. If there were some flexibility in that regard, that would be helpful, because changing the mode of the brain is complex. In some cases, the change is permanent. You cannot revert it. I would like to see greater flexibility in doing HA without having to buy more boxes just to do it. Another area they could, perhaps, look at is with OT (operational technology) specifically. Vectra is very specific to IT-related threats. It really doesn't have OT in its focus. We are using another tool for that, but maybe that is another area they can consider venturing into. It's being used by my team of four or five people. Once we hand it over to operations, then the team size will increase significantly. It will grow to about 10 to 15 people.
You are always limited with visibility on the host due to the fact that it is a network based tool. It gives you visibility on certain elements of the attack path, but it doesn't necessarily give you visibility on everything. Specifically, the initial intrusion side of things that doesn't necessarily see the initial compromise. It doesn't see stuff that goes on the host, such as where scripts are run. Even though you are seeing traffic, it doesn't necessarily see the malicious payload. Therefore, it's very difficult for it to identify these type of host-driven complex attacks. It only shows us a view of suspicious behaviours. It doesn't show us a view of key or regularly attacked company targets. This could be because we don't have one of the other tools or products that Vectra provides, such as Stream or Recall. My challenge with the detection alerting platform, Cognito, is it tells us this host is behaving suspiciously and is targeting these other machines, but it won't give you a view when a host is the target of multiple attacks. This because you may have a key assets, such as domain controllers or configuration management servers. These are key assets which may get targeted. If you're a savvy attacker, you spread out your attack across multiple sources to try and hide them across the network. That is where the solution falls a bit short. It is trying to build that chain of relationships across detections and also trying to show detections from a perspective of a victim rather than the perspective of an attacker. I have expressed these concerns to Vectra and they are currently in as feature requests. There is another feature in place which takes additional data feeds, such as DHCP IP allocation data. Their inputs are taken from Windows event logs, and that's the format they have in place. They use that to provide them with a more accurate view of host identities. If you are only relying on IP addresses, and IP addresses change over time, it's sometimes very difficult to show a consistent view of a system behaviour over time, as the IP can change per month. Unfortunately, because their DHCP data is taken from Windows host events and our DHCP data is taken from a Palo Alto system that generates the IP leasing, the formats are incompatible. I think taking different formats for that type of data is something else we have a feature request in for. At the moment, we don't have an accurate view, or confidence, that they are resolving when an IP address changes from host to host. So, we may be missing an accurate view of risk on some of those hosts. We also have the same problem with VPN and Citrix. E.g., if you're on the network and on IP address A, then you come in via the VPN, you're now on IP address B. Thus, if you're spreading your suspicious behaviour across both the internal network and VPN, then across Citrix, we don't get to join all that information up. They are seen as three different systems, so it causes a bit of a problem trying to correlate that type of event data.
Room for improvement depends on how their strategy and roadmap develops, as they have a lot of third-parties that they integrate with, e.g., more orchestration around what alerts and what to do with afterwards. They don't pretend to be working in that space. That is a third-party type activity. There are always the little things that they could do a bit better, like grouping or triage filters. Clearly, they've taken that onboard and developed those over the course of the last 18 months to two years to put these additional functions in. My guys are constantly saying, "Oh, it'd be useful to do this and useful to do that." The solution has not reduced the security analyst workload in our organization because we still need to SIEM. Unfortunately, while Vectra, for us, is a brilliant tool for network investigations, giving wonderful visibility, it doesn't go the whole way to replace our SIEM that is needed for compliance. So, I still have the same amount of alerting and logging that I did before. It gives us more defined ability to see incidents, but it doesn't give us enough information to satisfy a PCI or 27001 audit.
What do you like most about Vectra AI?
Thanks for sharing your thoughts with the community!