What is our primary use case?
Software Composition Analysis (SCA) is used to detect vulnerabilities in open source libraries, which are used by our customers for their own product.
We are a consulting company who provides consulting services to clients. We don't buy the software for our own internal use. However, we advise customers about which solutions will fit their environment.
Most of our clients use SCA for cloud applications.
How has it helped my organization?
For application security, the SCA product from Veracode is a good solution. It has a good balance. Altogether, the balance between the outcome of the tool, the speed of the tool, and its cost make it a good choice.
One of the reasons why we recommend Veracode because it is very important in that SAST and SCA tools, independently from the vendor, should work seamlessly within the build pipeline. Veracode does a good job in this respect.
In this day and age, all software is developed using a large amount of open source libraries. It is kind of unavoidable. Any product application has a lot of embedded libraries. In our experience, many times customers don't realize that it is not just a code that can be vulnerable, but also an open source library that they may take for granted. In many ways, this has been a learning experience for the customers to understand that there are other components to open source libraries, and that SCA is an invaluable tool to address those issues.
What is most valuable?
SCA provides guidance for fixing vulnerabilities. It provides extensive guidance for both writing secure code and pointing to vulnerable open source libraries are being used.
From the time it takes for the solution to detect a vulnerability, both in the source code and the open source library, it is efficient.
Within SCA, there is an extremely valuable feature called vulnerable methods. It is able to determine within a vulnerable library which methods are vulnerable. That is very valuable, because in the vast majority of cases where a library is vulnerable, none of the vulnerable methods are actually used by the code. So, if we want to prioritize the way open source libraries are updated when a library is found vulnerable, then we want to prioritize the libraries which have vulnerable methods used within the code.
The Static Analysis Pipeline Scan is faster than the traditional scan that Veracode has. All Veracode products are fast. I have no complaints. On average, a piece of code for a customer takes 15 to 20 minutes to build versus the Static Analysis Pipeline Scan of Veracode that takes three or four minutes. So, that is 20 to 30 percent of the total time, which is fairly fast.
What needs improvement?
Most of our time is spent configuring the SAST and SCA tools. I would consider that one of the weak points of the product. Otherwise, once the product is set up on the computer, it is fairly fast.
Like many tools, Veracode has a good number of false positives. However, there are no tools at this point in the market that they can understand the scope of an application. For example, if I have an application with only internal APIs and no UI, Veracode can detect that. It might detect that the HTML bodies of the requests are not sanitized, so it would then be prone to cross-site injections and SQL injections. But, in reality, that is a false positive. It will be almost impossible for a tool to understand the scope unless we start using machine learning and AI. So, it's inevitable at this point that there are false positives. Obviously, that doesn't make the developers happy, but I don't think there is another way around this, but it is not just because of Veracode. It's just the nature of the problem, which cannot be solved with current technologies.
Once we explain to the developers why there are false positives, they understand. In Veracode, embedded features (where there are false positives) can be flagged as such. So, next time that they run the same scan, the same "vulnerability" will be still flagged as a false positive. Therefore, it's not that bad from that point of view.
Veracode has a few shortcomings in terms of how they handle certain components of the UI. For example, in the case of the false positive, it would be highly desirable if the false positive don't show up again on the UI, instead still showing up for any subsequent scan as a false positive. There is a little bit of cluttering that could be avoided. However, that is not necessarily a shortcoming of the product. I think it's more of a shortcoming of the UI. It's just the way it's visualized. However, going forward, I personally don't want to see any more vulnerabilities that I already flagged as a false positive.
It does take some time to understand the way the product works and be able to configure it properly. Veracode is aware of that. Because the SCA tools are actually a company that they acquired, SourceClear, the SCA tool and SAST tool are not completely integrated at this point. You are still dealing with two separate products, which can cause some headaches. I did have a conversation with the Veracode development team not too long ago where I voiced my concerns. They acknowledged that they're working on this and are aware of it. Developers have limited amounts of time dedicated to learning how to use a tool. So, they need quite a bit of help, especially when we're talking about this type of integration between the SAST and SCA. I would really like to see better integration between the SAST and SCA.
For how long have I used the solution?
I have been using it for almost a year.
What do I think about the stability of the solution?
It is stable. One of the selling points is that it is a cloud solution. The maintenance is more about integrating Veracode into the pipeline. There is a first-time effort, then you can pretty much reproduce the same pipeline code for all the development teams. At that point, once everything runs in the pipeline, I think the maintenance is minimal.
What do I think about the scalability of the solution?
We have deployed the solution to FinTech or technology medium-sized companies with more than 100 employees.
How are customer service and technical support?
Their technical support is less than stellar. They have essentially two tiers: the technical support and the consulting support. With the consulting support, you have the opportunity to talk to people who have intimate knowledge of the product, but this usually takes a bit of effort so customers still like to go through the initial technical support that is less than stellar. We rarely get an answer from the technical support. They seem a lot more like they are the first line of defense or help. But, in reality, they are not very helpful. Until we get to the second level, we can't accomplish anything. This is another complaint that I have brought up to Veracode.
Which solution did I use previously and why did I switch?
One of the reasons why we decided on Veracode is because they have an integrated solution of SAST and SCA within the same platform. Instead of relying upon two different, separate products, the attraction of using a Veracode was that we could use one platform to cover SAST and SCA.
How was the initial setup?
The SAST tool is pretty straightforward; there is very little complexity. The pipeline works very well. The SCA tool is more complex to set up, and it doesn't integrate very well with the SAST tool. At the end of the day, you have essentially two separate products with two separate setups. Also, you have two different reports because the report integration is not quite there. However, I'm hopeful that they are going to fix that soon. They acquired SourceClear less than two years ago, so they are still going through growing pains of integrating these two products.
The setting up of the pipeline is fairly straightforward. It works a lot of the main languages, like Java, Python, etc. We have deployed it across several development teams. Once we create a pipeline and hand the code to the developers, they have been able to make a little adjustment here or there, then it worked.
What about the implementation team?
For both SCA and SAST tools, including documentation, providing the code, writing the code for the pipeline, and giving some training to the developers, a deployment can take us close to two weeks.
Deploying automated process tools, like Veracode, Qualys, and Checkmarx, does take more effort than uploading the code manually each time.
What was our ROI?
As long as developers use the tool and Veracode consistently, that can reduce the cost of penetration testing.
What's my experience with pricing, setup cost, and licensing?
Checkmarx is a very good solution and probably a better solution than Veracode, but it costs four times as much as Veracode. You need an entire team to maintain Checkmarx. You also need on-premise servers. So, it is a solution more for an enterprise customer. If you have a small- to medium-sized company, Checkmarx is very hard to use, because it takes so many resources. From this point of view, I would certainly recommend for now, Veracode for small- to medium-sized businesses.
Compared to other similar products, the licensing and pricing are definitely competitive. If you see Checkmarx as the market leader, then we are talking about Veracode being a fraction of the cost. You also have to consider your hidden costs: you need a team to maintain it, a server, and resources. From that point of view, Veracode is great because the cost is really a fraction of many competitors.
Veracode provides a very good balance between a working solution and cost.
Which other solutions did I evaluate?
There are other products in the market. However, some of those products are extremely expensive or require a larger team to support them. Often, they have to be installed on-prem. Veracode is a bit more appealing for our organizations who don't have larger AppSec teams or where budget is a constraint. In this respect, SCA is a good solution.
We have been using Checkmarx for years, but mainly for their on-prem solution. They do have an offering in the cloud, but we haven't done any side-by-side tests in respect to speed. We did do a side-by-side comparison between Veracode and Checkmarx two or three years ago from a technical ability standpoint. At that time, Checkmarx came in a bit ahead of Veracode.
Checkmarx is more complex to set up because it is on-prem with multiple servers as well as there are a lot of things going up. If you have a larger budget and team, look into Checkmarx because it is a market leader. However, when it comes to a price, I would choose Veracode for a smaller company, not a large enterprise.
Another consideration for Checkmarx, as an on-prem solution, is that you are pretty much ascertained that your code doesn't leave your company. With companies like Veracode, even if they are saying that you only upload the binary code, that's not quite true. The binary code can be reverse-engineered and the source code can be essentially reconstructed. For example, Veracode would not be suitable for a government agency or a government consultancy.
For DAST, our customers like to use Qualys Web Application Scanning. There are very few players out there that can test APIs, but Qualys is one of them.
Another promising solution that allows for testing APIs is Wallarm. We have done a couple of PoCs with them.
We tested Black Duck a few years ago, but they only had a SCA solution. They didn't have a SAST solution. I think they do now have a SAST solution because they acquired another company, Fujita.
What other advice do I have?
I don't think that Veracode has helped developers with security training, but it helps developers have a reality check on the code that they write and their open source library. That is the best value that developers can get from the product.
Veracode products can be run as part of the development pipeline. That is also valuable.
It integrates with tools like GitHub or Jenkins. At a high level, it does integrate with most of the pipeline of tools. It would be a showstopper if the incorporation of security was not in the developer workflows. We are past a time when developers or software engineers run a SCA or DAST scan on the code, then hand it off to the development team. What works instead is to inject a security tool in a development pipeline, which is why it is absolutely paramount and important that tools, like Veracode, be a part of the build pipeline.
We limited the user to SAST and SCA. We haven't used any of the penetration testing, especially for the DAST solution that they have. For that, they are behind the curve, meaning that there are other products in the market that are being established. In my opinion, they don't have a viable product for DAST, because I believe they are not even testing APIs. So, it's not mature enough. We also have never used their pen testing because that is one of the services that we provide.
At this point, Veracode is one of the best solutions available, though it's not perfect by any means, but you have to work with whatever you have.
I will give the solution a seven (out of 10). When they integrate the SCA and SAST portions more tightly together, I could probably bump it up to an eight. Also, if they make improvements to the UI and the support, they can get a better rating. However, at this point, I would still pick Veracode for a company who doesn't have a million dollar plus budget.
Which version of this solution are you currently using?