The solution is not in an optimal state. During POC, we analyzed tool is kept on upgrading. The patch deployment is happening in parallel, things that are working today are not working tomorrow. We eventually sorted it out with help of CSM.

We integrated this tool with other software such as Azure client, but many times without a valid or visible reason, the connectivity was breaking.

Improvement suggestions-  The dashboard creation for the reporting needs to be easier. Currently, the solution does not support multiple script executions and we would like to see support for this.

The solution lacks a little bit in the reporting features. Right now, the reporting test results that you want to read or you want to evaluate are pretty basic or too general. For example, If you want to check more deeply in an area that you found, hide responses, or maybe you want to tweak a little bit in how the results are shown, these are areas where the tool could improve. They are also missing the Microsoft diary application which would be useful if they could add in the future.

The performance of the tool needs to improve. Part of the problem is that this is a cloud-based deployment in a firewall setting, and our application will also be online. This means that there is an inside firewall and an outside firewall, so it is a different IP between connecting and executing it. Sometimes we feel that it is this aspect of the environment that is delaying the responses, rather than the actual application.

The reporting needs to be improved. They should include some additional reporting components because compared to other tools, like Sovereign, it does less.

Integration with monitoring tools, such as SolarWinds, needs to be improved. We can get performance testing information into our monitoring tools, but we can't map information about monitoring to the performance testing.