Tricentis NeoLoad Reviews

• Flexible license model
• Cloud integration
• Native mobile application testing

I had a requirement to test a mobile application for corporate that is used by users across the globe. I needed to do the test setup quickly to generate load across the region, and to do so with a solution that was cheaper than what we had. A NeoLoad license of four days with load generation from the cloud across regions helped to do the setup in three to four days and perform the testing.

It needs to support SAP GUI-based applications, which forms 60-70% of our portfolio, as well as Windows (client install) and Citrix-based applications.

I've used it for two years.

We had no issues deploying it.

There have been no stability issues.

We tested up to 1,000 users across four regions - North America, EMEA Latin America, and Asia-Pacific. It scaled sufficiently for us to be able to do that cross-regional testing.

It's excellent as the support is always available and helps us to troubleshoot any issues during execution with live support.

We were using HP Performance Center in-house for the previous six years. NeoLoad was tried as there was a need to test mobile applications with load generated from the cloud.

The initial setup is straightforward for both an in-house and a cloud one.

The implementation was done by an in-house team.

The licensing model is very flexible which supports a broad range of budgets, from small projects to enterprise-level setups.

Choose this if your portfolio is predominantly web and mobile. You need to reconsider if your portfolio has many SAP GUI-based applications.

This solution is mostly used by our customers to monitor the performance of the networks or of the server site.

We appreciate that this solution is very user-friendly, even if the user does not have a lot of protocol knowledge and experience.

We would like to see the addition of one-to-one integrations with the Tricentis Tosca suite to this product, which would then cover the end-to-end needs of our customers who are looking for a single vendor solution.

We have been working with this solution for 11 years.

We have found that the only stability issues, when using this solution, stem from the hardware in operation as opposed to the solution itself.

This is a fairly scalable solution.

We have found this solution to be very time saving for our customers, who report optimal resource usage since its implementation.

The licensing for this solution is renewable yearly, and covers all available features and technical support.

We would rate this solution a six out of ten.

There was a requirement from the project itself. So when I was onboarded to the project, I learned it myself and then continued on with NeoLoad.

It is for an Agile project. 

In the application, if it's a web application, I could see so many requests will be recorded. It will be a little clumsy with many request correlations and many things that need to be handled in JMeter since it consists of sub-requests and internal requests. So many things will be recorded, actually. 

But when it comes to NeoLoad, the maintenance will be easy. Then, whatever is required, those requests will be recorded. For example, in the same flow, JMeter, I could see maybe 35 to 40 requests. In NeoLoad, I could see only 20, but it won't miss any requests. It will bundle them as a single request. And then, it's pretty much straightforward for a performance tester, basically, to go through, and it is time-saving while using NeoLoad. The interface is easy.

I really didn't work on the cloud-based [version]. NeoLoad still has a cloud [offering], and it has pretty good integration. I heard that it's possible to integrate with JMeter as a tool as well.

Maybe I could suggest: I wanted to know more about the integration with DevOps for performance testing. The automatic integration process – how can we run the scripts automatically within a CI/CD pipeline? 

So maybe I wanted to know how to integrate with DevOps, actually. I'm not sure whether that option is there with the tool or not. 

In future releases, it would be good if extra added features for integration are added into NeoLoad.

I have been using it for the past eight months somehow.

It is pretty scalable. Learning-wise, it's pretty straightforward and flexible because if the person has little knowledge of performance testing and the process, they can definitely easily grab the knowledge from NeoLoad. 

And if they're still working on protocols like web and web services, with a minimum amount of knowledge, they can be able to operate within less time. So it's pretty much flexible.

I worked on LoadRunner earlier in my experience. It's like I worked on LoadRunner for more than six years. 

However, LoadRunner is ultimately the leading tool in the market. Because as long as the pricing matters, it has the cloud and on-prem [options]. Separate components will be having a spectacular view, analyzing the results and everything. Individual components will have their own features. But NeoLoad, as well, cannot be that hard. It will be good as well after LoadRunner.

The customer service and support are good. We obviously write a ticket, and within a day or so, they will get back to us from Tricentis. They support us. Support-wise, it's similar to LoadRunner since they license the tool. 

So, obviously, we will have support if there are any critical blockers on the load generator, execution issues, or other certain things. They definitely give the support.

We basically directly contact through emails.

We work with JMeter and BlazeMeter, depending on the project requirements. 

NeoLoad is generally better than JMeter. 

It is easy to use. There's a particular process where we need to enter a license key to run even with a minimum amount of users. But LoadRunner has a 50 free license. In NeoLoad, as far as I know, we definitely need a license key to execute tests beyond a single user. So, the licensing setup is there.

There are certain steps that need to be followed to complete scripts and executions. But I feel it couldn't be that difficult to learn. It can be learned very quickly.

For the client-side... pricing obviously comes into the picture in our company. Maybe when compared to LoadRunner, NeoLoad has less costs. Compared to that, it's somehow affordable. They used LoadRunner earlier, and they switched to NeoLoad. 

So I've used JMeter and NeoLoad, but NeoLoad is pretty much good, better than JMeter. JMeter is a free, open-source tool.

I would recommend using this solution. If the person or if the company or someone cannot afford it... for example, if LoadRunner is a thousand dollars and this one [NeoLoad] is seven hundred dollars, something like that... if the client is genuinely looking at pricing and they don't have a high budget, there are budget issues in the project... In that case, if they cannot afford LoadRunner, NeoLoad is definitely preferable as another best-licensed tool, better than JMeter. JMeter is obviously open-source, but NeoLoad is better.

Overall, I would rate the solution a nine out of ten. 

We use the tool to do POC level activity.

I like the solution’s performance and integration. Also, the tool’s help center is very responsive and helpful. They have always helped me within a short duration of time.

The solution’s pricing is higher compared to other tools. Though the product’s reports are accurate, it needs to be more detailed like other tools.

I have been working with the solution for six months.

The tool’s setup is straightforward. The installation takes a couple of hours to complete.

I would rate the solution an eight-point five out of ten.

The primary use case is for web applications and also SAP GUI—not SAP Applications but the UI version of SAP. For both these areas, it was working fine from the NeoLoad side.

The solution has been moved to the cloud version.

The first benefit is that it's quite cheaper. In the market there are mainly three tools which dominate: JMeter, LoadRunner, and NeoLoad. JMeter is open source. LoadRunner is quite expensive but has quite good features. NeoLoad is competitively quite cheaper compared to LoadRunner, and it provides great features, except in some areas. For instance, it doesn't allow some SAP applications. There are some areas where it's a little bit tough but other than that, it's quite good.

I feel that the codeless part, the dynamic value capture part is quite easy in NeoLoad compared to other tools. JMeter is a little bit tough, but LoadRunner gives you many features. For any codeless value, it is easy to detect from the NeoLoad side.

The SAP area could be improved. Not the GUI applications, SAP log-on, or something like that. We can see features and use NeoLoad properly in all the normal SAP log-on areas. I haven't tried it, but I have heard that Citrix-based applications don't work properly in the SAP application. In those types of different areas, there could be more protocol flexibility.

If there is a high number of users on NeoLoad—like around 5,000 or 6,000—it sometimes feels like that might be a bottleneck for the performance. That could be improved. With more users, there shouldn't be any issues from the tool side.

With scripting in a different browser, sometimes there are compatibility issues and it doesn't generate the script properly. Those types of crashes could be removed.

I was using NeoLoad six months ago because of some performance testing issues. Previous to using NeoLoad, we used LoadRunner.

The stability is good. It's quite reliable. Compared with a different browser, it sometimes creates problems and bugs. With the number of users increasing, sometimes the CPU and system where it was connected increases because of trade concurrency or something like that. That can be a little bit problematic. Other than that, things are fine.

Ongoing maintenance is not required. Whenever we have deployed the tools, if any version upgrade happens, we just check what the upgrade is exactly and someone on the deployment team connects the documents and everything. They check the document accordingly and update the version. It's not complicated.

We have roughly eight to nine performance testers, including the performance lead, the performance engineers, and the performance testers. Everyone will be running different projects. We'll be using the tool and running the performance testing based on the license and everything. The user count license will be ramping up the load. There are instances where multiple teams are using NeoLoad at the same time with different licenses. There has been no conflict error with each other, and we haven't seen a problem with different license things. Because we have two licenses, two tests can be executed parallel to each other.

The solution is scalable. After the number of users increases to 5,000 or 6,000, we have occasionally found problems. Below that, things are fine and the response times are good.

Tech support is okay. When we had 50 users, we reached out to the tech support team and within two to three days we received a response. The time took longer because it was a complicated problem. Response time is perfect.

I previously used LoadRunner. One of the reasons we had to switch to Neotys was because it was quite cheaper, but I still prefer LoadRunner to NeoLoad.

The deployment process is very straightforward. It's not complex. Only one associate deployed it, so not much effort was required.

There was no impact on the work that we used to do with performance testing when we moved to NeoLoad. Our normal performance testing was able to continue and it was going perfectly well. Return on investment was better because our deliverables were not getting impacted and it was cheaper, so return on investment increased a lot compared to LoadRunner.

NeoLoad is cheaper compared to other solutions. There are no additional licensing fees.

I would rate this solution 9 out of 10. I wouldn't give it a 10 because there are some issues that need to be resolved from the tool side.

Before using the tool, you need to understand the product you are going to design, the different types of performance applications that you are going to do performance testing with, and whether or not NeoLoad supports it. If yes, then go ahead. Otherwise, I would say go back to LoadRunner because whatever the application is, LoadRunner will support it.

• Graphical test creation
• Centralized test and results repository
• License management

With the simplicity of the test design interface, little domain knowledge is required for each scripted application, so we are able to generalize all of the performance testers to the point that any tester can update and execute any script instead of having individual domain experts who may be unavailable or busy. This has allowed us to keep more scripts updated easier than ever and keep a constant flow of performance tests in the pipeline, removing some dependence on human interaction.

If there was ever a way to bring the same level of ease to advanced scripting with Javascript to make it as easy as a regular test in NeoLoad, I would love to see it.

Some type of global test scheduler where different tests could be arranged on a calendar to automatically execute would help scheduling and provide visibility into the test environment.

Tighter integration with NeoSense and perhaps TFS integration to open defects for performance issues would also be nice.

I've used it for 12 months.

We've had no issues with deployment.

There have been no stability issues.

Running very large populations on one machine would max out NIC-level capacity far before CPU/memory would provide indication of bottlenecks, but running a distributed test alleviated this issue.

8/10

8/10

We previously used HP LoadRunner, and we switched due to cost, support competence, and the scripting interface.

It's straightforward to install and configure, but complex to convert scripts from a previous product to work in NeoLoad.

We implemented it in-house.

We've gotten 20% increase in tests-per-sprint and 35% decrease in time taken to create a script. We also have cost savings of over one million dollars annually.

Purchase a few extra licenses to allow for flexibility, utilizing the team server as much as possible.

• HP LoadRunner
• JMeter
• Appvance
• Rational Performance Tester

Take the time to see what NeoLoad can do when scripted from scratch and remember that it takes a bit of time to become an expert with the advanced features. Some complex actions may require written code, but there is very little that is actually impossible to test with NeoLoad. The limit is usually a function of experience with the software.

Wasted time in low-end work (scripting) is greatly reduced with NeoLoad. It enables my team to record and modify test scripts very quickly. This reduces the effort for testing and enables the team to do more high-value work like analysis of results. We use external data analytics tools (Tableau) for result analysis. The export raw data feature in NeoLoad provides the extension with Tableau.

Performance testing is done by testing frameworks in which NeoLoad is the driver of load. The ability to extend the tool is crucial for organisations. Our test framework is composed out of NeoLoad as a centralised component surrounded by Tableau for analytics and AppDynamics for deep-dive diagnostics. This load test framework enables the team to more quickly test, find and resolve issues and system bottlenecks.

The weakest feature of most of the load test tools is result analysis. It is important we can customise graphs to visualise results in such a way that stakeholders can understand the outcome of testing. To be able to visualise results better, we extended NeoLoad with Tableau Software.

With Tableau, you can centralize performance related data in a database. Data can come from any tool, from log files or RUM tools and system utilities. Tableau provides you with the ability to connect to any data source, analyse patters and find bottlenecks more easily.  NeoLoad has made huge improvements over the last few years and finally has the ability to view the raw data (and not only aggregated results). This is fundamental for any type of performance engineering.  Aggregated results hide bottlenecks. 

Customer support is very good.

The price of the tool is competitive.

The features introduced in the recently released new version of NeoLoad fit really well into the DevOps way of working, further reducing the time for rescripting. This enables rapid delivery and lean ways of working.

It gives us the ability to quickly create a script.

We are able to create load test scripts quickly in a fast paced environment, which in turn helps us identify performance issues.

The support needs to be improved, especially when you are looking for a quick turnaround because of the fast-paced environment we work in.

We've used it for five years.

We've had no issues with deployment.

It's rare, but sometimes the application will shut down.

We've had no issues with scalability. You just have to add additional hardware as you increase the number of users.

8/10.

We have a very good partnership. They are always willing to work with us and help resolve any issues or concerns we have. They keep us updated on new releases and features that will be delivered.

7/10.

We previously used HP LoadRunner, but it was not easy to create scripts for a flex application.

The initial setup was very easy.

We implemented it with our in-house team.

As you add more protocols and monitoring it does get expensive.

We looked at HP LoadRunner, SmartBear LoadComplete, and Appvance.

It is very easy to implement and you can start using it fairly quickly.