Apache JMeter vs Perfecto comparison

Cancel
You must select at least 2 products to compare!
Apache Logo
7,853 views|6,435 comparisons
87% willing to recommend
Perforce Logo
315 views|198 comparisons
88% willing to recommend
Comparison Buyer's Guide
Executive Summary

We performed a comparison between Apache JMeter and Perfecto based on real PeerSpot user reviews.

Find out in this report how the two Performance Testing Tools solutions compare in terms of features, pricing, service and support, easy of deployment, and ROI.
To learn more, read our detailed Apache JMeter vs. Perfecto Report (Updated: March 2024).
768,246 professionals have used our research since 2012.
Featured Review
Quotes From Members
We asked business professionals to review the solutions they use.
Here are some excerpts of what they said:
Pros
"We find the load testing feature valuable.""We are using it just for load testing. We are using its free version, and it is scalable.""The solution is free. You don't need to worry about licensing costs.""Scripting with the solution is good.""The ease of use is the solution's most valuable feature. Also, the ability to easily create test cases is also very good. It's easy to just ramp up on the solution.""We find the ease of use and the reports and graphs available valuable.""Very user-friendly and easy to use.""It is scalable. You can scale up to 1,000 users in JMeter. If you can put up four slave servers, you can easily ramp up to 1,000 users."

More Apache JMeter Pros →

"Perfecto has affected our software quality in a good way. It has allowed us to execute on-demand and on-choice. We also track the number of issues that we find in the product. Every single day, we tag the issues that we found. For example, if something was found by automation, that means it was found by a Perfecto execution. Over time, we realized the real value in tracking those numbers. We can see now that we have clearly been finding issues earlier. It has allowed us to catch our defects earlier, thus improving the quality of our applications.""I also like the reporting functions. We are constantly downloading these reports and sharing them with our final customers. They help us understand what kind of bugs are happening through the applications. The recording feature is handy because it lets us see a video of the process we run through the pipeline and discover the point at which the automation is breaking.""The automated test reporting functionality is the most valuable feature. We use the CI Dashboard. It's very important as it is the main reporting tool for our automated tests.""We're working in Agile and we need results ASAP. The fact that the lab provides same-day access to new devices is extremely important to us.""The most valuable aspect of the solution is that it covers all types of devices on the market allowing you to test different versions of an operating system.""The number one feature, which if we didn't have out-of-the-box would be missed, is the fact that we have video execution. That gives us the ability to view errors or defects in the progression, from beginning to the end of the video.""There are a whole bunch of things that I like about the solution, but I really love the interaction it has with mobile devices, the testing capabilities, as well as reporting capabilities that we get from the application. The reports are very detailed.""The quality of our software has improved since we implemented this solution."

More Perfecto Pros →

Cons
"Modeling a test is difficult. If you don't have much knowledge, you won't be able to do it easily. Testing APIs is also difficult.""JMeter should be more stable. Every time there is a new release coming up, a lot of its older functionalities or the new functionalities that are brought in are not very well-documented. It should be documented properly, and there should be proper use cases.""Both scalability and stability could be improved in Apache JMeter.""The plug-ins make the reports heavy and they have to be run in non-GUI mode.""Running JMeter in GUI mode uses a lot of memory, which means we need to switch to a non-GUI mode when using a heavy load.""Currently, the integration pipeline is implemented by using Jenkins or a similar tool platform. These are continuous integration tools. As far as I know, integration is done by using custom scripts. It would be good if the integration with a continuous integration pipeline, like Jenkins or Hudson, can be done out of the box without using a script.""The solution needs more metrics for reporting.""In Micro Focus LoadRunner we can go from the UI and we can configure it. There is no such feature in Apache JMeter. There should be UI-based recording history or logs."

More Apache JMeter Cons →

"When using devices on the cloud, it lags quite a bit at times. I know that these are real devices that are being projected on our laptop screens and monitors, but if the speed could be improved, that would be good.""I'm hoping that Perfecto will come up with browser testing as well because it would be easier to access it.""We've had a couple of issues lately with videos not loading or browsers dying after some execution, although that happens very rarely.""We feel that Perfecto is a little slow. If they could improve on that slowness in accessing the app, when we want to click a button, that would be great because we feel the difference. An improvement in the connectivity speed is required.""Its performance should be improved. Anything to speed up the user interface would be a great help. We've had a lot of pain with their migration from a product UI that was based on Adobe Flash to the new product that is based on HTML5. Migrations like that seemed to be very painful and not a real smooth process. We're still sort of recovering from that migration from old technology to new, and we haven't got all the functionality ported over that we used to have on the old UI.""The monitoring features, in particular network traffic monitoring, could be improved.""It would be nice if there were some kind of AI to compile a list of available devices. Currently, we have to look at the web interface to see the available devices, but the pipelines can't do it on their own there. We always need to do this manually, so it would be better if this feature were automated.""We don't use Perforce's BlazeMeter with Perfecto. From my perspective, it's not really relevant."

More Perfecto Cons →

Pricing and Cost Advice
  • "Free"
  • "I switched to Apache because it is free. Other tools are much too expensive and can cost up to $50,000 a year if you are looking at commercial options."
  • "This is an open-source solution, and there are no fees."
  • "This is an open-source product."
  • "I was using the free version of the software."
  • "In terms of open-source adoption, it is completely free."
  • "We are using the free version, and if required, we can easily switch to the other version."
  • "When comparing the price with Load Runner, and if the cost is an issue then JMeter is a better choice"
  • More Apache JMeter Pricing and Cost Advice →

  • "This is an expensive solution compared to others, by 30% to 40%."
  • "I am not sure about its pricing, but from our perspective, licensing has been easy. Anytime I have new users or requests for users that want to get added, it's a very simple process. I just give the architectural owner of the product the name and email address, and they're able to easily add a new user. We don't have any issues in regards to getting licenses, but I don't have any insights into pricing."
  • "Pricing is an area where Perfecto can do a little better. When we obtain additional licenses, we enter into negotiations with them."
  • "Although Perfecto is a good product for us to use, it is a bit expensive. It takes management a bit of work to find the appropriate funding for us to keep Perfecto. I imagine there could be some way to make it more accessible."
  • "Perfecto is about 30-40% cheaper than Device Anywhere. That was a big reason why we switched. Perfecto also solves some of the issues that we had with Device Anywhere. We have grown by 100% since we started to use Perfecto, and now we have devices roaming. When we look at the competition, we would still stick with Perfecto."
  • "Pricing-wise, it is fine. It is not as expensive as what we used to have in the past from HP, IBM, and others. It is decently priced."
  • "It's definitely on the higher end of prices for this type of service."
  • "Perfecto has definitely saved us on the costs and efforts of having to maintain our own virtual test environment. We lost about 20 devices in the past to maintenance and audit. That was a massive loss for us, as a company, because we were giving devices to someone, but don't know whether we would get it back or not. Having those virtual labs, we don't need to worry about these kinds of things. We are easily saving $5,000 to $10,000 a month on device costing."
  • More Perfecto Pricing and Cost Advice →

    report
    Use our free recommendation engine to learn which Performance Testing Tools solutions are best for your needs.
    768,246 professionals have used our research since 2012.
    Questions from the Community
    Top Answer:Postman lets you easily define variables, which then get updated automatically. This is a huge time-saver and makes processes very efficient. We can also export the test cases we create and share them… more »
    Top Answer:Blazemeter is a continuous testing platform that provides scriptless test automation. It unifies functional and performance testing, enabling users to monitor and test public and private APIs We… more »
    Top Answer:JMeter is basically the art of the entire performance testing process.
    Top Answer:Mobile testing is the most valuable feature as it has reduced dependency on physical devices. We are located offshore and we don't have the physical devices, and shipping physical devices after every… more »
    Top Answer:Perfecto's price is excellent compared to other products with similar features. It was the lowest of the three we evaluated. We also established a partnership with Perfecto, so they provide discounts… more »
    Top Answer:It is slow compared to physical device testing. The interactive speed could be improved. And sometimes we have issues with our app not working properly with Perfecto. Even though it is working… more »
    Ranking
    Views
    7,853
    Comparisons
    6,435
    Reviews
    45
    Average Words per Review
    487
    Rating
    7.6
    Views
    315
    Comparisons
    198
    Reviews
    18
    Average Words per Review
    1,397
    Rating
    8.6
    Comparisons
    BrowserStack logo
    Compared 35% of the time.
    Sauce Labs logo
    Compared 25% of the time.
    Appium logo
    Compared 10% of the time.
    AWS Device Farm logo
    Compared 6% of the time.
    Katalon Studio logo
    Compared 3% of the time.
    Also Known As
    JMeter
    Perfecto Mobile, Perfecto Web
    Learn More
    Overview

    Apache JMeter is an open-source Java application that tests load and functional behavior and performance in applications. Created initially to test web applications, it has expanded its functionality to test other functions. For instance, you can test a server to see how efficiently it works and how many user requests can be handled simultaneously.

    You can use JMeter to test functional performance and regression tests on different technologies. This Java desktop application has an easy-to-use graphical interface which uses the Swing graphical API. You can run JMeter on any environment that accepts a Java virtual machine, such as Windows, Linux, and Mac.

    What protocols does JMeter support?

    • Web - HTTP, HTTPS
    • Web services - SOAP/XML-RPC
    • Email service - POP3, IMAP, SMTP
    • FTP service
    • Database via JDBC drivers
    • LDAP
    • Native commands or shell scripts
    • TCP
    • Java objects

    How does JMeter work?

    JMeter sends requests to a target server by simulating a group of user requests. Then it collects and calculates statistics on the performance of the target. This target can be a server or an application.

    You can test the performance of static resources, such as JavaScript or HTML, and dynamic resources, such as JSP, Servlets, and AJAX. It is also helpful to determine how many concurrent users your website can handle.

    There are two main tests you can carry out with JMeter: load test and stress test. The load test models expected usage of a server by simulating multiple users accessing the web server simultaneously. The stress testing aims to find the maximum load capacity of the server or application.

    Apache JMeter Key Features

    • JMeter enables fast test plan recording, building, and debugging via a featured test IDE (integrated development environment).
    • Command-line mode allows carrying out load tests from any OS compatible with Java.
    • JMeter can extract data from most popular response formats, such as HTML, JSON, XML.
    • JMeter is entirely portable.
    • A multithreading framework allows you to simultaneously test multiple samplings and separate thread groups.
    • Thanks to the caching feature, you can conduct offline analysis or replay test results.

    Apache JMeter Benefits

    • As open-source software, it is freely available and continuously improved.
    • The setup is user-friendly, and no installation is needed.
    • The GUI (graphic user interface) is intuitive and easy to use.
    • You can write your own tests and use the visualization plugins to extend the testing.
    • It is a platform-agnostic tool. For example, JMeter can work with Linux by clicking on the JMeter shell script. On Windows, you can call up JMeter by starting the jmeter.bat file.
    • Since the test plans are stored in XML format, you can generate a test plan using any text editor.
    • You can simulate a heavy load on a server, a group of servers, a network, or an object. Use JMeter to test an application or server's strength and performance under different load types.

    The JMeter extensible core has numerous benefits:

    • Unlimited testing capabilities via pluggable samplers.
    • You can choose multiple load statistics with pluggable timers.
    • Visualization plugins and data analysis enable customization.
    • Continuous integration via third-party open-source libraries (Maven, Gradle, and Jenkins).

    Reviews from Real Users

    Stephen B., I.T. Architect, Analyst, and Developer at an educational organization, says, "The scripting ability is most valuable. It is easy to use. There is a UI, and you can go in there and figure those things out. After you've got a good set of tests, you basically have a scripted document that you can grab and execute in a pipeline. It is pretty quick to set up, and you can scale it and version control it."

    "I like the fact that JMeter integrates well with other tools," adds the Founder and Principal Consultant at a tech services company.

    A Quality Engineering Delivery Leader at a financial services firm says, “The performance of the solution is excellent. They have designed the product so that it is very easy to configure. You can basically do anything you like with the product. It's not very restrictive. We like the fact that the technology is open-source.”

    Perfecto is a cloud-based mobile app testing tool that is designed to help test applications on different operating systems and devices. The solution lets users automate processes related to advanced test scenarios and access browser versions and devices according to individual requirements. With Perfecto’s smart test execution capabilities, teams can manage testing across web and mobile platforms using a single script, conduct parallel testing for large test suites, and design advanced user-based test scenarios to accelerate app release cycles. The solution is also helpful for IT departments, providing insights into biometrics, device vitals, audio injection, network throttling, packet loss, and rotation data to meet desired end-user conditions that align with business priorities.

    Perfecto Features

    Perfecto has many valuable key features. Some of the most useful ones include:

    • Live Stream analysis
    • Reporting/analytics
    • Activity dashboard
    • Action-word testing
    • Bug tracking
    • Debugging
    • BDD & codeless automation
    • Model-based testing
    • Real-time monitoring
    • Role-based permissions
    • User management
    • Test failure analysis
    • Real user simulation

    Perfecto Benefits

    There are many benefits to implementing Perfecto. Some of the biggest advantages the solution offers include:

    • Streamlined test creation: When you use Perfecto, test creation is easy for your teams. Testing frameworks, like Appium, allows you to scale automated test scripts without any issues.
    • Scalable test execution: Perfecto enables you to accelerate your mobile app testing and is elastically-scaled with no single point of failure, so you can get the testing cadence you need. In addition, the solution makes it possible for you to execute tests from within IDEs as well as through CI servers like Jenkins.
    • Superior application quality for mobile: By using Perfecto, you can maximize continuous testing and shift quality by combining the solution with BlazeMeter. Together, these two solutions can help you measure mobile user experience, produce high-quality test data on demand, and create realistic mock services.
    • AI-driven test automation dashboards: The solution’s unique dashboards and heatmaps give you high-level visibility with timelines and trends.
    • Cloud-based lab: Perfecto’s Smart Testing Lab gives you the tools you need to quickly test your mobile app against thousands of permutations.

    Reviews from Real Users

    Perfecto is a solution that stands out when compared to many of its competitors. Some of its major advantages are that it has fantastic testing, reporting, and automation capabilities.

    Roland C., QA Lead at BMO Financial, says, “The most valuable would be their Live Stream analysis, where I can see the live analysis of all the executions on a single device or multiple devices as well as track them. The live analysis and reporting would be the single most valuable feature.”

    Andy B., Systems Engineer at a financial services firm, writes, “I really love the interaction it has with mobile devices, the testing capabilities, as well as reporting capabilities that we get from the application. Perfecto is excellent when it comes to executing cross-platform testing.”

    A Director, QE Automation & Analytics at a financial services firm, says, “The most valuable feature is cloud-based automation and testing.”

    Another PeerSpot reviewer, Kristina B., Product Manager at a manufacturing company, states, “The solution allows us to do seamless cross-platform testing, saves time, and comes with excellent customer service.”

    Sample Customers
    AOL, Orbitz, Innopath Software, PrepMe, Sapient, Corporate Express Australia, CSIRO, Ephibian, Talis, DATACOM, ALALOOP, eFusion, Panter, Sourcepole, University of Western Cape
    Virgin Media, Paychex, Rabobank, R+V, Discover
    Top Industries
    REVIEWERS
    Computer Software Company38%
    Financial Services Firm30%
    Educational Organization4%
    Renewables & Environment Company4%
    VISITORS READING REVIEWS
    Financial Services Firm18%
    Computer Software Company17%
    Manufacturing Company8%
    Government6%
    REVIEWERS
    Financial Services Firm44%
    Insurance Company11%
    Comms Service Provider11%
    Computer Software Company11%
    VISITORS READING REVIEWS
    Financial Services Firm19%
    Computer Software Company17%
    Manufacturing Company8%
    Recreational Facilities/Services Company7%
    Company Size
    REVIEWERS
    Small Business25%
    Midsize Enterprise24%
    Large Enterprise52%
    VISITORS READING REVIEWS
    Small Business16%
    Midsize Enterprise13%
    Large Enterprise70%
    REVIEWERS
    Small Business4%
    Midsize Enterprise4%
    Large Enterprise92%
    VISITORS READING REVIEWS
    Small Business21%
    Midsize Enterprise11%
    Large Enterprise68%
    Buyer's Guide
    Apache JMeter vs. Perfecto
    March 2024
    Find out what your peers are saying about Apache JMeter vs. Perfecto and other solutions. Updated: March 2024.
    768,246 professionals have used our research since 2012.

    Apache JMeter is ranked 1st in Performance Testing Tools with 82 reviews while Perfecto is ranked 8th in Performance Testing Tools with 23 reviews. Apache JMeter is rated 7.8, while Perfecto is rated 8.4. The top reviewer of Apache JMeter writes "It's a free tool with a vast knowledge base, but the reporting is lackluster, and it has a steep learning curve". On the other hand, the top reviewer of Perfecto writes "Its reporting allows us to have a clear view regarding what tests have been executed". Apache JMeter is most compared with BlazeMeter, Postman, Tricentis NeoLoad, OpenText LoadRunner Professional and Katalon Studio, whereas Perfecto is most compared with BrowserStack, Sauce Labs, Appium, AWS Device Farm and Katalon Studio. See our Apache JMeter vs. Perfecto report.

    See our list of best Performance Testing Tools vendors.

    We monitor all Performance Testing Tools reviews to prevent fraudulent reviews and keep review quality high. We do not post reviews by company employees or direct competitors. We validate each review for authenticity via cross-reference with LinkedIn, and personal follow-up with the reviewer when necessary.