Perfecto vs SmartBear LoadNinja comparison

Cancel
You must select at least 2 products to compare!
Perforce Logo
312 views|189 comparisons
88% willing to recommend
SmartBear Logo
883 views|438 comparisons
100% willing to recommend
Comparison Buyer's Guide
Executive Summary

We performed a comparison between Perfecto and SmartBear LoadNinja based on real PeerSpot user reviews.

Find out in this report how the two Performance Testing Tools solutions compare in terms of features, pricing, service and support, easy of deployment, and ROI.
To learn more, read our detailed Perfecto vs. SmartBear LoadNinja Report (Updated: May 2024).
770,292 professionals have used our research since 2012.
Featured Review
Quotes From Members
We asked business professionals to review the solutions they use.
Here are some excerpts of what they said:
Pros
"In terms of cross-platform testing, they offer all of it, every device available in the market. It covers real scenarios that mimic production so that we don't miss out on any devices that our clients might be using to run the applications we develop. It's been great and very helpful.""I also like the reporting functions. We are constantly downloading these reports and sharing them with our final customers. They help us understand what kind of bugs are happening through the applications. The recording feature is handy because it lets us see a video of the process we run through the pipeline and discover the point at which the automation is breaking.""The reporting feature is really tough to find in some of the other products that are competitors. Having your CITB type dashboard, where we can see the test results and see recordings of each test that passed or failed, is probably one of the distinguishing aspects of Perfecto.""The most valuable feature is automated testing.""The automated test reporting functionality is the most valuable feature. We use the CI Dashboard. It's very important as it is the main reporting tool for our automated tests.""The quality of our software has improved since we implemented this solution.""Perfecto has affected our software quality in a good way. It has allowed us to execute on-demand and on-choice. We also track the number of issues that we find in the product. Every single day, we tag the issues that we found. For example, if something was found by automation, that means it was found by a Perfecto execution. Over time, we realized the real value in tracking those numbers. We can see now that we have clearly been finding issues earlier. It has allowed us to catch our defects earlier, thus improving the quality of our applications.""Their team is really great to work with. They're very flexible, and they always show care. They prioritize our work, our company, and our working relationship. I appreciate the ad hoc sessions that they schedule to provide help with troubleshooting, provide the information that we're looking for, or do a demo of a new feature that they have. They're always willing and very quick to get that scheduled for us. I appreciate that a lot."

More Perfecto Pros →

"We are happy with the technical support.""It's a very simple tool for performance testing.""SmartBear LoadNinja is easy to use and implement."

More SmartBear LoadNinja Pros →

Cons
"It does well for mobile testing, but when it comes to the web aspect, it is lagging a little bit in terms of execution.""The monitoring features, in particular network traffic monitoring, could be improved.""When using devices on the cloud, it lags quite a bit at times. I know that these are real devices that are being projected on our laptop screens and monitors, but if the speed could be improved, that would be good.""There was a discussion about having the capability to export the test results to a certain tool that we use in our project. If that were added it would be great not having to manually take screenshots, put them in a document, and share them on the different test management tools.""I'm hoping they can support on-premises instances. We have been working on a JIRA integration with Perfecto—and I'm extremely impressed that they have that—but at this time they're not supporting onsite JIRA instances, which is what we have.""It would be ideal if there was a complete integration with other test management tools and other applications like HPLM, Micro Focus, or Microsoft Azure.""If we could run an accessibility test in Perfecto against builds, it would help us a lot. Currently, that's a very manual process for us. We haven't found a tool that can do accessibility scans for iOS and doesn't depend on engineering effort. Having a feature related to that would be really awesome for us.""There could be some improvements done on the interface. At times, there has been a bit of a struggle when finding things on the interface. A UI revamp would be a better option in future. That UI hasn't changed much in a long time, so I think they could just make it a bit better so that people could find stuff easily and intuitively."

More Perfecto Cons →

"On a smaller scale, there will be no budget issues, but as we expand to a larger user base, I believe we will face some pricing challenges.""As we ran the test, we couldn't see the real-time results of how the solution behaved for 200 to 400 virtual users.""It needs time to mature."

More SmartBear LoadNinja Cons →

Pricing and Cost Advice
  • "This is an expensive solution compared to others, by 30% to 40%."
  • "I am not sure about its pricing, but from our perspective, licensing has been easy. Anytime I have new users or requests for users that want to get added, it's a very simple process. I just give the architectural owner of the product the name and email address, and they're able to easily add a new user. We don't have any issues in regards to getting licenses, but I don't have any insights into pricing."
  • "Pricing is an area where Perfecto can do a little better. When we obtain additional licenses, we enter into negotiations with them."
  • "Although Perfecto is a good product for us to use, it is a bit expensive. It takes management a bit of work to find the appropriate funding for us to keep Perfecto. I imagine there could be some way to make it more accessible."
  • "Perfecto is about 30-40% cheaper than Device Anywhere. That was a big reason why we switched. Perfecto also solves some of the issues that we had with Device Anywhere. We have grown by 100% since we started to use Perfecto, and now we have devices roaming. When we look at the competition, we would still stick with Perfecto."
  • "Pricing-wise, it is fine. It is not as expensive as what we used to have in the past from HP, IBM, and others. It is decently priced."
  • "It's definitely on the higher end of prices for this type of service."
  • "Perfecto has definitely saved us on the costs and efforts of having to maintain our own virtual test environment. We lost about 20 devices in the past to maintenance and audit. That was a massive loss for us, as a company, because we were giving devices to someone, but don't know whether we would get it back or not. Having those virtual labs, we don't need to worry about these kinds of things. We are easily saving $5,000 to $10,000 a month on device costing."
  • More Perfecto Pricing and Cost Advice →

  • "Certainly, the cost could be reduced."
  • More SmartBear LoadNinja Pricing and Cost Advice →

    report
    Use our free recommendation engine to learn which Performance Testing Tools solutions are best for your needs.
    770,292 professionals have used our research since 2012.
    Questions from the Community
    Top Answer:Mobile testing is the most valuable feature as it has reduced dependency on physical devices. We are located offshore and we don't have the physical devices, and shipping physical devices after every… more »
    Top Answer:Perfecto's price is excellent compared to other products with similar features. It was the lowest of the three we evaluated. We also established a partnership with Perfecto, so they provide discounts… more »
    Top Answer:It is slow compared to physical device testing. The interactive speed could be improved. And sometimes we have issues with our app not working properly with Perfecto. Even though it is working… more »
    Top Answer:SmartBear LoadNinja is easy to use and implement.
    Top Answer:There are different plans, including monthly, half-yearly, annually, and so on. If you require more bandwidth or you want to do more testing, then there are some premium plans also. We have around 100… more »
    Top Answer:SmartBear LoadNinja presented issues around some use cases that we wanted to do. We were using the solution to simulate using a browser and to give some browser access to our use case for multiple… more »
    Ranking
    Views
    312
    Comparisons
    189
    Reviews
    18
    Average Words per Review
    1,397
    Rating
    8.6
    14th
    Views
    883
    Comparisons
    438
    Reviews
    2
    Average Words per Review
    629
    Rating
    7.5
    Comparisons
    Also Known As
    Perfecto Mobile, Perfecto Web
    SmartBear LoadComplete
    Learn More
    Overview

    Perfecto is a cloud-based mobile app testing tool that is designed to help test applications on different operating systems and devices. The solution lets users automate processes related to advanced test scenarios and access browser versions and devices according to individual requirements. With Perfecto’s smart test execution capabilities, teams can manage testing across web and mobile platforms using a single script, conduct parallel testing for large test suites, and design advanced user-based test scenarios to accelerate app release cycles. The solution is also helpful for IT departments, providing insights into biometrics, device vitals, audio injection, network throttling, packet loss, and rotation data to meet desired end-user conditions that align with business priorities.

    Perfecto Features

    Perfecto has many valuable key features. Some of the most useful ones include:

    • Live Stream analysis
    • Reporting/analytics
    • Activity dashboard
    • Action-word testing
    • Bug tracking
    • Debugging
    • BDD & codeless automation
    • Model-based testing
    • Real-time monitoring
    • Role-based permissions
    • User management
    • Test failure analysis
    • Real user simulation

    Perfecto Benefits

    There are many benefits to implementing Perfecto. Some of the biggest advantages the solution offers include:

    • Streamlined test creation: When you use Perfecto, test creation is easy for your teams. Testing frameworks, like Appium, allows you to scale automated test scripts without any issues.
    • Scalable test execution: Perfecto enables you to accelerate your mobile app testing and is elastically-scaled with no single point of failure, so you can get the testing cadence you need. In addition, the solution makes it possible for you to execute tests from within IDEs as well as through CI servers like Jenkins.
    • Superior application quality for mobile: By using Perfecto, you can maximize continuous testing and shift quality by combining the solution with BlazeMeter. Together, these two solutions can help you measure mobile user experience, produce high-quality test data on demand, and create realistic mock services.
    • AI-driven test automation dashboards: The solution’s unique dashboards and heatmaps give you high-level visibility with timelines and trends.
    • Cloud-based lab: Perfecto’s Smart Testing Lab gives you the tools you need to quickly test your mobile app against thousands of permutations.

    Reviews from Real Users

    Perfecto is a solution that stands out when compared to many of its competitors. Some of its major advantages are that it has fantastic testing, reporting, and automation capabilities.

    Roland C., QA Lead at BMO Financial, says, “The most valuable would be their Live Stream analysis, where I can see the live analysis of all the executions on a single device or multiple devices as well as track them. The live analysis and reporting would be the single most valuable feature.”

    Andy B., Systems Engineer at a financial services firm, writes, “I really love the interaction it has with mobile devices, the testing capabilities, as well as reporting capabilities that we get from the application. Perfecto is excellent when it comes to executing cross-platform testing.”

    A Director, QE Automation & Analytics at a financial services firm, says, “The most valuable feature is cloud-based automation and testing.”

    Another PeerSpot reviewer, Kristina B., Product Manager at a manufacturing company, states, “The solution allows us to do seamless cross-platform testing, saves time, and comes with excellent customer service.”

    SmartBear LoadNinja is a cloud-based next-generation tool designed to develop and implement automated load tests for web services and web applications. LoadNinja helps organizations test their web server’s performance during tremendous load and will determine scalability and define overall dynamics.

    Load testing is integral in today's aggressive marketplace so that organizations can quickly and easily determine how their web servers will operate during extreme load (requests). LoadNinja intuitively creates load tests using automation developed by documenting past interactions over web pages and reproduces these interactions with hundreds of virtual users (VUs) simultaneously. These tests help organizations fully comprehend what load their websites, pages, etc, can successfully sustain without interruption or downtime.

    LoadNinja provides various metrics, such as CPU usage and traffic generated during each test run. LoadNinja integrates well with many of today’s popular browsers and web servers.

    LoadNinja Testing - How it works:

    • Develop a UI or API test scenario: These are single-instance tests. Each test is representative of an actual application usage process, such as updating client information, making or tracking an order, etc. A UI test replicates the actions of one user on the tested website. An API test replicates the behavior of one client application. Organizations will run these tests numerous times to replicate tremendous load on the specified web application being tested.
    • Load testing scenario: This is the example of an organization’s load test. The scenario will integrate one of many single-instance tests, designate the number of virtual users (VUs), advance load parameters, and other specific metrics.
    • Work the scenario: LoadNinja processes the load tests in the cloud and provides cloud machines automatically for the entirety of the test. During a UI test, each VU utilizes a real browser to conduct a test action to interact with the organization's web applications appropriately mimicking real user responses, including “think times” and experiencing client-side scripts. Throughout the entire process, LoadNinja will create real-time charts so that organizations are able to best monitor test metrics and the overall wellbeing of the application.
    • Study the results: Once the test is finished, LoadNinja will create a report detailing all the critical performance metrics gathered during the test. This report will give a good determination of an organization’s server capabilities and if it meets the established SLA.

    Top Features:

    • Real-life load tests, simulated real user actions: LoadNinja will document real user actions on the site to be tested, then replicate those actions with the assistance of hundreds or even thousands of virtual users. This ensures load tests are as close to real-life processes as possible. LoadNinja does not just test one URL of a website; it replicates the entire user experience. This is one feature that differentiates LoadNinja from most other solutions in the marketplace.
    • Real browsers, real user replication: LoadNInja replays the documented traffic from real browsers. Every VU utilizes its own browser; there is no overlapping. LoadNinja precisely replicates each user action. Whatever code or script is used, it will be completely tested for every page.
    • Develop tests quickly and easily: Organizations are able to quickly develop UI tests by documenting user actions in the embedded browser. API tests can be developed using the embedded request editor. There is no coding, downloading, or installation of browser plugins necessary.
    • Scales easily: LoadNinja is a cloud solution; there is no need to designate drive space or add extra “agent” devices to save recorded scripts or test results. LoadNinja does it all intuitively and automatically in the cloud. Organizations simply choose the determined amount of VUs desired to replicate, and begin. It’s that simple!

    Sample Customers
    Virgin Media, Paychex, Rabobank, R+V, Discover
    Falafel Software
    Top Industries
    REVIEWERS
    Financial Services Firm44%
    Insurance Company11%
    Comms Service Provider11%
    Computer Software Company11%
    VISITORS READING REVIEWS
    Financial Services Firm18%
    Computer Software Company17%
    Manufacturing Company8%
    Recreational Facilities/Services Company7%
    VISITORS READING REVIEWS
    Computer Software Company15%
    Financial Services Firm11%
    Retailer9%
    Energy/Utilities Company8%
    Company Size
    REVIEWERS
    Small Business4%
    Midsize Enterprise4%
    Large Enterprise92%
    VISITORS READING REVIEWS
    Small Business21%
    Midsize Enterprise12%
    Large Enterprise67%
    VISITORS READING REVIEWS
    Small Business32%
    Midsize Enterprise10%
    Large Enterprise58%
    Buyer's Guide
    Perfecto vs. SmartBear LoadNinja
    May 2024
    Find out what your peers are saying about Perfecto vs. SmartBear LoadNinja and other solutions. Updated: May 2024.
    770,292 professionals have used our research since 2012.

    Perfecto is ranked 8th in Performance Testing Tools with 23 reviews while SmartBear LoadNinja is ranked 14th in Performance Testing Tools with 3 reviews. Perfecto is rated 8.4, while SmartBear LoadNinja is rated 7.4. The top reviewer of Perfecto writes "Its reporting allows us to have a clear view regarding what tests have been executed". On the other hand, the top reviewer of SmartBear LoadNinja writes "Easy to use with good documentation and helpful support". Perfecto is most compared with BrowserStack, Sauce Labs, Appium, AWS Device Farm and Katalon Studio, whereas SmartBear LoadNinja is most compared with Apache JMeter, ReadyAPI Performance, OpenText LoadRunner Professional, BlazeMeter and Tricentis NeoLoad. See our Perfecto vs. SmartBear LoadNinja report.

    See our list of best Performance Testing Tools vendors.

    We monitor all Performance Testing Tools reviews to prevent fraudulent reviews and keep review quality high. We do not post reviews by company employees or direct competitors. We validate each review for authenticity via cross-reference with LinkedIn, and personal follow-up with the reviewer when necessary.