Perfecto Valuable Features

Roland Castelino - PeerSpot reviewer
QA Lead at BMO Financial

The most valuable would be their Live Stream analysis, where I can see the live analysis of all the executions on a single device or multiple devices as well as track them. The live analysis and reporting would be the single most valuable feature.

We leverage Perfecto’s reporting and analytics a lot. From the CI Dashboard, it is mainly the status, which is the past, failure count, and time consumption, e.g., how much time did an average test or script take? Along with that, it provides the historical view compared to the previous result, e.g., am I a pass or fail? Also, the stack trace is very important. Whenever a pass occurs, we don't look beyond that. However, whenever a failure occurs, the stack trace information that it gives us is pretty critical for us when figuring out where failures lie. It gives a summary for the pass/fail count, total test count, the historical view, time consumption for each test as well as the total tests, and the stack rate of the failure.

Perfecto's analytics are very important since we use them on a daily basis. We run our executions daily. After every execution, we pull information from the Perfecto reporting system and share that with our stakeholders. Having this information accurately reported is pretty important for us, so everybody is aware of the current status of the product. That way, we can evaluate the health of the product or environment against that which has been executed. Therefore, it helps make those real-time decisions and highlights the impact to the business.

I found Perfecto to be pretty easy to use while executing against cross-platforms. The main reason is because the same script or test automation where we execute on multiple platforms has minimal changes that I need to do. Also, it is easy for me to set up an execution on one platform, then on another platform, either in parallel or one after the other. Parallel opportunities save me time. Once the execution has been completed across these different configurations, I can always check and compare, e.g., what are the differences and consistencies? 

We utilize Perfecto’s cloud-based lab to test across devices, browsers, and OSs. I use it occasionally for manual testing. Though, there are other team members who use it more frequently than I do. I use it mainly for executing my automated tests. We have the Perfecto lab, cloud devices, and machines. I can program my test to execute against any of those devices, which gives me more confidence in my product. I can compare and see how my product or application functionally behaves across these different devices and from a UI point of view, which helps me a lot.

The device lab is extremely important to our testing operations. We rely on having multiple devices up and running all the time. Whenever we kick off an execution, there are multiple reasons why executions may get triggered: 

  1. CodeCommit
  2. A scheduled job.
  3. Might be on-demand by any stakeholder. 

We need the lab to be available, as we need devices up and running for executions to take place. Also, the devices help since they allow us to have parallel execution, and not just wait for a sequential device to become free and available. Therefore, volume is definitely key. It also gives us an opportunity to compare execution across platforms in that space.

It is extremely important to you that the lab provides same-day access to new devices since we analyze that data every single day after execution. 

Perfecto provides their own framework called Quantum Framework. That is one option. The other option is, if I want to have my own framework, I can have a Java-based Maven project, take a Selenium library, AppiumLibrary, and REST Assured library, and utilize the open-source framework. It is easy for us to connect to Perfecto, no matter what framework we use, as long as it has these core libraries in it. I can design and structure it any way that I want. The execution will happen in Perfecto no matter what since they have support for these tools or libraries. It is pretty neat that way. We are not dependent on using just one particular framework to use Perfecto. While there are still some framework limitations, there is the opportunity to use multiple, different open-source frameworks, then pass the execution to Perfecto. We can use most frameworks, then design and craft it any way that we want, then just pass the execution to Perfecto.

View full review »
Glenford John - PeerSpot reviewer
Product Manager, Device Integration at a comms service provider with 10,001+ employees

The automation piece is the most valuable feature. Every time we had a new version of either OS or an application, we found that being able to automate the testing across different devices is very valuable.

Perfecto is great at executing cross-platform testing. From a carrier perspective, it's all mobile, but we also have web applications. We used to be able to test on Symbian, Windows OS, Android, and iOS. Today, we can go on a big screen and test on Chrome, Firefox, Explorer, and other browsers.

My team leverages Perfecto's reporting and analytics. When we start stress testing an application, we can look at all the reporting that comes from that, and we can tell at what part of the day the app is performing better.

Our analytics are very important to us. Presentations are important, so it's good to have reports with graphs that show the time of day or how the app was performing for months at a time. We use those reports to screen capture, go with the data, and to show to upper management. We can do a comparison of one version performance to another version of the application or device performance.

We utilize Perfecto's cloud-based lab to test across devices, browsers, and OSs. We are a carrier, so we have our devices with different OS versions of devices. Right now, it's mostly just Apple and Androids. Back in the day, they had many different versions of the OS, but the only other thing on top of that is web-based browsers. We don't test Linux, which is an operating system of Windows.

We don't use Perfecto's cloud. We use our own devices because we test on only our organization's devices. Unless we do some kind of competitive analysis, it's not really a test; it's just a comparison to see how other solutions compare to ours.

The range of open-source technologies that Perfecto supports is great. I have a team of contractors that report to me and do all the testing for our organization, and that team turns over every couple of years. We mostly utilize the in-house automation that is available for anyone who isn't a great programmer.

They have support for all these other drivers, and they have something for non-programmers who want to automate their work, which is amazing.

View full review »
AG
Project Manager at a financial services firm with 10,001+ employees

The mobile testing is the most valuable feature. Having the devices available virtually to test for all the changes we have enables our team to function. We're all working virtually, as my team is spread out globally. I have team members in India, Charlotte, and Toronto, as well as in Dublin. It would be impossible for all of them to have the actual devices to test all the changes, developments, and new applications.

In recent days I have been exploring the license keys we have with Perfecto and I found out that we have five license keys for web browser testing. That's when I started exploring that area and I'm planning to use it a lot for web browser as well.

In terms of cross-platform testing, they offer all of it, every device available in the market. It covers real scenarios that mimic production so that we don't miss out on any devices that our clients might be using to run the applications we develop. It's been great and very helpful.

The cloud-based lab, with all the devices that we need, is very important. If Perfecto were to go down, it would block our testing, as all testing of our mobile-related changes is done in Perfecto. That means it enables us to complete the tasks assigned to our team. It's 100 percent important for us.

Also, having access to new devices same-day enables the testers to test them right away. That makes everyone productive and efficient.

Because I'm a project manager, I don't use Perfecto’s reporting and analytics, but our business support manager presents me reports where I can see how many devices were used or how much testing was done. Our technical managers use the reporting and I know it has been a very useful feature of Perfecto.

And having the Perfecto tools accessible from a browser is convenient, efficient, and fast. It's easy to go to the browser and launch the application, log in with your account, and you should be good to go. You just select the device that you actually wanted to perform your testing on.

View full review »
Buyer's Guide
Perfecto
March 2024
Learn what your peers think about Perfecto. Get advice and tips from experienced pros sharing their opinions. Updated: March 2024.
765,234 professionals have used our research since 2012.
Bharat Malik - PeerSpot reviewer
Senior Automation Engineer at RBC

The most valuable feature is the ability to record the video of every test case that we run during the automation. This means that we can go back at any time and actually see the video where the test passes or fails.

We use Perfecto's cloud-based lab for testing across multiple OS versions. The online banking team uses it to test different Chrome versions on different OS versions, which is something that really helps. It holds all of the emulators and simulators, so it's very important because we depend on those tools for our testing.

Perfecto provides same-day access when new devices are released, which is important because we sometimes make requests for these. For example, when a new iPhone is launched, it is quite important to test. This is true for all of the latest devices.

Using the device lab has definitely helped us to scale our testing operations. At this point, we run tests across 300 devices and this is possible because our test execution time has decreased significantly.

View full review »
MB
Manager and Platform Lead, Canadian Digital Banking at BMO

The number one feature, which if we didn't have out-of-the-box would be missed, is the fact that we have video execution. That gives us the ability to view errors or defects in the progression, from beginning to the end of the video. That's like a full test case. If we were to go with our automation, we would need to go either with Perfecto or some other vendor because, out-of-the-box, Selenium doesn't come with videos.

Another thing that it supports, besides Selenium and Appium, is that it plugs into Robot Framework. That is really nice because the Perfecto folks have created an integration between Robot Framework and Perfecto. Also, REST Assured is taken into consideration. The only thing is that, with Rest Assured, because there is no browser, you need to specify a connection to Perfecto. You need to call keep-alive to say, "Hey, perfecto, track this." It would be nice if Perfecto could do it out-of-the-box, even though we did it with our code. There are also other REST API libraries for Python, such as a request library in Python.

As a regular user of the solution, day-to-day, I go through the reporting. I view the failures and the summary of test sets and test results. The reporting is a very important feature. There's an API to integrate with it within the code and that's really a plus. It integrates well inside. You can also tag it with which branch you're using from Git, or you can label it with whatever you'd like. It's a very cool feature that they added soon after they had the reporting.

For reporting, I use the CI Dashboard and I use anything to do with ALM because we have our jobs running through ALM for test management. You can report your results there. We also have Robot Framework still running so the dashboards we have are OLBQA56 or OLBQA3. Other than the CI Dashboard, I check out the Report Library quite a bit. I have my own filters on the reports and I'll filter based on the last 24 hours or 12 hours, as well as based on the job name. And, of course, I definitely use the Live Stream to see that my tests are up and running.

Also, Perfecto allows us to run on the cloud. Instead of running on our local machines, we can offload it to a cloud and we can run off servers that are not even local. That helps in case we do restarts. We don't need to worry about our machine restarting. Everything is on the cloud and running robotically.

View full review »
Kristina Bogojevska - PeerSpot reviewer
Product Manager at a manufacturing company with 10,001+ employees

Manual testing is definitely valuable. The insights feature is also valuable. I just started looking into that to get more data around the results of the testing. I'm looking to explore that one a bit more, but definitely, the manual testing and the insights are the top ones.

Their team is really great to work with. They're very flexible, and they always show care. They prioritize our work, our company, and our working relationship. I appreciate the ad hoc sessions that they schedule to provide help with troubleshooting, provide the information that we're looking for, or do a demo of a new feature that they have. They're always willing and very quick to get that scheduled for us. I appreciate that a lot.

View full review »
RG
Director, QE Automation & Analytics at a financial services firm with 10,001+ employees

The most valuable feature is cloud-based automation and testing. 

In terms of Perfecto's ability to perform cross-platform testing, I would rate it a ten out of ten. The first advantage is that we can take the entire execution to the cloud. We don't have to maintain a lab in-house. Second, we can have parallel test execution, which was tricky when we were doing everything in-house. Third, we have better browser automation and device test coverage.

When we maintain a physical inventory in-house, we cannot get all of the operating systems or different types of phones that are available on the market. Even when it comes to browsers, for example, if I have to run multiple versions of different browsers, whether it is Chrome, Edge, Firefox, or Safari, it is much easier when we can take it to the cloud. Perfecto provides that with all of the options for all of the browsers, as well as the real devices.

When we onboarded Perfecto, the test cases and the framework were already designed using open-source technologies. We used TestNG for the framework and Appium for the automation. As Perfecto supports Appium, it was easy for us to implement and we are still running these technologies.

View full review »
Rodrigo Candido Costa - PeerSpot reviewer
Software Quality and Automation Specialist at PRIME CONTROL

The most valuable feature is the ability to automate testing on real devices. We use Perfecto to test various mobile devices, like phones and tablets. We can use Perfecto to test across multiple platforms. We use it on the website to do manual testing and testing on all of the platforms I've seen before. We also use it through the CI servers, such as Jenkins and Azure DevOps, using authentication tokens to authenticate the process we are running. It also works with mobile operating systems, such as Android and iOS over the web testing tool.

I also like the reporting functions. We are constantly downloading the reports and sharing them with our final customers. They help us understand what kind of bugs are happening through the applications. The recording feature is handy because it lets us see a video of the automation that we run through the pipeline and discover the correct point at which the automation is breaking.

There is a dashboard where we can download those reports and see the results of our latest runs. We can do many kinds of filters and use some custom ones. The dashboard is fantastic compared to other tools I have used before. The analytics are crucial, too. We use QA analytics every day here in our operations. We use this feature to access the devices and see which ones are available for testing. 

The physical devices can be shared on the cloud, but sometimes a device is unavailable because another person already uses it. A tester can log on, select one of the available devices for manual testing, and later use the tools to automate the test. 

Perfecto's support for a broad range of open-source tools is critical because that is what we primarily use. In cases where we are using open-source tools, we use a lot of frameworks that rely on Python and Java. We also use Ruby-based technologies with Selenium or Appium as an intermediary. They always support us when it comes to implementing a new automated testing suite and provide many examples of how we can do it with each framework and language. They have excellent documentation and lots of models on their GitHub page too.

View full review »
AndyBrown - PeerSpot reviewer
Systems Engineer at a financial services firm with 10,001+ employees

There are a whole bunch of things that I like about the solution, but I really love the interaction it has with mobile devices, the testing capabilities, as well as reporting capabilities that we get from the application. The reports are very detailed. They provide all of the information needed for the testers and developers to really look at their applications and know if there are any issues or not.

Perfecto is excellent when it comes to executing cross-platform testing. They offer support, and their support is excellent. They're very responsive. With a lot of those cross-platform applications, we have to work together so that we can set up the configurations between the two interfacing applications, but it's very good.

When it comes to reporting, we take a look at how many users are using Perfecto, and for which devices. That helps us to narrow down exactly which devices are being used the most for the applications our teams are testing. I find the reports and the analytics to be extremely helpful. I cross-reference the user reports with the users' lines of business because it's helpful for us to know which users and which lines of business are using Perfecto.

The most widely-used reports are the individual testers' reports. When they run their automation scripts, it gives them a lot of the details about the test run, such as which components passed or failed. And when something fails, it points that out with an error so that the development and testing teams can look at what that error is pointing to.

And the cloud-based device lab is extremely important to our testing operations. The mobile device testing is one of the main things we use Perfecto for, although we do have a couple of teams that use the PC platforms as well. On a scale of one to 10, the importance of the mobile testing is a nine or 10.

Our testers have other means of testing our PC applications. They can test some of those applications from their local machines. They may or may not need access to Perfecto's PC resources, but for mobile devices, we have users who are offshore, and they do not have access to physical mobile devices. Having Perfecto's cloud-based application, users in India can access the cloud and do their testing. It's extremely important for our organization to be able to provide mobile devices for offshore testing.

We mainly use Appium among the open-source technologies that Perfecto supports. A couple of our PC teams may be using Selenium, but for the most part, the mobile users are using Appium, and the integration between Perfecto and Appium is excellent. If there is any issue for our Appium users, a lot of the time it's not on the Perfecto side, it's on our side; usually configuration issues. 

View full review »
VR
Manager at a insurance company with 1,001-5,000 employees

The inventory of browser options with the latest versions is valuable. There are various iOS and Android options. They are useful to us. We are using its cross-browser capability. It is certainly helpful for us. We have CrossBrowser for Mobile. We use Edge as our enterprise browser, but we know people use Chrome, Safari, and other browsers. We test our application through those browsers.

We also like some of the plugins and integrations that Perfecto has.

View full review »
reviewer1963857 - PeerSpot reviewer
Head of Quality at a comms service provider with 1,001-5,000 employees

Recently, the network testing component where we can use Perfecto on devices has been the most valuable feature. There is very little competition out there in this regard. We use it for 4G and 5G testing of launched 5G and 4G brands. So, the network testing can be accessed from anywhere in the world. That is very valuable for us.

Perfecto is very good when it comes to executing cross-platform testing. We have just gotten started on cross-browser testing. We already use Perfecto for the native-application testing on multiple devices. I think now we also use Perfecto for browser-related testing. So, all the cross-browser testing works well. We got a couple of licenses that we use.

We can mostly see usage over time using the device logs. This is what matters the most to us, e.g., what team is using a device for how long.

We use Cypress, and it supports it quite well. We use APM, and it also supports that quite well. We used to use the native quantum framework as well in the past, which was obviously well-supported, but now we have moved onto Cypress and APM. Also, we have Applitools, which is not open source, but a paid proprietary tool.

View full review »
Sameer Bakshi - PeerSpot reviewer
Test Specialist at IBM

People from all over the world can use it at the same time. It's not for a particular location. That's most valuable. Their cloud-based lab to test across devices, browsers, and OSs is the main part of it. People from offshore and onshore can access the devices. We can open everything on our system so we don't have to go looking for physical devices.

It saves on the cost and effort of having to maintain our own virtual testing environment. Even our onshore team is not in the city that we work in, so that helps a lot. Even if we didn't invest a lot in getting multiple devices, having to share those devices would become a hassle.

And same-day access to new devices is quite important. When OSs are updated or something new comes onto the market, upon our request, we can have new devices installed on the cloud and have access to them. We don't have any wait time and that's great.

Overall, it has helped massively to speed up our testing efforts and not just from a device point of view. Over the last two years, the performance of the Perfecto tool has picked up as well. It's much more real-time now, compared to what it was when I started.

Also, when I started using it, we did not have the web capabilities and we used to test only on mobile. We had Android and iOS, and that was great, but around 2020, they added web functionality as well, with Windows and Mac devices on the Perfecto cloud. It's great having everything together on one platform.

We use Perfecto's reporting and analytics for automation testing. Obviously, we need to see what the results are for whatever tests we perform from an automation perspective. Reporting results to our business team plays a huge role.

One other feature is the screenshots that Perfecto has. If I'm testing something and I want to take a screenshot, it's just a click away. That makes it much easier to record all the results.

View full review »
Govindaraj Ponnusamy - PeerSpot reviewer
Automation Architect at a tech consulting company with 10,001+ employees

It has a lot of features as well as 24/7 support. Everything is captured on their website.

The QR code automation for mobile is something that we have trained on using Robot Frameworks, which has not been supported well. We have used Perfecto to automate the QR code automation, which provides a very good value to us. It records the image file injection. This is a very good feature where you can inject the image and test it with the QR code, which is one of the main advantages of using Perfecto.

Currently, we are moving to the cloud. The performance and automation testing are very good. We compared the physical devices with the Android simulation, and its physical devices gave us better performance than the Android simulation.

The cloud and cloud mobile are very good features. They provide us with better performance.

Perfecto's reporting and analytics are very useful for us, and what we can see from Live Stream is really awesome. There are no delays at all. Whatever we are seeing in the mobile, the same thing will show in the Live Stream tool. It provides insights in the pie chart as well as the history and trends. So, you can go through it and find the number of tests executed, the pass/fail cases, etc. They have a report library that gives data about all the executions, including details of the history. 

The time stamp helps us out a lot by providing the duration for the previous execution and the current execution. So, there are a lot more reporting configuration tools, which helps us. Also, we can export this particular report and give it to anyone for review.

The lab has been very helpful for us when pushing API or image files. Anything you can push into the repository, we can test it. It is seamless. We can test it through cloud devices, which is very useful for us. 

It supports a lot of open-source technologies. We have been using Quantum Framework for more than eight years, which has been really helpful. Perfecto supports Quantum Framework, and this is where we can get Selenium as well as APM functionalities, which is really helpful for us. 

They have in-built functionalities, which are very useful for us, e.g., loading the APK and applications. We can also switch contacts from mobile to web and web to mobile. Those are all really helpful for us when implementing the new functionalities into Perfecto by using Selenium as well as the APM.

View full review »
FA
Automation Tester at IGM Financial

We've been able to automate test cases that are mobile-related. That's helping a lot with our automation. Obviously, you can't automate on a physical device, it's something that we do on a virtual machine. The fact that we can automate on mobile, Android and iOS, is working well for us.

We use the CI Dashboard. Perfecto's analytics are very important because that is where we have our test results, which we later input into Jira as well.

We use the device lab in our testing operations and mobile is one of the things we use. We have multiple devices, about 20, including Android and iOS. That's what we are running our automated test cases on. The device lab is extremely important because we are doing our automatic test cases and even our manual testing there. We don't have physical devices. We go into the Perfecto website and use all the mobile devices available on the cloud.

Also, we're working in Agile and we need results ASAP. The fact that the lab provides same-day access to new devices is extremely important to us.

We are mostly using Selenium and we are slowly putting our hands on Appium as well. That's working out well for us too.

View full review »
KC
IT Quality Control at a insurance company with 10,001+ employees

The reporting feature is really tough to find in some of the other products that are competitors. Having your CITB-type dashboard, where we can see the test results and see recordings of each test that passed or failed, is probably one of the distinguishing aspects of Perfecto. We also recently started using SSO for login which has streamlined our onboarding process and improved security.

View full review »
FZ
Associate Software Engineer at a financial services firm with 10,001+ employees

The automated test reporting functionality is the most valuable feature. We use the CI Dashboard. It's very important as it is the main reporting tool for our automated tests.

Another important feature is being able to manually control a device.

When it comes to executing cross-platform testing, we only use two platforms, but it's still pretty good.

We also utilize Perfecto’s cloud-based lab to test across devices, browsers, and OSs, and that is really important as well. The lab provides us with same-day access to devices.

View full review »
PY
automation tester at a computer software company with 10,001+ employees

Mobile testing is the most valuable feature as it has reduced dependency on physical devices. We are located offshore and we don't have the physical devices, and shipping physical devices after every new release would be a difficult task. But with Perfecto, it is easy.

Perfecto’s cloud-based lab to test across devices is very important because as a banking and financial project, we need to ensure that our app is working on all devices, irrespective of version.

View full review »
PM
Automation Consultant at a consultancy with 10,001+ employees

The CI dashboard tool is very good, as is the Live Stream monitoring. Whenever I want to monitor execution, I can open multiple tabs in Perfecto and it is easy for me to refer to the CI dashboard and the Live Stream. 

It's also very user-friendly. I can see and locate any issues that come in and it's easy to see which test case has failed. Under the Perfecto job name, if you click on it, it will show how many passed, how many failed and due to what issues, such as "element not found" or "object not found" or a Java error.

It also has video recording with the screenshots which is really good.

Another very good feature is the cross-platform testing. We are able to execute in multiple browsers and multiple Chrome, Safari, and Microsoft Edge versions. We have the capability in Perfecto to develop scripts. We just need to capture the capabilities of the different browsers and update them in our Quantum framework and we are able to execute. It's very easy. 

There is an option in Perfecto to capture the capabilities using the tool itself. If we want to run it in, let's say, a particular Android device, we can capture it from there and put it into the code and we can update the information and execute.

The range of open-source technologies that Perfecto supports is really good. I don't see any issue in using Perfecto for that.

In terms of its reporting and analytics, whenever I'm doing any execution for a new release, I just put in my release number and it's easy for me to look for what happened in the test case execution I ran for the previous month's release. I can easily locate it and I can get the information.

In addition, I can schedule the devices I need in Perfecto and I can block those devices so that other users on the team do not try accessing them.

View full review »
VG
Regional Head Customer Experience at a financial services firm with 10,001+ employees

The most valuable feature is automated testing.

View full review »
JR
Prin IT Bus Systems Analyst at a healthcare company with 10,001+ employees

One of the good things about Perfecto is the scalability that it provides. We are able to quickly scale up the devices. We can choose Android devices, web-based applications, or whatever we need, and we can scale up quickly and easily.

View full review »
AZ
Responsible for Test Management at a tech services company with 10,001+ employees

The most valuable aspect of the solution is that it covers all types of devices on the market allowing you to test different versions of an operating system.

View full review »
it_user510336 - PeerSpot reviewer
Associate - Projects at a tech services company with 10,001+ employees
  • Cloud-based architecture for manual testing
  • Performance testing & automation of browsers, mobile devices and gadgets
  • Enhanced reporting with video recording and proper steps
  • Wind tunnel
  • Appium + Selenium integration
  • Best-ever tool for RWD apps
  • Parallel execution of almost any browsers, mobile devices and gadgets
View full review »
it_user173214 - PeerSpot reviewer
Test Lead at a tech services company with 10,001+ employees

The cloud features are the most valuable.

View full review »
Buyer's Guide
Perfecto
March 2024
Learn what your peers think about Perfecto. Get advice and tips from experienced pros sharing their opinions. Updated: March 2024.
765,234 professionals have used our research since 2012.