Improved overall efficiency and is stable and scalable
Pros and Cons
"On a scale of one to ten, I would give OpenText UFT One a 10 because it is a reliable product, it works, it's as good or better than similar solutions especially because you get technical support from real people. Additionally, upgrades are always provided on a consistent basis."
"They need to reduce the cost because it is pretty high. It's approximately $3,000 per user."
What is our primary use case?
The use cases for OpenText UFT One vary from one department to another. We've got so many applications within Dominion Energy, but as of now, most groups are scripting the test cases themselves, even though they're not programmers and they don't have a true understanding of Visual Basic, which is a language used to script QTP. So the groups out there are doing it independently. I think they're doing mostly a record and playback, data-driven approach, which means they parametrize the data. But they're not specifically programmers, they can't make those scripts very sophisticated. And that's what I'm seeing. So it was my suggestion that we develop a framework for them in Selenium.
How has it helped my organization?
I don't think that OpenText UFT One has really improved it much. Until we move over to a framework where they don't have to spend so much time in creating data-driven scripts that become obsolete once a new version of the application becomes available. It may be doing some things for them, but I think it's probably improved their overall efficiency by maybe 20%. But once they have the framework, I think they will be able to operate this framework 24/seven in unattended mode. And that's when you see 100%, 110% improvement in efficiency. So we're not there yet.
What is most valuable?
We're not using the web services testing piece. They should, but I think they're using other open source tools such as Postmaster. But they're using QTP strictly for scripting automation test cases.
What needs improvement?
In terms of what could be improved, they need to reduce the cost because it is pretty high. It's approximately $3,000 per user and if we're going to spread this throughout the organization, we'll need to spend a whole lot of money. The company can afford it, but we're going to try to promote Selenium as the open source automation tool.
All of these automation tools are a tad finicky. They tend to freeze on us once in a while and we get an 85% pass ratio every time we run them, but 15% of the time these tools will fail. And it's not the tool, it's that the browser that they're opening may freeze up when it's time to do something on an application. I haven't looked at Selenium yet. I'm going to get some exposure to it later in the year or next year. But that's the tool that I'm going to focus on and replace QTP with. Because Selenium is free of charge and it's the standard in large corporations these days.
As for what should be included in the next release, I don't know much about that because I haven't used QTP in a while. I don't know how much better Selenium is than QTP except for the fact that it's open source. But as far as the features are concerned, I was okay with using QTP back in 2007 when I used it.
769,789 professionals have used our research since 2012.
For how long have I used the solution?
Since I'm not an automation tester, I last used OpenText UFT One in 2007. But now I'm promoting it. I'm also promoting Selenium as an open source solution for future automation testing because the company can set up that framework and everybody can use it. And I'm having a meeting with the users next week on that. So we're going to be promoting Selenium over UFT.
But I have used UFT within the last 12 months.
What do I think about the stability of the solution?
I think it's a stable product because it's been around for well over 14, 15 years now. And I think it's stabilized QTP and UFT.
What do I think about the scalability of the solution?
I think OpenText UFT One scales very well, but because it's not widely used... You can use one license per seat or per user who's automating it. So it doesn't need to scale, it works well enough with one single license per user. It's not meant for more than two users using the same license anyway.
Mostly developers use this product. They have a development background in Visual Basic and the use of the tool. With my current client, it's the business analysts that are doing the automation using this tool and it's not being used effectively. You have to have some form of development background, especially in Visual Basic.
How are customer service and support?
I've never used it for QTP or UFT, but I know some people who are supporting this product in the client site. They're okay with it. They get a response within 24 hours.
I'd give support a nine out of 10.
Which solution did I use previously and why did I switch?
I've always been familiar with QTP and UFT. The other product that's taken over the marketplace is Selenium because it is open source, free of charge. It is in 90% of all the organizations, whereas QTP I believe has lost the market share.
How was the initial setup?
The initial setup for UFT is straightforward.
What other advice do I have?
My advice to anyone regarding this solution is that if they have the money to purchase it, they could, but Selenium would be the first choice because it's more widely used.
UFT quite expensive. It's about $3,000 per seat, whereas Selenium is free of charge. So if you had 20 users who need to use it, you'd have to spend close to $60,000 on QTP plus annual maintenance costs. Whereas with Selenium, it's free of charge and you get all the support you need on the internet.
On a scale of one to ten, I would give OpenText UFT One a 10 because it is a reliable product, it works, it's as good or better than similar solutions especially because you get technical support from real people. Additionally, upgrades are always provided on a consistent basis. Whereas with Selenium, because it's open source, you're relying on the community to give you that technical support if you have issues and if you can't resolve them, there is really nobody to give you a patch or anything. So I think that with QTP having OpenText behind it, you've got some protection.
The price is only $3,000. I don't know how many QA analysts you would have in any given company. Probably no more than five or 10. So if it's a large corporation, it can easily afford $15,000 to $25,000. I don't see that being an issue.
Which deployment model are you using for this solution?
On-premises
Disclosure: I am a real user, and this review is based on my own experience and opinions.
With UFT 14.51 Micro Focus continues to improve and expand functionality
Last year I had the honor to attend the Micro Focus ADM Conference in Dallas, Texas. Participating with other users in a round table discussion, we spoke of what we loved most, and least, about the UFT automation tool feature set.
The results of this meeting can be seen in UFT 14.51. Micro Focus continues to show they are listening to their user base. They are committed to making changes large and small that makes UFT more user-friendly and efficient. Here is a quick dive into product changes, some undocumented, in 14.51.
Parallel Test Execution adds isolated execution
With a ParallelRunner utility, scripts can be executed on up to four different browsers simultaneously. Execution can be performed from a command prompt:
or by referencing a JSON formatted file.
Parallel execution was introduced in UFT14.50, but there was one drawback: It was difficult to handle events that caused conflicts when executed simultaneously. For example, if multiple tests attempt to perform an LDAP validation with identical credentials, UFT 14.51 resolves this with isolated execution using the ParallelUtil object. This tells other concurrent tests to pause so that the current code segments can execute without any overlapping interference.
You can see a demo of UFT 14.51 Parallel Execution in this short video.
'Open in Repository' speeds Object Repository access
A small change is in the context-sensitive right-click menu makes day to day work in scripts and the repository much easier. In prior versions, users could only jump to the Object Properties… dialog - which has no edit functionality- and then click on View in Repository in order to edit an object.
Now with the addition of the Open in Repository option, a path without the extra mouse click is provided. Users can now jump directly from the code to edit a problem object in the repository. This continues to reduce the "Clickitis" of UFT.
'Go to Definition' jumps to the function between linked libraries.
In prior versions the Go to Definition option only allowed users to jump from the Main Script to a library, or to a function within in the same library. The only way to jump to a function declared in another library was to search the entire project. Now users can jump to function definitions between external libraries.
Spy has a new Hover Mode
Users have long awaited the Object Spy to detect objects that appeared only when the mouse is floated over. This is a switch found at the top of the tool interface, and a message appears indicating the new mode is operational when activated. As a reminder, the Spy tool has undergone a lot of improvements in recent releases. It can display the properties of two objects at the same time for comparison. And it's no longer modal, so the user can now move the main IDE window freely, and even edit code, while the Spy tool remains open.
The tally of Fail and Warnings in the Results Viewer now report a count and percentage of reported Pass results.
Users can quickly identify tests with a large count failure from those with single faults from a high-level standpoint. This greatly addresses the prioritization of maintenance when multiple tests in a test suite have failed.
Persistent Watch List with Undocumented Methods Revealed
Set variable or object property in the Watch list and save the test. Restart UFT, reload the test and view the Watch window. All tracked variables and objects will return without retyping.
The Watch window holds another surprise. Many, if not all, undocumented methods are now displayed including .Highlight, .HighlightAllMatchingChildren, .Init and .MakeObjVisible. An additional issue has been resolved where only a partial alphabetical list of object properties was retrieved due to a timeout.
More support for StormRunner Functional with new AOM Methods and Properties
StormRunner Functional uses Amazon Web Services to create virtual test labs for testing Web and Mobile devices:
Six Different Screen Resolution:1920 x 1080 to 800 x 600
One of the major advantages of StormRunner Functional is that it spins up virtual environments only for the run time duration. Add the ability to run tests concurrently and this makes testing in the cloud significantly faster.
Pricing, Setup Cost and Licensing
UFT is a licensed product, but it has some advantages that make it a viable choice over other open source options.
The first cost advantage becomes evident when Micro Focus tools are combined with StormRunner Functional to run tests in the cloud. SRF runs under Amazon Web Services, spinning up test environments on an as-needed basis. This allows Micro Focus to offer customers a flat rate charge, instead of a pay-by-the-minute plan. This offering can be very attractive to budget-conscious users who have had the surprise experience of paying for machine instances that were inadvertently left idle for days or weeks in the cloud.
UFT has another advantage with the choice of VbScript as its programming language. It's easy to learn and quick to write test scripts. Where Java and Selenium require a much higher level of code density, complexity and multiple third-party support tools by comparison. The ROI of using an open source tool can be lost when the time and manpower needed to get up and running quickly is lost to a high learning curve and the lack of an on-demand customer support line.
Lastly, successful automation projects expand from an initial proof of concept application to other applications under other technologies. So another advantage to consider is the number new and legacy web technologies, including terminal emulators and Windows thick clients supported by UFT.
This makes UFT the “everything and the kitchen sink” of automation tools, with an easy to learn language, a solid history, and extensive support resources.
Initial Setup
Setup remains straight forward taking about 30 minutes to complete, including one system restart. The tool installs the bare minimum of add-ins. To add more takes less than 10 minutes.
Room for Improvement
With this release, the list of good features dominates over those on the desired list. But there are a few changes I'd still like to see.
• A user is forced back to the main script during debugging. 90% of code development and issues occur in function libraries. So having the tool jump back to the main script from it's last line of execution is problematic making debugging overly tedious. Fortunately, this is the only remaining source of "Click-itus" in the product.
• No RegEx support of integer properties. From an advanced user perspective, if the tool allowed for the RegEx "[1-9]\d+" in the Height and Width properties, a collection of visible objects could be returned. This would eliminate the additional code to search the outerhtml properties of all returned objects for textual cues like "DISPLAYED".
Other Solutions Considered
I have worked QTP/UFT and Selenium/Serenity engagements; however, I do compare functionality of other tools in my spare time, including Micro Focus LeanFT, TestProject.IO, SmartBear TestComplete, MABL and AutoBloks from the creators of Test Design Studio at Patterson Consulting .
Other Advice
Be sure to have new automation engineers trained beyond basic YouTube videos. Avoid on the job training. This will prevent rookie mistakes, producing more robust scripts and less maintenance. Micro Focus tool training is available from both Orasi and RTTS.
Micro Focus' Unified Functional Testing tool is returning as a major contender in the test automation field. This release continues to show a real commitment to adding more ease of use and providing more functionality to users.
Disclosure
I am a real user, and this review is based on my own experience and opinions.
Disclosure: I am a real user, and this review is based on my own experience and opinions.
While I personally have not printed code for several years, it is something missing compared to other IDEs. Both Eclipse and IntelliJ have a Print Code functionality.
Eclipse has a basic print from a the active window.
IntelliJ has the significantly advanced capability over Eclipse with printing selected text, the active window, or the entire project. UFT certainly should include this functionality in upcoming releases.
The new parent company of UFT, Micro Focus, is showing their user base that they get it.
They really get it! Changes in this point release alone remind me of the days when QTP was under Mercury Interactive's vision.
Here is a quick dive into many product changes, well beyond what is mentioned in the What’s New section of UFT’s User Guide.
Easier download
At times it felt like HP had a hard time getting out of it's own way to get new users to download and try UFT with a 60 day trial license. Micro Focus has begun to streamline that process. Here is the link to get exactly what you want. No muss, no fuss.
Yes, it records and plays back on Chrome and Firefox too.
Like Internet Explorer, each browser has a UFT Hook extension that must be enabled before object recognition will work.
It can now record and playback on Chrome with Polymer's Shadow DOM as well.
More Tech Stack Support
UFT 14.02 extends support to WebAgGrid objects and Firefox v.57.
Return of the Run and Spy buttons
My last review praised a new ability in UFT 14.01 allowing the Spy utility being left open while writing code, but lamented the loss of the Run and Spy buttons when a Function Library window had the focus. These buttons are now restored to their original state, substantially reducing "Click-itus" in the product. There are other design changes that really eases day to day use:
• Double-clicking a function in the Toolbox takes the user to the source code, rather than adding code to a random cursor location.
• Floating mouse-over popups now appear below the code, making it easier to select objects to send it to the Watch window.
• Sending values to the Watch window works on the first try, even if the Watch window is not active.
• There are fixes with Auto-code generation functionality.
• The Spy window split bar is unlocked, giving the user more view space to the property values by default.
Test Combinations Generator (TCG) gets a significant new option: Pull from UI
The TCG tool was introduced in UFT 14.00 to help users with large Data-Driven test design. It is actually two tools in one: First it is an extensive random data generator. Second it can take small groups of data and generate large combination scenarios organized into Happy Path, Error path and Regression groups.
The new Pull from UI feature now allows users to import data from populated WebLists in an application.
With a simple point-and-click, a list of countries above is imported into the TCG tool and then available for use in the Global table.
To understand why sourcing data from a web list is important, one should have an understanding why the Test Combinations Generator is so beneficial to data-driven testing:
• Need a quick list of random user names? But only with the letter "X"? It does it.
• A range of random dates? Available in nine common date formats? Yup it does that too.
• Same for random URLs, IPs and MAC addresses.
• How about a list of randomly generated emails and passwords that meet specific business rules? A little RegEx gets it for you.
• Combine eMails and Passwords and you have an instant set of unique credentials of simulated new users.
• Need a list of random part numbers in a custom format? Use the Regular Expression data generator to create combinations.
Hidden TCG Benefit: Learn Regular Expressions
A side benefit here about RegEx: The TCG tool comes with several pre-configured RegEx samples that can be modified. This means it can be used to experiment and learn RegEx much the way many of us learned to code: by modifying a working example and analyzing the results.
Test Combinations Generator (TCG) can really mix it up
While all this data can be exported into Excel sheets for multiple uses the Test Combinations Generator does not stop there. It can take a few short columns of data values and create larger scenario data sets. Take 10 first names and 10 surname names and TCG can create 100 family members. Add random dates in an 80 year span and you have a simulated population sample.
When using four or more types of data, the combinations can quickly become exponentially large. TCG tops out at just over 65,000 combinations, which might take months or years to run every combination just once.
So Micro Focus offers Pairwise, as well as Triplewise, combination sets. This achieves the most efficient combination coverage. This means your automation script can look for problems when combining two or three list values, without repeating every possible combination.
Finally, the TCG tool further allows the user to identify Happy Path data, as well as Error Path data.
This means you can further segment your data combinations into a small Smoke data set, a Negative data set, with the remaining being the Regression data set.
These data sets can be selected upfront for easy instant access to data driven testing from the Global data table.
To see more of UFT’s TCG tool in action, check out this video:
Another welcome change is online access to the Product Availability Matrix from the Help menu. In prior releases, this document has often been hidden in the Documents folder under the Start menu. It informs users what environments and configurations recent versions of UFT requires to run efficiently.
Stability Issues
UFT 14.02 remains extremely stable. It's been my tool of choice for nearly two decades because it is solid.
Scalability Issues
Scalability is entirely up to the framework design.
While Record and Playback is available for new users to learn the tool, it will result in fragile, high maintenance test suites. This is true of most automation tools, so a hybrid framework design approach is always highly recommended.
Fortunately, UFT is extremely flexible in design. Advanced developers can go so far as to design a framework which translates plain English like:
With the Business Process Testing (BPT) option, non-technical users can easily build test cases inside of ALM (Application Lifecycle Management) from scripted components designed by automation engineers.
Enhancing Object Methods
Scaling object class methods to add new functionality or extending existing methods is achievable with the Function Definition Generator Wizard.
This allows automation engineers to fully customize UFT methods, as well as add new functionality. These methods appear in the Intelli-sence dropdown of object classes. Even descriptions appear on the interface to help new team members who are just learning a new framework design.
Previous Solutions
Prior to using VBScript-based UFT/QTP, I used Mercury Interactive's C-based WinRunner up until 2004. Product support was discontinued in 2011.
Initial Setup
Setup remains very straightforward and takes about 45 minutes, including one system restart. The tool installs the bare minimum of add-ins. To add more takes less than 10 minutes.
Room for Improvement
With this point release the list of good features far exceeds those on the desired wish list.
• When a debug session ends, UFT forces the user back to the main script. 90% of code development and issues occur in function libraries. This is the only remaining source of "Click-itus" in the product.
• RegEx support of property strings, but not integers properties. If the tool allowed [1-9]\d+ in the Height and Width properties, the returned object collections would exclude all non-visible objects.
• The tally of Fail and Warnings in the results viewer lack a count of reported Pass results. From a high-level stand point, a Test with one failure and 99 Pass results looks just as bad as a test with 100 Fail results.
Pricing, Setup Cost and Licensing
For the price of five automation licenses, you simply would not be able to hire five manual testers.
Over two years, a successful automation project can mature to a 24/7 test execution schedule that outweighs the equivalent cost of manual testers. It also tends to expand from a initial proof of concept to multiple applications.
While UFT 14.02 is a commercial product, the sheer volume of internal tools focused on ease of use gives it an edge over other open source products by speeding test development.
UFT is the “everything and the kitchen sink” of automation tools with an easy to learn language, a solid history, and extensive support resources.
Other Solutions Considered
I only work QTP/UFT engagements; however, I do compare functionality of other tools in my spare time, including Micro Focus LeanFT, Selenium, and SmartBear TestComplete.
Other Advice
• Be sure to have new automation engineers trained beyond basic YouTube videos, and avoid on the job training. This will prevent rookie mistakes, producing robust scripts and less re-work in the future. Micro Focus provides tool training, as does Orasi and RTTS in New York.
• Consider expanding your Test Automation Engineer's toolbelt with Test Design Studio from Patterson Consulting. It includes a static code analysis tool, similar to Lint, but tailored to UFT. This allows UFT developers to efficiently analyze entire entire code base for errors in a single sitting, not just at run-time.
Micro Focus' Unified Functional Testing tool is shaping up to return as a major contender in the test automation field. This new release shows a real commitment to ease of use and hints at providing more superior functionality to users in the coming year.
Disclosure: I am a real user, and this review is based on my own experience and opinions.
By default UFT records and stores all objects in its Object Repository.
Before we start start, let's clear up any confusion new users may have: HP's Unified Functional Tester 12.50 (UFT) is the latest version of the QuickTest Professional (QTP) formerly from Mercury Interactive. They are essentially the same product, in the same way Word 2007 and Word 2013 are the same. There are a few new features, and all the stuff you know and love is still in there. It's just that, for long time users such as myself, someone has tidied up a bit, and I can't find anything. At the end, I will discuss a few things about HP's latest product, LeanFT. Also note that I am organizing these features starting with what beginners can handle, and then what they can leverage as their skills advance.
The features that are most valuable in UFT include the built in Excel data table, the Automatic Object Identification and the Record and Playback feature. That said, I will ask advanced automation engineers to bear with me, as that last answer will have raised huge red flag.
The Excel data tables are visible from the main interface for easy access. UFT can easily load external Excel worksheets to replace any that you have displayed. So if you want to edit them in Excel instead that's perfectly fine. Advanced users can have several test cases in individual Excel files, or as multiple worksheets in a single file. Aside from all the features Excel provides, it also allows design of data-driven tests, keyword-driven tests, and hybrids.
If you ever wonder why UFT licenses are just so darn expensive, it's because the developers bent over backwards to provide the "Kitchen Sink" of object recognition. By default UFT records and stores all objects in its Object Repository. It has at least 15 different add-ins to support multiple technologies, including .Net, Java and even Terminal Emulators. It automatically uses a minimal combination of unique object properties to do this. And in the case where no unique properties can be used, the object index is used as a last resort. However advanced users can also build objects on the fly in code, either with Descriptive Programming or a Descriptive Object. All three approaches support CSS, xPath and Regular Expressions, so that you can reduce maintenance if portions of the object property value changes regularly. There is even a feature to identify objects by image called Insight, and another to create hotspot virtual objects. While these last two are not as reliable as Descriptive Programming, they are there as your options of last resort.
Next, to clarify: Record and Playback should never be the primary way to create automated test scripts. The code is not optimized, and will often be so brittle that it will often take a few attempts to create a script that can simply be executed repeatedly. This can cause new users to inadvertently trash the product online from sheer frustration. Consider this: If you had only a hammer as a tool, would tell the internet it sucks because it's only good for building birdhouses and doghouses, but not suited for completing residential buildings? I should hope not. The best way to look at Record and Playback is as a tool with a few specific purposes: First if you are unfamiliar with the VBScript language syntax, it builds code automatically for you to inspect and learn. Second it comes in handy when first building objects with descriptive programming. It allows you to quickly inspect how UFT would choose to identify an object, if your initial attempts are failing. Lastly it is a good way to quickly create a Proof of Concept, showing that, yes, the tool understands your particular web application.
The best example of improvement within the organization is a closer working relationship with my fellow manual testers. While project managers would like have me automate 100% of everything, and secretly kick their manual testers to the curb, it's simply not possible. To avoid the natural animosity this idea can promote, I work with a manual tester on each regression release. Essentially I work down through a list of the tests I have automated, and they work up executing the remaining manual tests. We both end up in the middle, finishing a job much sooner than expected. We both feel we are an essential part of the team, and we don't feel overwhelmed by the amount of work we are expected to perform.
It also puts us on more even playing field with our Developers. Most of them think I am still just a blackbox manual tester with a tool they view as nothing more than a toy. On several occasions Developers have stated that the defects I have uncovered are caused by the tool itself. I have had accusations that it's my tool that is causing memory leaks, or that it has covertly acquired Local Admin Rights and have changed all manner of random settings. I have found, repeatedly, that I must defend and prove that I don't have the skill, nor the time, to create such fantastical code. After proving two or three items are actually the developer's issues, usually on a system without my tool installed, I generally get some "street cred" with developers. After that point we work together to be more efficient.
Deployment and setup of UFT can't be much simpler.
HP has a policy that software updates for the QTP/UFT products are only available to licensed users with a service agreement. This is fully understandable from a business perspective. However this policy extends beyond version upgrades to software patches, and it backed up by HP's highly paid lawyers. The problem this poses is that any potential customer that downloads the tool for use with a 30-day trial license must work with an unpatched version that is often less stable than the patched version in use by licensed long-term customers. The problem here is that HP wants potential new customers to try their product, but policy prevents them from showcasing that product in the best light, thus shooting themselves financially in the foot. I would sincerely hope HP CEO Meg Whitman, who is a brilliant businesswoman, might have a chance to read this article, recognize the policy flaw, and resolve it for the betterment of her company's bottom line.
That said I would recommend anyone interested trying the tool for the first time to use the latest release of UFT which is 12.50. If you have an earlier version such as 12.02, even with a patch, I would recommend the upgrade as well. This also gives you access to LeanFT at no additional cost which we will discuss shortly.
I would like to see the "double clicking a function in the keyword list takes me to the function source code" changed back to "double click a function keyword takes me to the function reference". I would like to see the person who thought that would be a grand idea removed, to prevent other such grand ideas from taking root in the product.
Seriously I would like to see a Static Code Analysis component added to the product. For those who are unfamiliar with the advantage this tool provides, it is simply this: It scans all your project code all at once, and gives you a list of where all potential errors exist. Which is significantly better than finding errors one at a time at run-time. Amazingly this professional level tool is available on the internet for just about every known programming language... except VBscript. Google it if you don't believe me.
In fact there is paid Static Code Analysis tool specifically for UFT users called Test Design Studio from a third party at patterson-consulting.net. I highly recommend including a license to compliment the HP IDE.
Get the latest patch for UFT.
Don't use more than two Actions. They only serve to needlessly complicate your project. Functions work just as well without the overhead of additional Excel pages that probably with never get populated. Similarly, if you find yourself struggling with the decision to use a Function or a Sub, make it a Function and forget about it.
Don't let the tool organize the folder structure of your project. It's buried pretty deep by default. It's easier if you have a project folder off the root of a drive that contains the folders: Tests, Functions, Environment, Results and Documentation.
Learn what Regular Expressions are and how to use them for simple pattern matching. Don't be put off by their complexity, a basic understanding goes a long way in this field. Much the way a little salt will make a bland soup better, but too much ruins it.
There is a lot of flexibility and functionality in UFT. You can store data in many different places. It does not mean you should try to utilize every one of them. Anyone who has worked with Photoshop, as an example, knows there are a hundred imaging functions, but at most six are all you need to be proficient at it. The same goes for storing data with HFT.
If you can't get the tool to recognize an object on the first day, go make friends with the developer. Show them the list of Add-In support UFT provides and ask them to point out which ones they are using in the environment. Then save them at the last second from being hit by a bus. Some day down the road you will have to call in that favor.
Roll your own results reporter. My results go out to another Excel file with links to screen captures.
LeanFT is a hybrid solution for those who are looking to take advantage of Selenium. You might think one of those advantages might be speed. Because the most commonly used language of Selenium is Java, there is no doubt that it has speed over UFT's VBscript. However, this has about as much meaning in the automation field as noting that a rocket-equipped Jaguar will out-pace a Bugatti on an open stretch of desert road. Bring them into the reality of city driving with streetlights, hairpin curves and pedestrians and speed makes it more likely both car and animal will crash into a building.
The major weakness with Selenium that LeanFT addresses is it's identification of objects. The problem here is that much of the object identification available to Selenium users is hard to decipher, and difficult to maintain, particularly with xPath. Take this example:
SPAN. SPAN[2]. A. SPAN? This is this an index to an unknown level of HTML code and impossible to maintain when it changes.
LeanFT brings to Selenium many of the object identification techniques noted above that QTP/UFT engineers how been comfortable with for years without being forced to learn the complexities of xPath. This is important to me as I can now port my own custom object recognition technique from UFT directly into Selenium. In addition reporting results can be sent back to HP ALM (formerly Quality Center). LeanFT also has access to other common frameworks like TESTNG and JUNIT, as well as source control tools such as GIT and SVN.
I personally have a relationship with contacts inside HP. Years
ago I made the decision to be a Track Speaker at a Mercury
World conference in Orlando (which is now HP Discover in Las Vegas and
London). My decision was a financial one, I simply could not afford the
airfare, the hotel and the ticket into the conference. But I learned that, if
accepted, I could at least attend the conference for free. I did
this to meet others like myself face to face and maybe get on the inside
track. I found ten other people who knew more about this tool than I do,
and I am friends with nine of them. At that initial conference I was
fortunate enough to sit in on a discussion group about the future of the
product. At the end they asked for volunteers for the Beta program and I made
sure they had my number.
Disclosure: I am a real user, and this review is based on my own experience and opinions.
I did want to post an update. In regard to HP's rule on obtaining patches for their products under the 30-day trial licenses it now can be done without an SAID. These patches are now available for download to anyone with a HP Passport account. There are several entry points from a Google search to do this. You can try this URL to start: ovrd.external.hp.com
The GUI has automatic settings and doesn't require much skill to use
Pros and Cons
"Micro Focus UFT One gives us integration capabilities with both API and GUI components. I like the user interface. It doesn't require that much skill to use and has automatic settings, which is useful for users who don't know what to select. It also has dark and light themes."
"The AI feature needs improvement. For banking applications, we input formatted text from documents, but the AI feature is recognizing three fields as one field, e.g., for a phone number, it puts all 10 digits in the international code or country code. Then, the script fails."
What is our primary use case?
We use it for automation. It helps to automate test scenarios for graphical user use cases.
How has it helped my organization?
Historically, we have faced a lot of maintenance issues with automation using traditional UFT, because UFT has a mechanism for identifying an object where you have to add object properties. However, if a change happens in the application and your object properties change, then you have to go and update the object properties again, only then can you use those scripts. So, we were using a lot of personnel for script maintenance. Whereas, in UFT One, I like that our maintenance costs have been reduced by a lot because UFT One is using an artificial intelligence feature to identify objects visually.
We use it to do multi-platform testing.
What is most valuable?
UFT One Automation provides Codeless Test Automation.
The solution will automatically run a script, so you need less knowledge to run a script.
OpenText UFT One gives us integration capabilities with both API and GUI components. I like the user interface. It doesn't require that much skill to use and has automatic settings, which is useful for users who don't know what to select. It also has dark and light themes.
It improves automation efficiency.
What needs improvement?
The AI feature needs improvement. For banking applications, we input formatted text from documents, but the AI feature is recognizing three fields as one field, e.g., for a phone number, it puts all 10 digits in the international code or country code. Then, the script fails.
For how long have I used the solution?
I have been using UFT for the last seven or eight years. UFT One was just launched three or four months back, so I have been using it for a couple of months.
What do I think about the stability of the solution?
It is reliable. Sometimes, the GUI does crash.
What do I think about the scalability of the solution?
It is highly scalable.
There are around 150 users of UFT One with 8,000 test scenarios running four times a month. We are also running around 500 scripts in UFT One.
How are customer service and technical support?
Our internal team is sufficient for technical issues.
Which solution did I use previously and why did I switch?
We have used a lot of different tools. We have used Selenium and Python as well as Java-based REST API for regular testing. With UFT one, we have all the solutions under an umbrella, so we don't have to think about other tools. It also supports API and HTML testing. Selenium only supports Java, and there is no support for HTML.
How was the initial setup?
We didn't need to do too much with the initial setup because there is an installation team.
It takes one or two days to create test automation scripts.
What was our ROI?
Object maintenance is reduced.
What other advice do I have?
We have not yet implemented the license for the AI features. However, I got a chance from OpenText to join a Hackathon for India when they launched the product, which included the AI feature. I am hoping that my company will implement this feature soon because the solution's AI capabilities will reduce my test creation time.
Every day, tools are getting smarter. UFT One is like this.
Before implementing, do a demo with your existing applications.
I would rate this solution as an eight out of 10.
Disclosure: PeerSpot contacted the reviewer to collect the review and to validate authenticity. The reviewer was referred by the vendor, but the review is not subject to editing or approval by the vendor.
Thank you for sharing valuable feedback about your experience with UFT One. We are
glad to hear that UFT One has not only reduced your maintenance cost, but has
also sustained its purpose: to be a reliable and scalable testing tool that
serves your business needs.
As always, your business means a lot to us, so thank you again for taking the time to review UFT One.
Outstanding UFT solution, but there are issues with the scripting
Pros and Cons
"The high-level security, high standard and compatible SAP are great."
"The solution does not have proper scripting."
What is our primary use case?
We are currently using it for migration.
How has it helped my organization?
Micro Focus UFT One is useful. However, there is an issue with the scripts. We are going to collaborate with multiple automatic specialists to identify the problem. If we can fix the issue, we will continue with UFT, otherwise, we'll switch to other automation tools.
What is most valuable?
The high-level security, high standard and compatible SAP are great.
What needs improvement?
The solution does not have proper scripting, which impacts the solution. We are currently deciding whether we want to keep the UFT and will decide by the end of December. We paid a lot of money for the UFT, and we will only drop it as a last option.
For how long have I used the solution?
We have been using this solution for four years. It is deployed in the client-server application with SAP.
What do I think about the stability of the solution?
The performance is not great, which is why we are currently conducting a review.
What do I think about the scalability of the solution?
The solution is very scalable for a UFT. We have more than ten people using this solution.
How are customer service and support?
There is a gap in technical support which is also part of our review. We've raised issues in the past, which have not been fixed in two years.
Which solution did I use previously and why did I switch?
We used QTP and LoadRunner in the past.
How was the initial setup?
Our deployment was completed in-house, and we have an in-house software development and architecture team. We do all our products and services and also provide services to third parties.
What other advice do I have?
I rate this solution an eight out of ten. Micro Focus UFT One is outstanding. All HP processes are excellent. I used to use HP Test Director, HP QC and HP ALM. So I am confident that Micro Focus UFT One is useful.
Disclosure: I am a real user, and this review is based on my own experience and opinions.
Enables us to quickly obtain detailed product behavior information, but continuous testing needs improvement
Pros and Cons
"Has improved our organization by allowing us to obtain fast, detailed information about the behavior of our products and to supply this to the customer, enabling us to work together without the need for special programming knowledge."
"There is a lot of room for improvement when it comes to friction-free continuous testing across the software life cycle, as a local installation is required to run UFT."
What is our primary use case?
We primarily use the solution as a front end for testing for our customers, to automate installations, for behavior testing, and for various types of API testing. We mostly use the technology on our websites, and sometimes on older technologies, such as for Oracle Forms applications.
How has it helped my organization?
One of the ways the product has improved our organization is that we are able to quickly get detailed information about the behavior of our applications, and we can provide this information to our customers through screenshots and additional information so that they can also easily check the reason for the defect or bug. We can work together without our customers needing special knowledge of programming. This is very important.
UFT allows us to install our applications much more easily, without our customers having to do anything. They don’t even need to click on anything. We can use UFT One to install via scripts. This eases the installation process.
The solution has allowed us to reduce test execution time. If we use it in continuous integration or in headless mode, it improves performance. Between the normal run mode with debugging, and the fast mode in Jenkins, it can reduce it by about 30 percent. That's a lot.
We can run the solution on virtual machines. This greatly affects our ability to control machine configuration and allocate appropriate resources for testing. We wouldn't be able to conduct tests or to carry out work without this solution. This is both very helpful and useful and we consider this a necessity.
What is most valuable?
The most valuable features include
the simplicity with which the product can be maintained
the ability to reuse its components
the record and play
AI
We haven't been using the AI feature for very long.
These features allow us to provide good functionality to all our customers without the overhead of maintenance costs, while at the same time allowing us to work with many customers with varying capabilities on different projects. With only a few technicians we can help a lot of customers.
Running the solution on virtual machines allows us to run tests in parallel. It reduces a lot of the time it takes to test or to do certain kinds of work. We are dealing with customers who give an API to their customers and they're using our tools in the background. As a result we must use it to scale the load for these tests. This is a very important and useful feature.
What needs improvement?
There is a lot of room for improvement when it comes to friction-free continuous testing across the software life cycle, as a local installation is required to run UFT. Most of the time, administrative rights are required which necessitate much trouble to integrate it seamlessly. When integrated, it works fine, but to maintain it in CI, special systems and privileges must be utilized. This is challenging for us.
In addition, UFT One has a Jenkins plugin that provides us the connection we need to OpenText so that we can obtain our UFT test cases. The problem is that the plugin does not come with exception handling, meaning that if we enter the wrong credentials we don’t know why it does not work. This can lead to the Jenkins server crashing.
Another issue is that we can't address the UFT output to the Jenkins console. This means that when carrying out our tests in a continuous integration server, we cannot know what the UFT tested, step-by-step.
The usability can also be improved. When we receive new versions of UFT, some of the icons are altered so that things are not recognizable to us or to the customer.
Another issue is that the application requires slow work. If you go too fast while debugging, the Step Over button may easily change to the Stop button.
The Git integration is also a point when it comes to continuous integration. There are aspects that are not recognized by Git. We cannot do a side by side comparison of changes, such as comparing the QSL side and the object repository side.
When they updated UFT from version 14 to 15, they changed the data table structure of UFT, such as the first data line turning into the column name. This is a problem as our customers may have different versions of UFT. An example would be if we wish to change the data table of version 15 but a customer has version 14, it can be problematic. This destroys the tests.
Another question we have is why everything is in read mode during the execution. With other IDEs, like Visual Studio, you can change the variables while you execute or debug something, and this is not possible in UFT. It's only in read mode, so you can’t play with variables or objects.
Also on our list is the fact that UFT allows you to work on 11 or 12 tests. If you want to change something with search and replace, you can only change it in the 11 or 12 tests that are open in the solution. But what if we have a 13th test case that is not included in the solution? We then need to open that test after we have already searched and replaced. That's a little bit inconvenient because other IDEs give you the opportunity to make those changes everywhere, in every script, not only the 11 or 12.
We have already addressed some of these issues with technical support, but not all of them have been handled. For example, we brought up the issue of the icons changing with every version some years ago and nothing has happened. It gets worse and worse from version to version.
We also have menus and instructions for our customers, but because all the screenshots become outdated with the next version, we have to do maintenance on them all the time. And it’s not because of new functionality. Most of the time, only the icon style and the design is changing and sometimes it’s the positioning that changes and we are not able to reconfigure it. We end up having to do a lot of work without any need for it.
The old VBS language can be a nuisance. It could be easier to use and it could be better integrated in continuous integration pipelines. And it could always be faster.
For how long have I used the solution?
OpenText UFT One has been available for around two years. We have been using it since inception.
What do I think about the stability of the solution?
The stability is very good, except for the example I mentioned regarding the data table. Most of the time you can switch to the next version without any problems. The old features and behaviors are, in terms of the code base, the same. It’s just that you have to find the icons, asking yourself “Where is my feature?” But the stability is very good.
What do I think about the scalability of the solution?
We use the product as often as we can. Between 50 to 100 people are using the solution. We are constantly looking for additional customers and projects so we have ongoing plans to increase usage.
The overall scalability is very good and utilizing the licensing server allows us to scale the solution as we need. One area which can be improved involves the running of instances on a single machine.
How are customer service and technical support?
Overall, if you are able to reproduce an issue, their technical support can help you. But sometimes it can be very hard to find a technician with a high level of technical background and knowledge of the product, so that they can understand the situation, the problem, and the behavior. This can be a challenge. Sometimes we have had to escalate to get a technician with the necessary background and knowledge.
Which solution did I use previously and why did I switch?
We utilized QuickTest Pro (known now as OpenText Unified Functional Testing) for between eight and 10 years.
How was the initial setup?
We found the initial setup to be very easy. It is very robust and leaves no room for making errors. The availability of config files for setting up all the installations from a single master configuration is nearly perfect. The customer would have no problem simply opening the machine and using it.
As for deployment, the time can vary. Sometimes there are only minor changes and sometimes there are a lot more changes. Including tests, and to be sure it’s working in all cases, it should take no more than one business day.
It’s the same for upgrades. OpenText support has advised us that, in case of an error or a problem with upgrades, they cannot be sure whether that problem would exist on a clean installation. So we always uninstall the entire product and install it on a clean system.
We use one or two people for deployment and maintenance, in the role of test automation engineers.
What was our ROI?
Even without being able to provide exact figures, this product has given our company a return on its investment.
What's my experience with pricing, setup cost, and licensing?
If you use it all the time and for different use cases then it is a good price. If you only use it one time a day for half an hour then it is pricey.
What other advice do I have?
The ability of the solution to cover multiple enterprise apps, technologies, and environments is very important to us and it forms part of our company policy. It is a point we had to validate before going with this solution. The reason for this is that we must meet the technical needs of our customers, many of whom lack a technical background.
UFT One provides cross-browser and desktop application support, although the cross platform support, which is not good, is not so important to us at the moment. These capabilities are important to us because our customers are using different kinds of technologies, some that are newer, some that are very old, and all kinds that are in between. To provide a good solution, the cross-browser and cross-platform functionalities are very helpful and necessary.
UFT One gives us integration capabilities with the API and GUI components, which is very important to us since we must occasionally alternate between the two. We can use the API to make calls through scripts, so we don’t have to use the GUI for UFT One. That’s why it’s important for us to have the REST API.
We can run the solution on virtual machines. This greatly affects our ability to control machine configuration and allocate appropriate resources for testing. We wouldn't be able to conduct tests or to carry out work without this solution. This is both very helpful and useful and we consider this a necessity. We have 100 percent usage of UFT on virtual machines -- All our instances are running on them. This allows us to help the customer access his application under test. The customer can configure the system with permissions and the like. All these points are, in some cases, not possible on hardware in our company, because of political restrictions, security reasons, et cetera.
The solution has allowed us to reduce test execution time. If we use it in continuous integration or in headless mode, it improves performance. Between the normal run mode with debugging, and the fast mode in Jenkins, it can reduce it by about 30 percent. That's a lot.
Overall, it's really easy. Try it out. There is nothing one can do wrong.
Which deployment model are you using for this solution?
On-premises
Disclosure: PeerSpot contacted the reviewer to collect the review and to validate authenticity. The reviewer was referred by the vendor, but the review is not subject to editing or approval by the vendor.
Good user interface, but its tracing functionality needs improvement
Pros and Cons
"It is a stable solution."
"They should include an automated feature to load backlog tests."
What is our primary use case?
We use the solution for API functional testing to analyze user interface, business logic outputs, etc.
What is most valuable?
The solution's user interface is good. It is easy to use for a nontechnical test as well.
What needs improvement?
They should improve the solution's tracing functionality. There should be an automated feature to load the backlog of test requirements.
For how long have I used the solution?
We have been using the solution for eight years.
What do I think about the stability of the solution?
It is a stable solution. I rate its stability as an eight.
What do I think about the scalability of the solution?
We have three solution users in our organization.
How are customer service and support?
The solution's technical support team is good. Although it could have been faster while resolving the licensing error issue, we encountered some time back.
How would you rate customer service and support?
Positive
How was the initial setup?
The solution is easy to install. The setup process takes a few hours and requires only one executive.
What's my experience with pricing, setup cost, and licensing?
The solution's price is reasonable compared to other vendors. I rate its pricing as a four.
What other advice do I have?
I advise others to speak with automation engineers to know the success criteria for the solution's proof of concept. Moreover, UFT needs to give insights on production status like Worksoft or Tricentis. Thus, I rate it as a seven.
Which deployment model are you using for this solution?
On-premises
Disclosure: I am a real user, and this review is based on my own experience and opinions.
Nice article. Have you used the Web Services module of UFT for API testing?