The ability to customize automation using code is the most valuable feature. With the release of 12.5, UFT now includes a LeanFT license which will plug in to Visual Studio and allow usage with C#.
A close second would be the Object Spy, i.e. how UFT identifies objects. Although you can code with CodedUI and Selenium, the object identification tool in UFT is far superior.
Improvements to My Organization
Due to cost, my current employer doesn’t use UFT. However, I’ve used it at many different locations in the past. Automation in general will always provide value in test coverage. UFT bridges the coding gap by using VBScript, which is easier for less technical personnel to pick up. This greatly increases the number those people who can use the tool. The competition will require a developer-level skill set to get the same functional benefits.
Room for Improvement
There are some command-line and other crude methods to integrate UFT into non-HP software suites. This area could be improved, but overall there is little incentive for HP to do so.
Use of Solution
I’ve used UFT for just over 12 years. In comparison, I’ve used CodedUI for about six months, and Selenium for only two months. As you may imagine, I have a significant grasp on UFT and what it can do. An argument could be made that with the same experience in the other two competitors, these tools may be on more equal footing.
As with any code base, well-designed and implemented automation code will make for easy maintenance, and a stable code base that will scale very well as the complexity of the suite grows. Nothing will save you from poor coding practices.
Customer Service and Technical Support
I’ve had little direct support from HP. I’ve instead used HP partners to get licensing and support. Specifically Orasi, they have been very helpful in the few support areas I’ve needed. I would rate them 9/10. As an advanced user, there are issues I’ve come across that Orasi wasn’t able to find a solution for. However, they did their due diligence and escalated to HP as appropriate.
It is very straightforward in its install and setup. An extra layer of complexity exists when trying to integrate it with Quality Center, but this is done via some settings and an add-in. The nice thing about UFT is that it’s a mature tool that has massive user-forum support on the internet. Although there is a lot of support for C# (Coded UI) and Java (Selenium), there isn’t as much in regards to the testing side of those languages.
Pricing, Setup Cost and Licensing
Cost is the biggest issue with UFT. It is not cheap. However, when evaluating the full cost (not just the licensing), I believe UFT is actually a cheaper solution in the end. That being said, seat licenses that are tied to a specific machine (can be moved via support) run approximately $10,000 with tax and associated annual maintenance agreement. A concurrent license runs approximately 17K with tax and maintenance agreement.
From an ROI standpoint, you need to look at the automation effort in comparison to the manual work it would reduce, as well as increased code coverage and a consistent level of testing. In most cases, it will take 2-3 years before the automation suite is significant enough to start seeing its cost even out. Any organization contemplating automation should have that type of commitment to see the automation effort become successful.
Other Solutions Considered
I’ve evaluated a few different automation products. Only Selenium and CodedUI come close to the functionality and adaptability that UFT provides. I support UFT as the best solution due to the skill set needed to operate the tool. VBScript is verbose, but otherwise very easy for a non-coder to pick up. Selenium and CodedUI both require OOP languages that are more complex for the tester to pick up.
Overall my main concern, is with resources. There aren’t as many Selenium/CodedUI professionals in the marketplace therefore when it’s time to deal with turnover, it’s very difficult to find experienced automation personnel. That is not the case with UFT. An added plus for UFT is how it handles poorly designed and implemented web applications. I didn’t fully see this until I began to use CodedUI and Selenium.
If this is the first time implementing a solution, I would say make sure to read up on what it will take to implement. Get as much knowledge ahead of time to make it smoother. To hit the ground running, it is best to organize your manual tests so that automation can begin as soon as possible. What test cases are for Smoke tests? What test cases are for Regression? Starting automation without defining the work to be completed will waste precious time -- time you are paying for idle licensing.