What is our primary use case?
We use UFT Pro for all user testing platforms. We use the standard installation but we use UFT in two models. One is used for testing all functionalities in our environment and the second is one we are developing to use as a solution to test the availability of the environment in production. So UFT will check out the performance of the production environment every 12 minutes to be sure that the entire environment is stable. If we don't have any problem, the information is stored in a database and we do a BVD (Bank Vault Drawer) analysis of the information in the database for checking all banking applications.
The application we are developing is J2EE (Jave Enterprise Edition) and we will have the information about the functionality. If we have a problem we call a team to send issues to them so they will work on this application to correct the problem in production. Using the same of monitoring, we will be able to monitor all availability and all access to an application really using only UFT.
We use UFT to model for testing functionality in environmental tests, and after that, we use UFT to check and monitor all access and all applications to be sure these things are correctly functioning or not.
How has it helped my organization?
It gives us the opportunity to serve banking clients while conforming to industry regulations.
What is most valuable?
The ability to evaluate live applications in our production environment for unusual behavior and determine problem areas and solutions is the most valuable aspect of this solution.
What needs improvement?
As far as things that can be improved, it is a good solution so I think I can only do a comparison. We also use QC/ALM (Application Lifecycle Management [Quality Center]). It's a global solution that is managed with information from UFC from all over the environment. It has to be integrated with UFT. Really UFT could have this functionality built-in.
We have 40% advantages and 60% disadvantages in our setup of UFT. This is because with UFT, we also have the problem that we have to use Windows Server and I would like to use Linux. For Selenium, we can use Linux so we have good performance. But we can't use UFT with Linux.
It is impossible because in UFT we have to develop for UFT with VBScript and VBScript is only for windows and not for Linux. Another problem currently with the UFT — I think it is resolved in the new generation of UFT — is that we can't run tasks in parallel. In the new version, we can improve our workflow if we can choose to allow multiple tasks at runtime. So there is a problem with that currently.
In Selenium, our development is done with Java technology — J2EE. So if we have an online community and we have a Selenium grid, we can run multiple tasks in realtime. We can't do that in UFT now because of its requirements, so it's a problem for us. When they come out with a solution for this issue, the product can be more flexible like Selenium and it will be a great benefit to us.
To make UFT better, Micro Focus has to make UFT work in a stable environment. Right now, UFT is a problem all the time. It would help to have a community and a special forum for UFT, and even that is missing. We have good forums in Java and for Selenium, so it is possible to get solutions easily for those products. I think it would not be hard to do for UFT, and it would be better for UFT users if we had a good website. Users could help themselves and share knowledge and address problems and make up for the lack of support. We also don't have training for UFT. It is like they just made a product and don't care to support it. It is a good product, but not so perfect that it doesn't need support. I have to go to France to get certified. We don't have that ability here in Morocco. We cannot send everyone there, so it is a problem.
For how long have I used the solution?
We have been using this solutions for five years.
What do I think about the stability of the solution?
The UFT product is not stable. We often have issues in production that we have to deal with. We have specific people who are our support connections at UFT. If we need to report an issue we send an email to support and wait. After that, we correct the problem with their instructions or we install patches that they send. Patches are not a real solution. Because of this, the support does not seem very good to me in making the product stable. Patches do not make an application stable and may not be widely tested. They may cause other issues.
For comparison, for insurance clients, we use Selenium which is always stable or we fix it quickly ourselves. We can't do the same with UFT.
Actually, we communicate with our DevOps (Development and Operations). So, I integrated UFT with Jenkins for testing and better communications. But the integration is not stable. Because it is not stable and does not function well, we have an extra nightly job to use Jenkins for checking in if all the environment is okay with tests created in UFT. Some days this tests okay. Some days it does not test okay. If it is not okay we need to reboot the system and generate a new job. It has a problem 50% of the time. So, it's a problem. It's not stable.
What do I think about the scalability of the solution?
You can scale the product in some ways by integration. You must purchase expensive licenses which they have two kinds: seat licensing and concurrent licenses that can be shared. Each license is expensive, so scaling is expansive.
How are customer service and technical support?
The support for UFT is not good. When we send email to forums or support, we may get a response, and maybe we won't. When we do, the solution is not always good.
Which solution did I use previously and why did I switch?
I switched from development to quality testing. Since that time, I used many products including UFT. We had some solutions in basic. After that, I used several products for testing including UFT, Selenium, Cucumber, QCALM (also known as just QC), Dynatrace and more to check on the environment that things like memory and CPU were functioning as expected. Some of these things I would still use depending on the situation. It is not necessarily the product that made us need to switch to UFT. It is the business need and regulations.
How was the initial setup?
Our installation was straightforward. We install the instances ourselves.
What about the implementation team?
We use our own team who works to do the installation and maintenance.
What was our ROI?
It works as a solution to serve a certain clientele that we cannot serve with other products.
What's my experience with pricing, setup cost, and licensing?
I think the cost of UFT is too high and obviously expensive, especially if you consider that there are other even better products, in my opinion, that are open-source. Because of the expense, we use UFT only with big companies. For a small company with a smaller budget, we can't choose UFT because UFT is very expensive.
Which other solutions did I evaluate?
Because we need to properly support people in banking and follow regulations, we use UFT. It is really more of a political decision than a proper choice as to the products we would prefer to use.
We had Selenium first because it runs on Java — which is a stable language — we could use it and adapt it for all we needed to do. As a developer in Java with 10 years of experience, I could resolve our problem myself and not need any help. If I have a problem, I check the internet, I go to stackoverflow.com or visit one of the many forums for Java. So I guess my problem is that with Selenium I can check the problem, fix it myself, and I can do it right away without having to wait for a response from support.
A second benefit to Selenium is that it is open-source. It's not a costly choice. We have the opportunity to install in whatever platform we want, and that is good for us — It could be Unix, Windows — It doesn't matter. It is good as a more flexible solution.
Third, we use the platform for continuous integration. We have Dockers which we use for all containers and helps us prepare all our environments in simple ways. It's very easy to use, very easy to deploy, it's very easy to install and very easy to understand. The framework we use with Selenium is something we can use for all the functional testing for insurance products. Selenium would be what I would use for banking if it were possible.
What other advice do I have?
I prefer other products like Selenium to UFT, but each product has its advantages. For example, in UFT we can test HTML protocol for the web applications and also desktop applications. Selenium is for web applications only. That is its limitation. If you have to test both and want to install only one product, UFT has an advantage.
Because of all the problems and limitations of the UTF product, I would rate it at only a four out of ten (where ten is the best and one is the worst). By comparison, I would give Selenium an eight out of ten. You can see I think UFT is not my favorite product and it is not good for everyone.
Disclosure: I am a real user, and this review is based on my own experience and opinions.