What is most valuable?
The biggest advantage of Business Process Testing is that not only technical skilled people can participate with test automation but also business analysts, which will create better test cases then test automation experts can. The maintainability is also much better than in traditional approaches, but that also might depend on the architecture which is used.
How has it helped my organization?
Because everyone can create test cases, not only the quantity but also the quality is improved, which results in less defects in production. Also, there is less dependency on technical consultants so they can focus on other tasks.
What needs improvement?
The run speed is not as fast as if a local script would be running, but that is one of the downfalls of using a client/server approach.
For how long have I used the solution?
I started using it for about four years ago. Back then we used ALM 11 and BPT with QTP technology.
What was my experience with deployment of the solution?
We have a DTAP environment, deploying in production takes a bit more effort than in a test one because of the security dependencies we have.
What do I think about the stability of the solution?
There are no stability issues.
What do I think about the scalability of the solution?
Scalable is key for BPT as for one project we are using it with 50-100 users. When scaling up, it’s important to have a small team which is responsible for the framework while the rest of the team is just using it.
How are customer service and technical support?
To be honest, technical support is not that great. We use CGI for technical consultancy who are a bit faster and better than the support of HP. The R&D part of HP is helping us with issues that they cannot solve.
Which solution did I use previously and why did I switch?
Before using BPT, I mostly used TD/QC in combination with QTP. We used it with a custom framework. The main issue with most test automation frameworks (including the last one) is that they are highly technical and are hard to maintain. This resulted in a high dependency on third parties instead of being able to let the scrum team being able to be self sufficient.
How was the initial setup?
Test automation specialists helped us with setting up the framework. It should be done with experts because the architecture has to be setup in a way that matches with the client.
The BPT framework is very flexible, which can result in suboptimal results when used by inexperienced users.
What about the implementation team?
CGI helped with implementing and training the teams. We have three defined levels of test automation; level one is creating test cases, level two is modifying components, and level three is changing the framework components.
What was our ROI?
ROI is in most projects within a year if you compare it with manual testing. For smaller companies the HP tooling is indeed an expensive one.
ROI is not easy to calculate because there are some benefits which are not comparable with currency. For example, how do you rate quality.
What other advice do I have?
It has room for improvement, but in my opinion it’s at the moment “best in class” when used in a (bus) DevOps environment. The most important part is that the components and flows should match with your product under test. Involve the business at the start of the implementation phase.