Conformiq Review

Feature-rich stable tool with multiple options to control output, good integration with other tools, and knowledgeable support team

What is our primary use case?

We had our test cases in BPT format. However, the creation of duplicate components resulted in significant manual and automation maintenance efforts. The implementation of the concept of model-based testing was thought to reduce that effort.

Within the MBT Tools space, Conformiq has a lot of options for data optimization and path optimization logic. Its keyword-based modeling approach had fit properly with our automation framework. 

How has it helped my organization?

Conformiq generates a unique and optimized set of test cases. The test cases are optimized for both data and path. 

It offers easy maintainability of models - a functional change may only require updating the model at appropriate places (and tool will take care of TC generation) when compared to updating the entire test case suite. It maintains test cases to the current application version and removes test cases for obsolete functionality. 

Since the models map to current application version, it acts as a knowledge repository for any future training.

The UI definition of models is based on keywords. With proper integration to the automation framework, in-release/in-sprint automation has been achieved. 

What is most valuable?

The core functionality of the tool is automated test generation of optimized test suite; the tool has extensive list of options for the same. The product's integration with other tools sets it apart. It has integrations with many upstream (for requirements mapping) and downstream (export of test cases to various tools) products. It is like "plug-and-play". 

For any customized downstream tool, like our proprietary automation framework, support is provided for custom development. It is has features - Business AD - to use in Agile implementations. The latest version seems to have support for BDD/Gherkin as well, which we have not used much. 

What needs improvement?

Though I understand every tool will have limitations, an effort can be put in to better understand and acknowledge the issues (not every issue is a problem with Conformiq) encountered in real-time implementation, even if there is no solution.

Even though the 4.1 version is a far-improved version from its earlier avatars, the performance of test generation is still an issue on real-time models we have.

The reverse engineering logic, to generate models with existing manual/BPT test cases, did not work for us.

For how long have I used the solution?

We have been using Conformiq Creator for the past four years and six months.

What do I think about the stability of the solution?

The output of Conformiq is predictable - it generates the same output for same input and settings again and again. Its integrations with other tools like Rally, ALM (both Import and Export), MS Excel, GitHub work well. 

This helped us expand the solution to multiple applications under test, rather than worrying about whether the output is correct.

What do I think about the scalability of the solution?

Performance of test generation process takes a hit, but we have really huge/too many models. We have used other modeling techniques to work around the issue. Also, understanding the way the tool works (ie., internal processes/threading, RAM and processor usage) had helped us create manual processes to work around the issues.

A proof of concept has been completed to schedule/automate the trigger/export test generation; with proper toolset availability, automation framework tweak and further support from Conformiq team, Conformiq can be integrated into DevOps tool chain.

How are customer service and technical support?

The team is knowledgeable. They were able to answer all our technical queries in relation to the product itself. The team was able to answer questions related to real-time implementation and the process surrounding it most of the times; in other instances, it could have been better approached, if not answered to our satisfaction.

Which solution did I use previously and why did I switch?

For optimization, we were using another tool, but it was giving only test data combinations and not test cases. There was not much integration with other tools, including any of the automation frameworks.

How was the initial setup?

The reverse engineering option to convert existing BPT/manual test cases into Conformiq models did not result in a usable and scalable solution. Therefore, we had to engineer the models brand new and it took a lot of effort.

Also, our models grew too huge in a short span of time and ran into issues with TC generation. However, the Conformiq team were able to re-engineer the backend logic for faster processing and made a huge improvement.

What about the implementation team?

It was implemented through an in-house process and technical approach definition .

What's my experience with pricing, setup cost, and licensing?

Test Cases can be generated with a local system as the server. However, depending on the number of models and their size, it is better to set up a system with higher RAM and a faster processor and use it as a server. 

Which deployment model are you using for this solution?


Which version of this solution are you currently using?

Creator 2.x to 4.1
**Disclosure: I am a real user, and this review is based on my own experience and opinions.
Find out what your peers are saying about Conformiq, Broadcom, Cognizant and others in Test Management Tools. Updated: September 2021.
536,548 professionals have used our research since 2012.
Add a Comment
ITCS user