Oracle Application Testing Suite Review
Load testing is a valuable feature however the tool runs out of memory when the recording flows are long


Valuable Features:

  • Recording and playback
  • Load testing 

Improvements to My Organization:

Our benchmark load tests for a major release involving various scenarios with a combination of different virtual users relied on the recording, playback and eventual load testing capabilities of OATS.

Room for Improvement:

  1. The tool runs out of memory when the recording flows are long and involve many screens despite increasing the memory settings for OpenScript 
  2. We found the tool impossible to use for client side performance testing without filtering out JS, CSS, PNG etc as the method would error due to a Java method size limitation. 
  3. Parameterization of variables using regular expressions was not a foolproof way of running tests as it would mostly work for a single user but fail during load testing. 
  4. Parameterization involving SQL queries was not straightforward 
  5. Except a YouTube video/PDF which covers some basics and is probably outdated now, there are no tutorials available for other functionalities that the recording tool provides.There is no in-depth tutorial of the load testing interface and troubleshooting steps. 
  6. Collection of system metrics - although there are multiple parameters defined, it did provide a holistic picture of the state of the system.We had to implement our own way of obtaining this. 
  7. Load testing web interface did not provide details about errors. The only way to correlate was to inspect server logs.

Use of Solution:

One year.

Deployment Issues:

Yes, with 12.2.x

Stability Issues:

Yes, 12.2 was not stable. Upgrading to 12.4.x resolved a lot of previously known issues

Scalability Issues:

No issues encountered.

Previous Solutions:

This tool did a much better job recording the http requests during web load testing than other open source tools. Cookie management and other url parameters were handled efficiently and were easier to identify in the tree view.

Initial Setup:

It was straightforward for the most part except that at times the services wouldn't run and un-installation messed things up.

Implementation Team:

In-house.

ROI:

I believe we made a wise choice despite some shortcomings and hiccups.

Other Solutions Considered:

We evaluated other options.

Other Advice:

Read the tutorial and try out a few scenarios before going full throttle.

Disclosure: I am a real user, and this review is based on my own experience and opinions.

Add a Comment

Guest
Why do you like it?

Sign Up with Email