The thing that is most valuable to us is the desktop console application CA Introscope Workstation. It is very easy to use. We tried other tools before (IBM Tivoli, for example), but the console was very difficult to use. CA Introscope Workstation is the first tool that was quickly embraced by our developers and became one of the tools they use on a daily basis. This is very important because, in reality, only developers can really interpret the metrics collected by the tool.
Improvements to My Organization:
CA APM is mostly used during production post-rollout week. This is when our developers can quickly pinpoint any performance issues we might be having in production. Since the tool gives us near-real-time information, we lag only 15 seconds, we can quickly see bad trends and respond to them. Before APM, we were more reactive then proactive.
Room for Improvement:
We are constantly adding new technologies to our stack and monitoring those is sometimes a pain because CA APM does not support some of these out of the box. For example, we have recently added things like MongoDB, Elasticsearch, memcached, etc. All of these tools come with excellent monitoring APIs accessed using REST, but CA APM doesn't come with support for collecting metrics via a REST endpoint. This really should be addressed – too many services come with a simple REST API utilizing JSON which should be very simple to collect.
Use of Solution:
We have been using CA APM for over five years to provide real-time monitoring of our IBM WebSphere-based production systems. We're currently in the process of upgrading to v10.
CA APM performs really well and I cannot remember the last time it crashed. Granted, we are not an enormous deployment!
No issues encountered with scalability.
Over all these years we really only contacted customer support once. The issue was resolved quickly and professionally.
This tool quickly becomes very important to your organization. It is ok to use a local partner or vendor to help you out, but use that for some knowledge transfer. In the end you will need an in-house expert or two. As your applications change, so will the tool's configuration and it is best if you can do that yourself.
Cost and Licensing Advice:
The good thing about licensing is that you only pay for the application serving machines you monitor. That means that all other backend servers come for free – stuff like your RDBMS, NoSQL databases, search servers (lucene, solr, elasticsearch), cache servers (memcached), etc.
Other Solutions Considered:
We tested IBM Tivoli based solutions – it was a logical choice since we use IBM WebSphere. However, the tools were difficult to use and the whole user experience was just awful. We then piloted CA APM (CA Wily back then) and everybody fell in love with the console and the information it provided.
Plan for the involvement of your developers. This tool monitors your production applications, but only the developers know exactly what to monitor. Sure, the tool can give you metrics for method call, but somebody needs to give you a list of methods to monitor – you cannot monitor every single method! You will need input from the people who created the applications you want to monitor.