2014-03-23 16:24:00 UTC

When evaluating Application Performance Management, what aspect do you think is the most important to look for?


Let the community know what you think. Share your opinions now!
Guest
101101 Answers
Vendor

Hi,
Full disclosure I am the COO at Correlsense.
2 years ago I wrote a post just about that - "Why APM project fails" - I think it can guide you through the process of the most important aspects of APM tools.

Take a look - feel free to leave a comment:
http://www.correlsense.com/enterprise-apm-projects-fail/
Elad Katav

2016-04-17 11:44:45 UTC17 April 16
Consultant

In order to evaluate/benchmark APM solutions, We can based on the 5 dimension provided by Gartner:
1. End-user experience monitoring: the capture of data about how end-to-end application
availability, latency, execution correctness and quality appeared to the end user
2. Runtime application architecture discovery, modeling and display: the discovery of the
various software and hardware components involved in application execution, and the array of
possible paths across which those components could communicate that, together, enable that
involvement
3. User-defined transaction profiling: the tracing of events as they occur among the components
or objects as they move across the paths discovered in the second dimension, generated in
response to a user's attempt to cause the application to execute what the user regards as a
logical unit of work
4. Component deep-dive monitoring in an application context: the fine-grained monitoring of
resources consumed by and events occurring within the components discovered in the second
dimension
5. Analytics: the marshalling of a variety of techniques (including behavior learning engines,
complex-event processing (CEP) platforms, log analysis and multidimensional database
analysis) to discover meaningful and actionable patterns in the typically large datasets
generated by the first four dimensions of APM

In other side, we tried to benchmark internally some APM solutions based on the following evaluation groups:
Monitoring capabilities
Technologies and framework support
Central PMDB (Performance Management DataBase)
Integration
Service modeling and monitoring
Performance analysis and diagnostics
Alerts/event Management
Dashboard and Visualization
Setup and configuration
User experience
and we got interresting results

2015-04-27 09:04:55 UTC27 April 15
Vendor

I Check about a Customer Experience Tool, I think the organization have to improve your client experience and if you don't have a tool to view the Improve Points this is so hard to do!!

2016-10-18 02:26:59 UTC18 October 16
Real User

Speed to get data into the platform is one of our most important metrics. We NEED to know what is going on right now, not 3-4minutes ago.

2015-12-03 02:14:39 UTC03 December 15
Vendor

Visibility into the application transaction and all the magic behind the curtain

2015-09-09 20:49:30 UTC09 September 15
UserTOP 10

Monitoring end user experience (including impact of external partners). It's the price of admission

2015-09-03 13:04:02 UTC03 September 15
ConsultantPOPULAR

Being able to see every transaction and tracking of the performance of what the end user sees.

2015-08-13 21:31:19 UTC13 August 15
Real UserTOP 10

Know and meet de NFR's. Find correlations and have superb SLA management and reporting.

2015-08-11 11:09:49 UTC11 August 15
User

Hi As per exp.. customer always look how much faster application its open, how many users are accessed in a faster way...how are then got an errors....application page load....end user request load on application ....how the server resource utilized the application....is there any more resource are need to run the application

2015-08-06 07:04:21 UTC06 August 15
Vendor

In my opinion, an APM solution should demonstrate the following characteristics:
Ease of use. The solution should be easy to use from installation, configuration and maintenance/support.
Fulfill the business need. There are so many solutions/vendors out in the marketplace that claim to have the RIGHT solution for you. After carefully evaluating some of the solutions, you find that they do not satisfy your business need and promises to have the additional functionality in their future roadmap. Why wait?
Integration. An APM solution should have the ability integrate with various other tools/systems without disrupting your current environment.
Cost effective. In most cases, "you get what you pay for". Free isn't always the best solution for your business need, as well as paying a substantial price tag for a solution and still does not give you the value you need.

2015-07-23 18:20:30 UTC23 July 15
Vendor

Easy deployment and integration. E2E monitoring on a component level. Automatic CI discovery and mapping. Support of ITIL processes - incident, configuration, service level management...

2015-07-16 07:48:43 UTC16 July 15
Real User

Application Performance Management is a shared concern. APM tools should be used, easily, by all people involved in the build and run process, to share a common view of which part of the application/system is used and how it impacts the whole performance of the service according to the user/customer point of view.

2015-07-15 12:57:41 UTC15 July 15
User

My scope for a monitoring system is the capability to monitor:
- Servers
- Switches
- Firewalls
- Workstations
- FC switches
- Storage (SAN, TapeLibrary,...)

These are the main components where the monitoring system has to get as much data as possible out of the systems.
Here is the HA of the monitoring system very important.

We have tested CACTI and Nagios and SCOM but we are the most happy with OpManager of ManageEngine. because the correlation with the other components is so easy and logic we prefer and recommend the solutions of ManageEngine.

2015-07-01 12:09:31 UTC01 July 15
Real User

In my view following aspects should be consider:

- Server Monitoring capabilities
- Ease for user i.e. easily configurable and end-user experience
- Alerts and events management
- Dashboards available so you can visualize the stats
- Analyse performance and diagnose problem
- Must be under budget

We mostly recommend Nagios and Cacti as tools to our client.

2015-06-30 10:10:21 UTC30 June 15
Real UserELITE SQUADTOP 5

These posts hit my main points on the nose.
Bob_J " 1. End-to-end transaction visibility 2. Scalability 3. Low/no over-head 4. Ease of use "

reviewer142791 - "Being able to see EVERY transaction. No sampling or just high water marks. If you don't have information on all of your data, you can't effectively benchmark and trend performance over time."

I would also look for transaction topology mapping. I picture paints a 1000 words in expressing urgency.

2015-06-25 14:12:29 UTC25 June 15
ConsultantTOP 20POPULAR

When you want to evaluate an APM Solution in my option the most important aspect is
1) What are my organization needs and if that solution partialy complies to the needs.
If we know for what we need the APM Solution we go for the other things:
2) Monitor End-to-end transactions
3) Integration with monitoring tools
Usualy after implementing the APM Solution the client wants an unified solution that agregates the data from the APM and from the monitoring tool.
4) Synthetic transactions
5) Cost-benefit
Every organisation has it's own needs. Sometimes the client doesn't choose the solution they really need because of the price, and in time they discover that their choise is not fullfilling it's purpose and they need to buy other solution.
6) Event correlation
It is ideal to correlate the events from a server with the events from the APM. The correlation helps the administrators to solve the issue ASAP when you have all the data.
7) Easy to use
If the solution is easy to use, it gives the administrator the time to develope new rules and it give him the ability to learn the solution faster.
8) Ability to analyse application code for flaws
If you see errors in the APM from your applications, usually the application owner is asking what it will be the cost for troubleshooting and for error remediation. If the APM is able to provide the flaws, than the cost of repair is much lower than expected.

2015-06-01 16:27:54 UTC01 June 15
Vendor

1. The APM solution getting implemented should not have an additional overhead on the system performance in terms of server resources. ( Very Important aspect)
2. The APM solution should be compatible with the technology /software you are trying to use.

2015-05-08 00:18:28 UTC08 May 15
Real User

1. End-to-end transaction visibility
2. Scalability
3. Low/no over-head
4. Ease of use

2015-03-28 00:52:18 UTC28 March 15
Real User

Well, the most important: the solution covers your application stack and your ecosystem, and this happens at the implementation and in the roadmap of your company. Then you should decide having in count: easy of use, data integrated from multiple sources, transactional view and in-depth root cause analysis available. All platforms of this type performs well and are based in very similar concepts. After a pilot or PoC you will have more light in this field.

2015-03-19 11:12:57 UTC19 March 15
Vendor

Ease of implementation, scalability, breadth of monitoring from L3-L7 of OSI model and low overhead for agents.

2015-03-06 15:58:17 UTC06 March 15
Real User

Easy to implement, use; Automatic rootcause, proactive and helps in optimal use of applications

2015-03-05 03:40:07 UTC05 March 15
Real User

Number of systems monitored and the amount of traffic they transverse through you LAN and WAN

2015-03-03 13:00:35 UTC03 March 15
Real User

Dashboard information to show summary, but easy drill-down access to more detailed information; easy configuration and user adaptation.

2015-02-24 17:37:35 UTC24 February 15
Real UserTOP 20

end user experience, network visibility, ease of use

2015-02-08 14:35:37 UTC08 February 15
Vendor

1. Low Overhead
2. Ability to use it in production, test and development
3. No Sampling or just turning it on when there is something suspicious going on -this is to late to find the root cause
4. Ability to follow exactly end to end transactions from the click/touch to the database
5. Real User Monitoring for Web, Mobile Apps and Rich Clients
6. Ability to give APM data to different audience in a convenient way. eg. CEO can view high level data on a tablet, Developer can deep dive on stored data a long a bug report, operators can have their personalized view of the system on the ops screens

2015-01-06 11:59:57 UTC06 January 15
Real UserTOP 5

Most vendors have similar transaction monitoring capabilities so I look at the End user experience monitoring features to differentiate. Not only RUM (mobile and web) but also Active Monitoring through synthetics.

2015-01-05 20:15:38 UTC05 January 15
Consultant

It depends on multiple factors, some of them are Real time monitoring, scalability, easy of use and licensing cost.

2014-11-08 12:59:15 UTC08 November 14
Real User

1. Presentation of key performance data
2. Compactness and minimal dependencies
3. Customization
4. Platform independent

2014-11-04 10:35:04 UTC04 November 14
User

One suite which would cover
1. End-to-end business/audit transaction flow irrespective of technology stack/Integrated components,
2. NFRs/Infrastructure/capacity metrics and their management as well
3. lightweight,
4. Easy configurable/manageable.
5. Plugin availability for integration for customization.
6. Of Course budgetary/cost saving in long run.

By default APM tool/suite should provide evidence/root cause of performance bottleneck and notify reaching threshold of configured KPIs.

2014-11-03 16:01:40 UTC03 November 14
Real UserTOP 20

1. How the tool affects application load is most important. An APM tool is useless if it slows down the production environment.
2. Ease of installation.
3. Ability to drill down into the application stack for root cause analysis.
4. Alerts on performance and errors.
5. Ability to customize the data stream.

2014-10-24 03:27:14 UTC24 October 14
Real User

1> Monitoring application performance with the help of APM introscope.
2> Being able to see EVERY transaction. No sampling or just high water marks. If you don't have information on all of your data, you can't effectively benchmark and trend performance over time

2014-10-07 07:30:48 UTC07 October 14
Real User

End user metrics including Network latency, load time and application Launches.

2014-09-02 14:02:43 UTC02 September 14
Vendor

Being able to see EVERY transaction. No sampling or just high water marks. If you don't have information on all of your data, you can't effectively benchmark and trend performance over time.

2014-07-25 13:17:00 UTC25 July 14
Vendor

Nothing can be more important than end user visibility.
If it wasn't for the end user, what metrics would have any value since we talk about Application Performance Management?
The judges of the performance is the end users - period. It can never be the tools!
In a APM solution, there needs to be a direct correlation of what a user does to what resources that particular action demands from the infrastructure. Of course the time spent in those resources is also valuable.

2014-07-03 13:25:46 UTC03 July 14
Consultant

Check resources availability.

2014-06-10 08:23:02 UTC10 June 14
User

To know NFR's

2014-05-05 08:59:54 UTC05 May 14
Vendor

For one, the APM tool needs to be stable and be able to notify me when it is unable to complete the configured monitoring. After that, it is important to be able to monitor log message occurrences for the software that my company produces.

2018-01-18 17:01:08 UTC18 January 18
ConsultantTOP 5LEADERBOARD

You should list till what level the monitoring is required.

Starting from
1. Application layer
2. Database layer
3. Code Level
4. Network & Infrastructure Level

This will help you select the right bundle of the tools to achieve your requirements.

2017-12-22 06:36:32 UTC22 December 17
Real UserTOP 5LEADERBOARD

As I believe you cannot manage what you cannot measure, I give the most importance to data quality. No sampling, no gaps, just the real thing!

2017-12-14 15:53:35 UTC14 December 17
Consultant

I am not an actual user of these products. I am a consultant that offers comparative research support for clients with vested, potential investment or consumer interest in them or at times, for the developers themselves.

2016-09-21 23:23:55 UTC21 September 16
Vendor

Measure every users transactions and trace them through to back end for immediate root cause analysis.

2016-01-25 16:48:27 UTC25 January 16
ConsultantTOP 10

Knowing business needs and capabilities, and map them to the best tool.
Getting the full picture, quickly find problem and getting pointed the right way

2015-12-18 13:27:13 UTC18 December 15
Real User

Manageability, stability, and ease of use

2015-12-17 08:22:49 UTC17 December 15
ConsultantTOP 20

Query / Transaction response for expected load conditions.

2015-12-16 05:05:31 UTC16 December 15
User

Two factors affect application performance management. First user interaction (workstation, data flow, access, response rates) and second backend systems (replication, db updates).

2015-12-11 18:17:31 UTC11 December 15
Real UserTOP 10

When it comes to performance, I monitor how long it took to refresh the screen for the application.
Scalability, low impact of the performance etc

2015-12-09 14:38:14 UTC09 December 15
Real User

End User Experience along with its operation time,slow operations and the application performance.
For the web based applications,I think we should look the TCP and HTTP errors which are basic parameters.
For SSL Based applications,it is TCP and SSL errors.

2015-11-27 09:48:33 UTC27 November 15
Real User

Clear full stack tracing... starting at the end user all the way to a minor external service call somewhere in a microservice sometime in the entire transaction. ..

2015-11-22 18:36:16 UTC22 November 15
Real UserTOP 10

Price
Ease of Installation
Customization
Reporting

2015-11-18 13:25:01 UTC18 November 15
Real UserTOP 5

Ease of use, Predefined customizable reporting capability, high visibility, high capability of raw data and reporting import export

2015-11-12 15:42:38 UTC12 November 15
Real User

How quickly you can ID bottlenecks while providing the performance that the customer demands.

2015-11-08 16:05:40 UTC08 November 15
Real UserTOP 10

Knowing what you wil do with results. Match them with technical and functional requirements and think about what information you would need if you need to troubleshoot the applictaion chain. Then look for a tool that supports this.

2015-11-06 13:32:23 UTC06 November 15
Real User

cost per port, scalabity for Tier1 operations, ease of deployment and use

2015-10-19 11:06:23 UTC19 October 15
Real User

low impact in the overall performance and ease to identify bottleneck

2015-10-16 12:30:21 UTC16 October 15
User

1. Identify problems and pinpoint root-cause.
2. Understand your application or website’s readiness before go-live.
3. Shorten application release cycles.
4. Understand and optimize your end-users’ journey.
5. Understand outside forces affecting your application and/or website.
6. Simplify deployment and management of your APM platform.
as http://www.researchinaction.de/

2015-10-14 06:44:26 UTC14 October 15
Vendor

scalability and flexibility come first, in order for an Organization to -proactively- runs its business and -effectively- adapt to this fast growing business environment. Then comes all the other obvious APM capabilities and benefits customers expect, some listed above.

technologies (big data, apps, mobile, cloud, etc) have been quickly evolving... an organization can only run its IT Operations and be more proactive if it relies on APM that are built on 3year old (or less) technology stack, so the APM is flexible, scalable and can adapt to any business and technological changes, in an ever increasing big data, mobile, cloud and highly integrated environment.

most other APMs that are built on 5-15year old technology stack may have more "analytics" pre-configured, but it takes weeks, months to add more capabilities to them and make them scalable and flexible to have these legacy systems keep up the pace with this ever/quickly changing world.

take a look at:
Germain APM (Application Performance Management)
Germain CRT (Code Review Tool)
on www.germainsoftware.com

2015-09-24 18:12:31 UTC24 September 15
User

Enable Complex Distributed Application Architecture Discovery.
Enable Business Transaction and Internal Call Discovery, bottleneck and latency discovery.
Enable Business and IT Customisable Dashboard build.
Enable Alerting based on Pattern / Baselining.

A Way To Be Pro-Active whether used in QA Environnement, or Efficient on Production issues analysis.

A Must Have In Today, RealTime, Distributed, Multi-tiers Applications

2015-09-11 10:33:20 UTC11 September 15
Vendor

Visibility into the end user experience.

2015-09-04 15:06:56 UTC04 September 15
Vendor

24/7 transparency along the whole process chain with user experience as the key performance indicator

2015-08-25 15:53:55 UTC25 August 15
Real User

Check resources availability.

2015-08-12 05:58:11 UTC12 August 15
Vendor

TIme to value and ease of use

2015-08-09 08:46:51 UTC09 August 15
User

Ease of use, price, and scalability

2015-08-06 13:53:49 UTC06 August 15
User

Easy of diagnoses and analysis. Getting to the actual root cause quickly.

2015-07-21 15:55:25 UTC21 July 15
User

Price, scalability and easy of use

2015-06-22 12:40:44 UTC22 June 15
User

Ease of Integration ... There are almost 7 different tolls currently in play and each group is in his own world...We want to add capability not add double the cost

2015-06-12 18:15:48 UTC12 June 15
User

User response with the ability to highlight any component that is negatively impacting this response from the user through the network back to the server, database server, and storage.

2015-06-09 20:41:08 UTC09 June 15
Real User

Transaction detail and End-User experience metrics.

2015-05-27 15:26:59 UTC27 May 15
User

Ability to discover flow transaction, capture all business transaction. Easy to use and to deploy. Also have customize dashboard, as my user comes from different role (management, it operation, application owner, business development, etc)

2015-05-27 05:56:25 UTC27 May 15
Vendor

Ease of use

2015-05-17 12:07:36 UTC17 May 15
Real UserTOP 5

1) Capabilities
2) Synthetic transactions
3) Open Source approach
4) Integration
5) Cost-benefit
6) End-user Support
7) Dashboard
8) Event correlation
9) Easy, user-friendly configuration and setup
10) No invasive monitoring

2015-05-14 12:11:05 UTC14 May 15
Real User

Monitoring application performance with the help of APM
Visibility to see All transaction. Not a sampling or just high water marks.
The ability to see what the End user is experiencing is most important.
Problem solving not finger pointing

2015-04-26 15:04:31 UTC26 April 15
User

Self learning from the topology.

2015-04-22 10:52:52 UTC22 April 15
Real User

End user visibility

2015-04-17 10:08:29 UTC17 April 15
Real UserTOP 5LEADERBOARD

Thanx Ariel for starting interesting APM ongoing forum, lot of valuable info/inputs from our peer's couldn't stop liking those valuable comments. I would personally suggest ease of APM solution which supports multiple diversified Apps ,DB's,O/S all in one shot for complete system including Upstream/Down stream systems and produces transaction level reports which can be easily drilled down and can be understood by Non technical management teams also.

2015-04-15 09:09:50 UTC15 April 15
Consultant

- each transaction must be monitored and analyzed
- the solution has to correlate traffic generated by a user with its true EURT measure
- any kind of metrics has to be suppored by its pcap, so the system has to be able to extract automatically the pcap related to the analyzed detail
- great reporting and dashboarding

2015-04-15 08:59:04 UTC15 April 15
Real User

All functionality being equal, I want ease of deployment and low management overhead

2015-04-13 18:21:15 UTC13 April 15
User

User Visualization and interpretation.

2015-03-24 11:36:27 UTC24 March 15
User

Level of monitoring for Deep dive Analysis
Transaction level monitoring

2015-03-23 09:11:20 UTC23 March 15
Vendor

Depth of analysis and ease of use

2015-03-11 07:11:13 UTC11 March 15
Real User

Most efficency in performance and root causes for issues

2015-03-06 17:47:29 UTC06 March 15
Real User

Cost, flexibility and restitution quality.

2015-02-26 14:27:13 UTC26 February 15
Vendor

Low TCO Real and Synthetic User Experience Monitoring

2015-02-20 02:01:55 UTC20 February 15
User

Application component deep dive

2015-02-04 16:58:38 UTC04 February 15
Real UserTOP 5LEADERBOARD

The ability to monitor everything across the stack.

2015-01-20 22:16:23 UTC20 January 15
User

Approval to run in the government space.

2015-01-13 14:34:55 UTC13 January 15
UserTOP 20

Action base monitoring

2015-01-06 21:27:41 UTC06 January 15
Real User

In depth coverage of the Stack

2015-01-02 05:41:53 UTC02 January 15
Real User

Ease of use is probably the biggest item to consider for APM. If no one will use it then it's just wasting money.

2014-12-26 19:10:22 UTC26 December 14
Real User

Ease of use and limited/no performance impact form the toolset itself.

2014-11-24 09:15:46 UTC24 November 14
Real User

VM’s, scalability, easy of use, accessibility

2014-10-08 08:54:28 UTC08 October 14
Real User

1. Doing what it has promised, helping root cause analysis
2. End-2-end transaction monitoring (end user+data center) without sampling
3. Discovery of application topology
4. Not affecting apps that are monitoring

2014-10-03 12:06:15 UTC03 October 14
Vendor

Easy to deploy, accurate results on transactions.

2014-09-18 13:37:39 UTC18 September 14
User

Finding the bottlenecks and Drill down the bottlenecks to increase the performance of the application.

2014-08-05 10:43:19 UTC05 August 14
Consultant

The excellent end user experience regarding on transaction and also conversion rate.

2014-08-04 17:25:56 UTC04 August 14
Real User

Ease of use, self discovery capability and ability to interoperate with AANPM

2014-07-01 23:05:43 UTC01 July 14
Vendor

Automation and Visibility

2014-06-25 12:54:22 UTC25 June 14
User

Accessibility - Manageability - Stability - Availability

2014-06-25 10:10:38 UTC25 June 14
Consultant

Manageability of the tool

2014-06-20 11:14:31 UTC20 June 14
Vendor

Software lifecycle

2014-06-13 07:37:11 UTC13 June 14
User

Price, scalability and easy of use

2014-06-11 14:51:29 UTC11 June 14
Real User

Dealing with VM's

2014-05-27 20:27:41 UTC27 May 14
Find out what your peers are saying about Dynatrace, AppDynamics, New Relic and others in Application Performance Management (APM). Updated: September 2019.
366,090 professionals have used our research since 2012.
Sign Up with Email