Application Performance Management (APM) Forum
Jun 24 2020
How is synthetic monitoring used in Application Performance Management (APM)?
Michael SydorSynthetics for production, are best used when there is little or no traffic to help confirm that your external access points are functioning. They also can be used to stress test components or systems - simulating traffic to test firewall capacity or message queue behavior and many other cases. You can also use synthetics to do availability testing during your operational day - again usually directed at your external points. Technology for cloud monitoring is generally synthetics. And the ever-popular speedtest.net is effectively doing synthetics to assess internet speed. The challenge with synthetics is maintaining those transactions. They need to be updated every time you make changes in you code base (that affects the transactions) and to cover all of the scenarios you care about. And also the HW requirements to support the generation and analysis of what can quickly become thousands of different transactions. Often this results in synthetics being used every 30 minutes (or longer) - which, of course, defeats the usefulness as an availability monitor. Real User monitoring is just that - real transactions, not simulated. You use the transaction volume to infer availability of the various endpoints, and baselines for transaction type and volume to assess the availability. This eliminates the extra step of keeping the synthetics up-to date and trying to live with the intervals at which you have visibility into actual traffic conditions. But it will take extra work to decide which transactions are significant and to establish the baseline behaviors, especially when you have seasonality or Time-of-Day considerations that vary greatly. However, I'm seeing that the best measure of transaction performance is to add user sentiment to your APM. Don't guess at what the transaction volume means - simply ask the user if things are going well, or not! This helps you narrow down what activities are significant, and thus what KPIs need to be in your baseline. A good APM Practice will use both synthetics and real-user monitoring - where appropriate! You do not choose one over the other. You have to be mindful of where each tool has its strengths, what visibility they offer and the process that they need for effective use.
Sunder RajagopalanSynthetic monitoring helps simulate traffic from various geographic locations 24/7 at some regular frequency, say 5 minutes to make sure your services are available and performing as expected. In addition, running Synthetic monitoring along with alerts on some of your critical services that are dependent on other external connections like Payment Gateways, etc. will help you catch any issues with external connections proactively and address them before your users experience any issue with your services.
NetworkOb0a3I think different shops may use the term differently. In regards to an industry standard the other replies may be more appropriate. I can tell you that where I work we refer to SEUM (Synthetic End User Monitoring) UX and Synthetic (both user experience monitors) monitoring as simulating actual human activities and setting various types of validations. These validations may be load times for images, text, pages, or validating an expected action based on the steps completed by the monitor. We target all aspects of infrastructure / platform for standard monitoring and then for any user facing service we try to place at least one Synthetic / UX monitor on top of the process. I often find the most value from our Synthetics comes in the form of historical trending. Great examples of NOC wins have been patch X was applied and we noticed a consistent 3 second additional time required to complete UX monitor step Y. Another value from Synthetics is quickly assessing actual user impact. More mature orgs may have this all mapped out but I have found that many NOCs will see alarms on several services but not be able to determine what this means to an actual user community until feedback comes in via tickets or user reported issues. Seeing the standard alarms tells me what is broken, then seeing which steps are failing in the synthetics tells me what this means to our users. I think that one of the great benefits to an open forum like this is getting to consider how each org does things. There are no wrong answers, just some info applies better for what you may be asking.
May 31 2020
Below are the rankings. What do you think? Gartner reports these four solutions as Leaders: Cisco (AppDynamics) Dynatrace New Relic Broadcom These are the Visionaries: Splunk (SignalFx) Datadog Only one Challenger: Microsoft Eight Niche Players: Riverbed (Aternity) IBM Instana Oracle SolarWinds Tingyun ManageEngine Micro Focus Thoughts?
Tjeerd SaijoenIf you ask what I think of the 2020 Gartner Magic Quadrant, after more than 30 years of experience in mainframes, distributed environments, I would say look to a piece of software and not how to resolve performance problems in a multi-cloud environment. To give you a few examples, if you have a performance problem and you have for example an APM tool, but the problem is because of a wrong setting in a database parameter, you don't find it with any of these tools. We had situations where they were running problems for over a year and all the vendors were not capable of finding a solution. You need a total overview of an environment including virtualization, infra, databases, web servers, load balancers, etc. otherwise an APM tool will help if it is an APM problem, but if it is a configuration problem you will not find it. Same with infrastructure, if I have for example old bios and the latest drivers I will get a lot of system interruptions. Most of the APM tools will not give you a solution, these are only a few examples, so you need to ask first what do you like to do with an APM tool or do I need more and is there a solution for resolving problems quickly and react proactively. And yes I agree with Gartner but only if I like to monitor and need a sexy interface.
Radoslaw RuncI cannot speak about all range of ranked products because I don't know them well enough. I can speak about current Broadcom offering. It seems that this platform is strenthening since next to new recently issued new release of APM Broadcom introduced new release of Apps Synthetic Monitor. Also part of APM license is currently module App Experience Analytics for user experience monitoring. More than that APM,ASM and App Exp Analytics are part broader of AIOps family next to Uunified Infrastructure Management , NetOps - Spectrum and ML solution called Data Operation Inteligence (DOI).All these solutions are beeing glued with automation mechanisms coming from Broadcom One Automation platform.
Rob SalmonI think that a major player in this space -eG Innovations - is missing. I have personally used the product to manage our server farms/cloud platforms since 2004 and could not be more pleased. Functionality, cost, ease of use, ROI are all magic quadrant worthy and they are a leader in root cause analysis. The #1 call is why is it slow? eG gives you the answer usually in four clicks. Huge value and huge ROI in my humble opinion.