The tool is flexible to handle multiple complex scenarios, which is one of the good things that led us to decide on Apica.

Doing the URL monitoring without having to write a script is pretty neat and straightforward. I can spin up a check within a few minutes using the URL monitoring functionality. That is one of the easiest features that gets my team onboarded to use the solution within a couple of days. It is the easiest feature to use, very helpful for doing a quick setup, and delivering to other partner teams in a very short time.

The alerting feature from Apica for email alerts and integrations with other alerting platforms has been pretty helpful.

The flexibility of Apica is big in terms of the range of protocols it can monitor and scale for multiple scripts . That has been one of the bigger influences when making the switch. Also, it is not just about the flexibility of the tool, but the flexibility of the team who we work with. The product teams with whom we had interactions have been very flexible. We were able to request Apica to implement new features into their tool set, and they were more than willing to accept and implement them in their production tools. That is flexibility in a vendor relationship I have never seen when I have worked with other vendors. The product team was flexible enough to listen, accept, and implement the features that we wanted.Another key deciding factor for us when going with Apica.

All their pops: They have locations all around the world. Having all the locations around the world is very useful, especially when you're in a license market.

Their alerting system: The way their systems alert you is top-notch. 

Its flexibility is pretty high. You have all your points of pops where you can go to Ireland, Sweden, or X location. I don't think they really need to improve on their flexibility. There are so many settings, different optimizations, and scripting options that you can do. 

The way you script Apica is probably the easiest way of working that you've ever seen. For a QA person, it is very easy because they have the understanding of the tools and what they have to offer. From the complexity side, it is very possible to do pretty much everything on Apica: down to logging in and up deposit, doing other processes inside your website, and loading slot machines to make sure external providers are loading correctly. Because in the world of gaming, you don't buy all your own slot machines. You have the likes of Pragmatic, Betsoft, NetEnt, where you have to make sure all their services are up as well.

From our standpoint, there are a number of valuable features. The WebHooks are obviously really great. The alert framework is really good and then the reporting and visualizations that you get from the dashboards are good. Those three areas are primarily what my team's focused on in terms of usage from day to day.

The dashboard view tells you the health of the services that we have monitored, and how the health of the entire infrastructure is doing at a glance. My teams have given me a lot of good feedback that I just keep the dashboard up on my workstation during the day, and if we get an alert, I can immediately go and investigate if I'm in the dashboard. I can also sometimes catch an event as it's occurring so save myself a little bit of time and be able to get in and see what's going on more quickly as a result of that. From the alerting standpoint, that in conjunction with the dashboards that you get, really compliment each other because then you can drill down and actually get into what's happening from a transactional standpoint or transactional perspective, and see where within the transactions that we're monitoring, where what steps are failing, get more details on why those steps are failing and work to mitigate and resolve those issues based on that.

It's that visualization component that really ties everything together and the drill-down capabilities that you get starting from the dashboard that really makes Apica very useful from a day to day support standpoint.

There are a lot of capabilities that we're not really taking advantage of that we could. There are a lot of opportunities to grow in terms of how we're using the framework, especially when it comes to doing things that are more complex, like facilitating deeper checks via multi-protocol based scenarios that tie in with ZebraTester automations that get created or more advanced regression-based scenarios that we might want to set up in the synthetic checks. We're using around a third of all the capabilities that we have available to us so we definitely have a lot of room in terms of what Apica offers for growth and for expansion of our use cases.

The alerting is impressive. 98% of every alert I see come out of Apica is a valid alert. The other 2% of the time, we will get an alert or something will not be right which is because we overloaded our Apica infrastructure with something that we were doing. It was a self-inflicted thing. If you actually remove that from the equation, what you're really talking about is that it's nearly a 100% success ratio of events to real events. 

It's been fantastically accurate at identifying events. The sometimes frustrating part of that is convincing other people that what we're actually seeing, coming out of Apica is a real issue and it needs to be addressed because a lot of times people will just not be convinced by the data that they're seeing until well after the fact. As we've been using the platform more and more, there are more teams out there that are understanding that when a team member brings something up from Apica, it's not to be taken lightly.

From my perspective, I would say it has saved us costs involved in monitoring. It's enabled my resources to work more focused. It's enabled them to work more accurately. It's enabled them to work more authoritatively and it enabled them to work more adeptly. From an operational standpoint, I would say that it's at least improved our monitoring efficiency by 5% to 10%.

It has centralized the monitoring into a single pane. With the wide variety of testing tools out there in the market, we have been able to streamline them more into a single, proactive dashboard that manages the scheduling, configuration, and alerting with a very robust API. Those big benefits really brought together our synthetic monitoring objectives.

They have given us a lot of flexibility in order to get past our security. Overall, they provide a wide range and customization, which helps us get through any issues, but there isn't one solution for every organization or a good wide-breadth of options.

Their platform does allow us to import JMeter on our files. That provides us some flexibilities, allowing us to use different tools. Their API is also very robust so we can do integrations quite easily by using their core product, synthetic monitoring, as a base. We use that as a trigger for other platforms that will compliment resolution. If the Apica platform continues to monitor successfully, then we can take that result and implement the next action with it.

It is very accurate. It certainly provides a lot of detail. The quality of the alert is based on the quality of your script and what you are monitoring. This does depend on the ability of the developer along with their strategy and design of the script to capture what they need to be able to monitor performance or availability. Overall, I think it provides a lot of details, if you script it to capture what you need it to.

We like the scripting features and the scenarios. It allows us to set up exactly how a customer would log in, what they would type in, where they would click on the screen, and then takes screenshots of it so that we can actually see it happen and see what they see at that time.

We also use it for up-down checks for a lot of our websites that we make ourselves for our customers to make sure the sites are up or down. It's not part of the Synthetic side of it, but we also use the ZebraTester. We're actually implementing various homemade tools on our site as well by API.

We use ZebraTester for some of the sites and other things before they even become into the Synthetic side.

It is highly flexible when it comes to websites. There are a few things that it does fall down on, but for the most part for logging into a website to check to make sure elements are loaded on the screen, it's highly flexible. If I don't want a certain element to load, I can block it or I can ask it to ignore it. If I need to check for a certain element, I can do that as well.

As always, within the IT industry, everybody's always looking to upgrade and update everything else like that. Apica has been one of those things but it's really hard to replace because it offers us the unique capability to see what the customer is seeing. A lot of other ones can do Selenium script and things like that, but there's a lot in Apica that we use right now. We utilize a lot of the scenario options in Apica right now, and there's a lot of other ones that do parts of it, but it doesn't do everything that Apica does.

Apica is indispensable in a few things that we do. It currently is the only one that we have that catches CDN outages. We have many tools that monitor our customer sites, but a lot of those are API logins. If we had a CDN outage and the site didn't load all of its elements, we wouldn't be able to tell that. Apica can tell that because it's looking for particular elements on the screen. Indispensable may be a strong word, but we do highly rely on it for some things.

We use Selenium scripts and we were able to do more specific checks, so it makes it feel like we're actually a customer logging into one of our sites, checking their accounts, and logging out.

The scripting feature has kind of saved money and resources. When we first got it set up, it was a pain because we didn't have the script set up before, but now we have it setup and it's running on multiple checks. Multiple checks, meaning, our Synthetic login checks range around five to 550 checks. Now when we have scripts set up to make the Selenium check, I can pump out new Selenium scripts for one of our online banking customers in five minutes.

Alerts are always accurate, but they might not always be useful. Apica alerts on two different things: one, when an element that is in the script cannot load, and two, when a part of what's loaded comes up with a certain internet error code, a 500 or 402 or something like that. It's always accurate because those things are always not doing that, or they're getting the errors, but it may not actually be as useful. To deal with that, we generally either have to block the URL that's throwing the error code or whatnot, or we have to verify the elements.

It's very accurate but sometimes not useful. It's also noisy. When Apica alerts, it does not have a pull-in time or anything else like that, unless for elements or error codes. It does for SLA times and variances, but not for the other ones. It could be that it's a one-time blip and something didn't load on the screen, it alerts immediately right then. If it loads the next time, it's not going to alert. If it's still set up, it alerts. It can be noisy.

This level of alerting accuracy has saved us time and money in operational costs. With CDN issues, it lets us know, for instance, that we have a homemade monitoring system for our products as well that monitors to make sure that things that should be there are there, but it doesn't actually take into account if the webpage itself is loading. A number of times we've had major CDN outages where our homemade monitoring tool is fine because everything is loaded by an API, but the webpages are not. When that happens, Apica tends to go alert hard and that lets us know that "Hey, we need to go check over here as opposed to over here." That saves us time and money on troubleshooting.

We have two different approximations in terms of how much it's saving us. The way that we do our major incidents, is that we do it per customer. If we have five customers down for five minutes, that's 25 minutes of downtime. I don't have an exact number. I know that things like that when it affects our entire environment are pretty substantial.

It has also saved costs involved in managing monitoring. It has at least saved us in the cost of that it gives us one pane of glass to go to for Synthetic monitoring. I can actually send one of our analysts to go look and if they want to know if a page loaded, they don't actually have to go log in, they just have to log into Apica and check to make sure Apica's running well. That saves time, which saves them money.

There are several features that are really good. The first one is the flexibility and the advanced configuration that Apica offers when it comes to configuring synthetic checks. It provides the ability to customize how the check should be performed and it is very flexible in the number of synthetic locations that it can use. It allows us to run scripts from different locations all over the world, and they have a really good number of these locations.

There is also the ease of use. The user interface it provides is really advanced, but at the same time, it is really easy to use. That's a really good feature when it comes to daily use and our daily processes on the platform.

It is also very good in terms of the range of protocols it can monitor. Even if, at the moment, we are only using the HTTP protocol and browser synthetic checks—it's mainly the emulation of the end user browser—they also provide other protocols, such as DNS verification.

The features that we use probably 99 percent of the time are the HTTP and HTTPS checks. We set up a lot of them. This is 99 percent of our current usage of Apica Synthetic as well as some full browser checks, but this is a lower amount. We also use some scripts that utilize the platform, but our usage of them is very small. 

An very important point for using Apica is the ability to have Chinese probes, which is not common among these types of tools. Other important things were the Grafana native integration and PagerDuty integration, which are all tools that we use extensively.

Our application SREs do script checks in such a way that closely mimic our customers' actions using the platform. Because there are so many different ways and options to be able to configure checks to closely mirror your applications' capabilities, it provides a lot of optionality for teams to create the right type of check that can notify when there are any issues. At the end of the day, we want our monitoring tools to be able to catch any outage before our customers do. This is where Apica Synthetic does a great job.

There is definitely a lot of flexibility. I haven't run into any issues or heard of any issues from our SRE teams that said they weren't able to get Apica Synthetic to monitor or script in such a way where it monitored their applications effectively from a synthetic perspective. 

We mainly use the ZebraTester and the browser checks. These are the most important scripts that we're using on Apica. 

With the ZebraTester, the ability to have and store dynamic variables, when setting up the monitors, means you can extract that value and use it in a subsequent service call. This is something that has made our lives easier. The most complex monitoring processes are for security purposes: You need to have a fresh token for the user when, for example, he tries to log in. That token keeps changing. To be able to get the results of other service calls that are depending on the login, you need to use that token in the subsequent service calls. Being able to extract that token, store it in a variable, and use it in the other service calls is one of the most complex things. This is one of the features that I like the most because it helps us in configuring these services, in a certain flow, without the need to re-record the whole thing. Being able to extract that value from the service calls is something that has made monitoring a lot easier.

For the browser checks, the screenshots that are available help the engineer or the operator who is on the shift figure out what's wrong or what step is failing.

Also, the flexibility of the solution in terms of the range of protocols it can monitor has been great. The product has been working as expected and it has helped us to cover something like 95 percent of the outages or issues that we have had.