What is our primary use case?
We're a banking software company, so we use it for Synthetic logins to test how one of our end users would log into our product for a customer, how long it would take, what loads, and then log them out. Then we test how long it takes to do that entire process.
On the Synthetic side, we only use it SaaS-based. We actually put it through an SSO. We use Okta for an SSO. That's how we're securing our connections there. Security-wise, Apica's got a couple of things in the works that are going to help them out, but they're not there now. In particular, they're coming out with a key chain that allows us to save. You can hash passwords and users, they don't have that right now. Passwords and logins are set in plain text.
How has it helped my organization?
Before we had this particular product we were using SolarWinds to do something like this. The problem with that, however, was that was an internal check, which means it was coming from our network. With Apica, we could do the same type of check, and then we could also make a scenario script that would go and click the things we wanted to click, but it would come from an external source that we did not control. That would give us a better baseline for what the customer sees, as opposed to what we expect to see from our network.
What is most valuable?
We like the scripting features and the scenarios. It allows us to set up exactly how a customer would log in, what they would type in, where they would click on the screen, and then takes screenshots of it so that we can actually see it happen and see what they see at that time.
We also use it for up-down checks for a lot of our websites that we make ourselves for our customers to make sure the sites are up or down. It's not part of the Synthetic side of it, but we also use the ZebraTester. We're actually implementing various homemade tools on our site as well by API.
We use ZebraTester for some of the sites and other things before they even become into the Synthetic side.
It is highly flexible when it comes to websites. There are a few things that it does fall down on, but for the most part for logging into a website to check to make sure elements are loaded on the screen, it's highly flexible. If I don't want a certain element to load, I can block it or I can ask it to ignore it. If I need to check for a certain element, I can do that as well.
As always, within the IT industry, everybody's always looking to upgrade and update everything else like that. Apica has been one of those things but it's really hard to replace because it offers us the unique capability to see what the customer is seeing. A lot of other ones can do Selenium script and things like that, but there's a lot in Apica that we use right now. We utilize a lot of the scenario options in Apica right now, and there's a lot of other ones that do parts of it, but it doesn't do everything that Apica does.
Apica is indispensable in a few things that we do. It currently is the only one that we have that catches CDN outages. We have many tools that monitor our customer sites, but a lot of those are API logins. If we had a CDN outage and the site didn't load all of its elements, we wouldn't be able to tell that. Apica can tell that because it's looking for particular elements on the screen. Indispensable may be a strong word, but we do highly rely on it for some things.
We use Selenium scripts and we were able to do more specific checks, so it makes it feel like we're actually a customer logging into one of our sites, checking their accounts, and logging out.
The scripting feature has kind of saved money and resources. When we first got it set up, it was a pain because we didn't have the script set up before, but now we have it setup and it's running on multiple checks. Multiple checks, meaning, our Synthetic login checks range around five to 550 checks. Now when we have scripts set up to make the Selenium check, I can pump out new Selenium scripts for one of our online banking customers in five minutes.
Alerts are always accurate, but they might not always be useful. Apica alerts on two different things: one, when an element that is in the script cannot load, and two, when a part of what's loaded comes up with a certain internet error code, a 500 or 402 or something like that. It's always accurate because those things are always not doing that, or they're getting the errors, but it may not actually be as useful. To deal with that, we generally either have to block the URL that's throwing the error code or whatnot, or we have to verify the elements.
It's very accurate but sometimes not useful. It's also noisy. When Apica alerts, it does not have a pull-in time or anything else like that, unless for elements or error codes. It does for SLA times and variances, but not for the other ones. It could be that it's a one-time blip and something didn't load on the screen, it alerts immediately right then. If it loads the next time, it's not going to alert. If it's still set up, it alerts. It can be noisy.
This level of alerting accuracy has saved us time and money in operational costs. With CDN issues, it lets us know, for instance, that we have a homemade monitoring system for our products as well that monitors to make sure that things that should be there are there, but it doesn't actually take into account if the webpage itself is loading. A number of times we've had major CDN outages where our homemade monitoring tool is fine because everything is loaded by an API, but the webpages are not. When that happens, Apica tends to go alert hard and that lets us know that "Hey, we need to go check over here as opposed to over here." That saves us time and money on troubleshooting.
We have two different approximations in terms of how much it's saving us. The way that we do our major incidents, is that we do it per customer. If we have five customers down for five minutes, that's 25 minutes of downtime. I don't have an exact number. I know that things like that when it affects our entire environment are pretty substantial.
It has also saved costs involved in managing monitoring. It has at least saved us in the cost of that it gives us one pane of glass to go to for Synthetic monitoring. I can actually send one of our analysts to go look and if they want to know if a page loaded, they don't actually have to go log in, they just have to log into Apica and check to make sure Apica's running well. That saves time, which saves them money.
What needs improvement?
Alerting needs improvement. It's a little noisy. It needs some better options. Currently, they have an issue, when you set up a Synthetic monitor, you can set up where it's monitoring from a data center that Apica owns. However, for each data center that you attach to a monitor, that's considered an extra license. That's a bit iffy. They're usually behind on the version of Chrome that they're using for the Synthetic monitors. Currently, they're using Chrome 85, they're 11 iterations out of date. They're trying to get that fixed up with something called Evergreen, which will basically be a Chrome browser that'll stay constantly up to date, but it hasn't been implemented yet.
The problem with that is that we generally test our product with the newest versions of Chrome and everything like that, so sometimes we've run into issues. Also, when they updated to Chrome 74, we lost some monitoring capabilities that we had before that did not transfer over with this new version of Chrome.
I'd like easier access to the API. Their API, it's not bad, it's just bulky. It's a little unwieldy in the way it has to be used. One of our app developers is currently working with them and he wanted to do a number of calls to the API, and he was not able to do that. They had to make special changes to our API to make the number of calls he wanted to make. It didn't seem to be scaling as well as we thought it would. But they worked with us to actually get it to do that. That's a plus point.
I'd like to see more abilities to do mass changes to checks in the GUI, in the interface. Things like setting a mass amount of blocks for checking a bunch of checks and saying, "Make sure that this URL is blocked on all these checks." Currently, we can only do that through the API, and last time we had to do that, we actually had to use Apica support to do it.
Finally, they have an audit log system called Journal. However, it can only check, if I remember right, two weeks at a time. That becomes really difficult when you need to check on something that you need to go back multiple times and you don't know the exact dates of the thing that changed. For example, I had a user who got changed in one of my checks and I needed to find out when it got changed. It ended up being three months ago, but I had to go back in two-week increments until I could find it. Their Journal, their auditing system, needs a little bit of work.
For how long have I used the solution?
We've had Apica for five years.
We are using the SaaS portal.
What do I think about the stability of the solution?
The stability is not bad. We've never actually had an issue with Apica, the product. The alerting really comes back down to how this thing alerts, how the alerts are sometimes not useful. That's the worst I could say about the stability. We can turn them off or we could filter them through a third-party.
What do I think about the scalability of the solution?
From what it does, it scales pretty well. I can easily go in, pull on a check, assign a scenario to it and boom, I'm done. I've done that many times. A couple of weeks ago I spent the day creating very specific Selenium checks for very specific parts of a customer's website to make sure that they load properly. It handles it quite well.
We have roughly 100 users using this solution.
We take Apica data and we send it to our data warehouse so that we can do SLAs for our customers. There's me who sets up all the monitoring in there. And then we have our NOC, who will go in there and they use it to actually make sure sites are up and everything. It's used throughout the company in all ways: business side, maintenance, and monitoring.
I do the maintenance of Apica.
I have 683 checks. For the Synthetic login, the checks, it's 400 checks. Those are the ones that we mimic the login like a customer would log in. For the VT checks, which are basically just up and down checks, we have 112 of those. That's not just our customer sites, we also use this product for our site as well for corporate sites.
We do have plans to increase usage. When a new customer gets added in, they get a check as well. Every customer gets a check.
How are customer service and technical support?
Support has never been a problem. Their support is top-notch. We either email or get our client experience manager on the line. They have been top-notch, willing to help, willing to go the distance. I have very minimal complaints about support. The one complaint I did have, they actually addressed it very quickly.
Which solution did I use previously and why did I switch?
We started with SolarWinds, and after that, we moved to Apica. We then got rid of our SolarWinds integration and went to LogicMonitor. LogicMonitor has its own website monitoring tools. However, the problem with LogicMonitor's website monitoring tools is that it's very hard to set up a script the way that Apica does. They also don't provide screenshots of what happens. We've looked at a number of other vendors as well. The problem always comes down to it doesn't do the things that Apica does.
How was the initial setup?
The initial setup was very complex. I wasn't even part of the initial setup, but I know it was very complex because we needed an external source for our checks, but we needed to be able to mimic logging in like the customer did. This was back in December of 2014. I have a feeling nowadays though, they probably have this down to a fine science of how to get people implemented and their stuff up and running.
What was our ROI?
We have seen ROI. We use it for all of our customers and it does help us. A lot of times it can catch things that happened to the site, but don't happen to the API. We've seen a good return on that.
What's my experience with pricing, setup cost, and licensing?
Pricing is based upon not so much users, but the number of checks you're going to create. Make sure when you set up an account with this, to request more licenses for checks, for any type of check, than you actually need. This will save in the long run. They're really good about setting this up and getting you more licenses but there's always a cost with that.
If you think you're going to need 100 checks, make sure you get 110 licenses. Then remember if you want to do multiple-site checks, not just one-site checks, you're going to get a license for each site.
With all companies, you get the base product, but the base product's not all that you want. You want it with a whole bunch of other stuff with it. We can safely assume that there are probably other costs to add things. Things like additional integrations with other products are not included in the standard license.
What other advice do I have?
The biggest lesson I have learned from this solution is the sheer number of sites that can load when you load one website. We do online banking, but when you load online banking, it also loads 50 other URLs as it loads through there. That might include Google, Facebook plugins, or things like that. It has really opened my eyes to how many things load when you just open up a single webpage, even if there's that much on the webpage itself. It's very comprehensive when it comes to website monitoring.
I would rate Apica Synthetic a seven out of ten. We've had our problems with it and we're still waiting on some add-ons and features, but for the most part, it's never wrong. It's just sometimes noisy and feels old. The UI is very basic. It's not bad, it's not ugly, but it's basic. It uses old browsers.