What is our primary use case?
Synthetic: To replace and improve on the current solution and integrate it with the multiple platforms that are already available in the organization.
LoadTest: To replace the current solution with a newer and better platform which would not only expand on the current capabilities but also reduces the infrastructure maintenance.
We have a mix of hybrid models. We have the orchestration platform as SaaS in Apica's cloud. However, the agents that we run checks (or load test from) are located on-prem.
How has it helped my organization?
One of the biggest advantages of moving to Apica is the ability get to a hybrid model with the orchestration engine in the cloud and our agents on-prem. We also have access to Apica's cloud agent across the globe. That has changed the way that we have our load testing and synthetic monitoring setup at this point. Previously, it was always internal. Now, with this change , we can test and run checks from anywhere across the globe. If a certain location is not available , it just takes an email to their customer support, then it is spun up within 24 hours. That flexibility has changed the way that we perceive our Synthetic monitoring and load testing, not just in the US, but globally.
Availability of extensive API options is an enabler integrating both synthetic and load testing platforms with a wide variety of tools and processes.
What is most valuable?
The tool is flexible to handle multiple complex scenarios, which is one of the good things that led us to decide on Apica.
Doing the URL monitoring without having to write a script is pretty neat and straightforward. I can spin up a check within a few minutes using the URL monitoring functionality. That is one of the easiest features that gets my team onboarded to use the solution within a couple of days. It is the easiest feature to use, very helpful for doing a quick setup, and delivering to other partner teams in a very short time.
The alerting feature from Apica for email alerts and integrations with other alerting platforms has been pretty helpful.
The flexibility of Apica is big in terms of the range of protocols it can monitor and scale for multiple scripts . That has been one of the bigger influences when making the switch. Also, it is not just about the flexibility of the tool, but the flexibility of the team who we work with. The product teams with whom we had interactions have been very flexible. We were able to request Apica to implement new features into their tool set, and they were more than willing to accept and implement them in their production tools. That is flexibility in a vendor relationship I have never seen when I have worked with other vendors. The product team was flexible enough to listen, accept, and implement the features that we wanted.Another key deciding factor for us when going with Apica.
What needs improvement?
The accuracy of alerts can be improved a little bit. Right now, it's good in terms of alerting immediately about failures or changes in response times. However, what we have seen happen is the number of alerts that we are getting is very frequent, and we would like to tone down the number of alerts. An option to aggregate alerts for a check from multiple locations is not available and creates duplicates. As a platform, it does send us good alerts, but it could be improved a bit.
For the Synthetic platform, their dashboard could be a little better. I don't have many options to configure my dashboard to make it suitable for a power user versus a leadership person. The dashboard for our CTO is the same as the one that our technical staff is using. We have made this request to Apica, and they're going to make some changes to it. However, at this point, the dashboards and the way they present the checks could be better. For the level of metrics that it provides for each check, they could provide it at a step level or page level on the landing page rather than having us click a few more times to get that data. That would save a lot of time. These two would be really good changes to help increase efficiency from the current tool and current features that it provides for synthetic monitoring.
As far as I am aware, the Apica platform doesn't allow us to execute scripts from JMeter or LoadRunner out of the box, but they do offer conversion utilities.
For the LoadTest platform, they could increase the efficiency in terms of the results produced. The reporting structure of the results could be improved a bit. Apart from that, it is a pretty good platform.
For how long have I used the solution?
We have been using Apica Synthetic and LoadTest for about a year now.
What do I think about the stability of the solution?
It has been very stable and availability has been at 99 percent, if not more, so far. In the last year, there was only one issue with availability, which was resolved within a matter of minutes. We were informed right away from support team that there is an issue with availability. Therefore, it has been pretty good and reliable.
What do I think about the scalability of the solution?
The scalability of the orchestration platform is pretty good. Because it's hosted on the cloud, we haven't seen any slowness in the scalability of the platform. It could be a little better for the LoadTest platform though.
- For the Synthetic platform, the scalability is pretty good as we increase the number of checks. The application has been able to scale without issues.
- The LoadTest platform could be a little better. We have seen improvement over the last couple of months and the vendor is ready to provide more scaled solutions for the LoadTest platform going forward.
There are a total of 300 to 350 users on the Synthetic platform. For the LoadTest platform, there are about 100 users in total across the globe.User roles vary in ranges from technicians (the people who write scripts and run load tests), all the way up to managers and directors. This is across the board for both platforms, Synthetic and LoadTest.
We have about 4,000 checks on monitors running at any one point in time.
How are customer service and technical support?
The thing that makes the product so easy to use is the support. They made our experience a lot better than what we had have to deal with working with other vendors. Their support is unbelievable.
Which solution did I use previously and why did I switch?
We wanted to replace our current existing solution for synthetic monitoring with a newer, advanced solution that integrates with multiple platforms.
We are replacing Micro Focus Business Service Management (BSM) for synthetic monitoring and Micro Focus Performance Center for load testing.
How was the initial setup?
Our initially setup from Apica's side was pretty straightforward. Since this was a hybrid model, we just had to procure some servers internally to setup as on-prem agents, and that was it. Compared to other tools that we work with, this was at least 60 percent easier to set up.
Apica anticipated our needs during our implementation process. They had a pretty good survey of our requirements and use cases before the implementation. They came up with quite a few recommendations that helped us to design the solution better for our current setup.
It took about three to four months for the basic setup to be completed because of the hybrid model.
Our idea was to set up on-prem agents as quickly as we could to facilitate the communication from Apica SaaS to the on-prem agent. We also wanted to have both the incumbent team and the new team comfortable with the new setup and decommission the old tool. That was our implementation and migration strategy.
What about the implementation team?
I would give them a five out of five on the level of support. From this particular team that I worked with, they were amazing. The willingness to change their tool, or even remove features, based on our feedback is unheard of. I have never seen a vendor do that. I have the highest regards for the Apica support team.
Apica was able to come in, bringing in technical resources, and help change/convert the old scripts. This helped us to speed up the process of the migration. Without their help, I think the migration would have taken a lot more time.
What was our ROI?
Based on our prediction model, we should be at least saving 10 percent of our investment with Apica versus using our incumbent tool.
What's my experience with pricing, setup cost, and licensing?
Apica is pretty cost-effective if you buy both solutions together: Synthetic and LoadTest. If you are going for one solution, the cost is on par with any tool in the market.
In terms of licensing, the major advantage with Apica is you don't have to pay for their cloud hybrid, which is included in the license. For example, our company is heavily dependent on using cloud agents for monitoring after load testing.
Which other solutions did I evaluate?
We also looked at Catchpoint. We looked at them in a very elaborate way, doing a pilot and PoC, but couldn't align them properly with our needs.
With Catchpoint, the con was the flexibility in terms of working through enhancements and changes. It was not flexible enough to accommodate the changes. The scripting engine for Apica is very flexible, which was another major con from the other tools that we evaluated. In Apica, I can write my own custom code, which I can't do in the other tools. This was a big pro for Apica when we were making our decision.
The major factor that influenced our decision to go with Apica is that it is the same platform for both load testing and synthetic monitoring in terms of scripts. I could write a script for my load testing and plug the same ones in for my synthetic monitoring. So, it helps us in terms of our DevOps model, where I don't have to rely on two different tools to write different scripts, which is the current model right now. Instead, it increases the synergy between multiple teams to use the same platform and reduces the overhead of script creation from testing to synthetic monitoring, which is a big plus. I'm hoping in terms of efficiencies over time to reduce the number of resources that we rely on for writing script.
What other advice do I have?
For Synthetic, we still need to see the alerting in real-time to see how it improves.
There is around a 15 percent adoption rate in the company for tool use. However, for our target audience, the total is around 80 percent. Our idea is to get to 100 percent adoption for our intended group of users. That is the goal for the middle of next year.
Apica is a pretty flexible tool. The amount of features that they have to offer is very high compared to other tools on the market because of the flexibility that they have to offer. The product team behind it is very committed to making those changes and the customer happy. I would advise if you are looking at a solution which is easy to maintain and can handle a lot of checks at the same time, then Apica would be the best solution to go with. If you have to do a complete on-prem solution, that is available with Apica as well. It is a pretty good solution in the current market.
At this point, I would rate it a nine out of 10 because there are still some enhancements that need to happen for the platforms: Synthetic and LoadTest.
Which deployment model are you using for this solution?