What is our primary use case?
Our use case is using it to reduce the time to assemble everyone, especially when reacting to major incidents, and to reduce the time spent doing manual call-outs and engagements in the course of them.
The background, and why we looked into getting a tool in the first place, is that when we were engaged on an incident and it rose in severity or the scope of impact broadened, there were many different checkboxes that we could check: "Okay, now we need to re-engage this person, this person, and this person." Also, just in the average incident-information process, we needed additional speed in getting people engaged, rather than manually looking up the on-call information to see who's engaged or on-call. We would have to manually call them and possibly get voicemail and have to try the second number.
We use it for every bridge call that we host and every engagement of an on-call group that we need. We use it multiple times a day, every day.
How has it helped my organization?
Our performance showed us that, for major incidents, we spent over 40 minutes just making manual call-outs. That is why we implement the tool in the first place and that time has been cut down to two or three minutes. We've had tremendous gains on that.
It's a lot easier to create and manage schedules, especially in comparison to the on-call scheduling creation in ServiceNow. That has always been something of a bear to operate. We've found it's a lot simpler in Everbridge.
It enables everybody on our team to focus on their primary responsibilities, driving toward restoration, instead of being distracted by manually calling folks.
What is most valuable?
Creating the templates and being able to create my own variables are helpful features.
Their latest features are going to allow me to be a bit more flexible with using Everbridge for internal communications. We started using it for internal incident notifications a few months ago, and having that group lookup, allowing me to create a relation between a property and a variable in the template, and who should be contacted as far as a group in the organization goes, is going to allow for some nice self-service for our internal folks when we transition to a different Everbridge organization for our internal coms.
What needs improvement?
With their templates, you can only have a maximum of three phases: new, updated, and resolved. It's not always that easy when we open up a call, that we identify who we need, page out, and we're good. A lot of time it requires multiple page-outs. Being restricted to those three phases, there's no way to say, "I want this variable to be persistent, and this one to not be." Everything that you select will be brought over as you continue. In our environment, as we have many different call-outs that have to happen, even though they are incredibly simple to select and execute now in Everbridge, it is quite the long list. I would like to make it a bit easier and more intuitive. I would like to see a bit more flexibility and tighter control over the templates and the variables you can create.
Also, they still have a limitation due to their partner, I believe it's Twilio, where, if you're on an incident call, there is a four-hour time limit. We often have calls that go over four hours in length so people have to drop and rejoin to reset their four-hour timer. It's a minor inconvenience, but it's not ideal. That is pretty persistent with any IT alerting partner you go with.
For how long have I used the solution?
One to three years.
What do I think about the stability of the solution?
We've been using it for a year. There have been a couple of instances of our encountering some weird issues with the calls being dropped, or people not being able to hear. That was more towards the beginning part of our go-live. We ended up identifying it through troubleshooting with Everbridge and Twilio and different networks. It was an issue with an AT&T circuit somewhere. They were kind enough to give us a different set of bridge numbers so that we were going on a different path. Since going to those numbers, we haven't encountered that same sort of instability or call-quality issues.
Other than that, occasionally a service advisory will go out from Everbridge where a certain communication path is having issues. But those are typically quickly resolved. We are pretty comfortable with the stability.
What do I think about the scalability of the solution?
We've had no problems with scalability. We brought in Everbridge shortly after we had bought another company and started merging together into a new, unified company. That, obviously, brings some substantial scaling challenges. But we've encountered no issues with adding many, many more users to the system over the last several months.
We only have a couple of hundred people directly interfacing with Everbridge, maybe a few hundred. But as far as the users we're communicating to, it's a few thousand and that keeps increasing as we progress.
How are customer service and technical support?
Everybody I've dealt with at Everbridge tech support, barring one individual on their staff, has been pretty nice to work with. They're knowledgeable, they're helpful, they want to assist. There have been a couple of times that it's been challenging to get the escalation that we were looking for. We were hoping to get some more urgency, especially when we were first going live, with the instability issues we were seeing. It took a little longer to get that escalation that we were looking for, but once it happened, we certainly got the amount of attention that we needed on the issue. For the most part, after that experience, it's been pretty good.
Which solution did I use previously and why did I switch?
We were not using a different solution previously for IT alerting. The on-call schedules were managed and stored in ServiceNow. As I said, the reason behind getting IT Alerting was that everything was manual. We weren't using a competitor like PagerDuty or xMatters.
How was the initial setup?
The initial setup really depends on the environment you are operating in. They can easily integrate and do imports from an HR system and from ServiceNow and from many different applications. They do have a lot of good options that you can easily get things set up with. For internal reasons we couldn't do it, but I definitely saw that the ability was there to do it. I would call it straightforward.
We did the deployment in under a month. It was a pretty aggressive time frame.
We didn't really have an implementation strategy. We were focused on getting the users that we wanted to bring over from ServiceNow identified and on marrying up that data with what was going to be in Everbridge. You can pass that information along through the API connector. We ended up just doing an export and then manually uploading it to Everbridge.
It was a matter of identifying who was in the system that we needed to get in there as a contact. From there, our strategy was to get meetings scheduled with the high-level folks who pass information down through the disparate on-call groups that they're in charge of, so they could let them know what changes were coming.
One big part of the overall strategy was having executive backing, because going from one organizational culture, where folks are used to having a certain amount of time to respond to a bridge call, to Everbridge, where we wanted to have the system escalate, and escalate quickly - since when we engage those folks on a bridge call, it's because we're losing money and our customers are losing money - you have a lot less time to respond to a call before it escalates. Obviously, people who are living on-call schedules are not going to like that kind of news. If you don't have that executive backing, then people aren't going to be as quick to adhere to the new organizational culture of, "you need to be on a call within a few minutes, if we start paging you." There would be no more of this, "I'll be there, when I can, in 15 minutes."
What about the implementation team?
We had our integration consultant from Everbridge on hand. There were no third-parties.
What was our ROI?
We have definitely seen ROI, not in terms of dollars and cents, but the mean time to restore from major incidents has been more than halved in terms of duration. Being the company we are, in the financial world, and with how many transactions are processed through us in a second, the potential financial savings from even just a minute of reduced outage time can be substantial.
What's my experience with pricing, setup cost, and licensing?
For the one-way license, which refers to someone is just on the receiving end, it's very affordable. I was actually surprised that it was a really good price. The two-way license, like for an on-call resource who is actually going to be in a calendar and be paged, it is a bit more expensive, but for the gains that we've realized, it's certainly worth the price.
Which other solutions did I evaluate?
We looked at MIR3 - they are called OnSolve now - and we looked at xMatters. MIR3 just didn't check enough boxes. It didn't seem like a good solution for storing and managing and quickly engaging people on bridge calls. xMatters and Everbridge seemed to have a better, more intuitive user interface, more robust options, better reporting options, and more flexibility.
What other advice do I have?
Get that executive leadership backing, and make sure that you're not just going to use Everbridge to page out to people in a different manner. You should look to set that "timer" pretty low on bridge engagement and get people used to responding and getting on bridge calls immediately, because every minute of an outage is lost money.
Determine up front if you are going to do the group integration from whatever application you might be looking to do the user-sync with. If you are going to have an application with on-call schedules maintained, such as Service Now - as I believe there is an option to turn on group sync - be careful about turning that group sync feature on. User sync is pretty straightforward but we were warned against using the group sync feature for various reasons, even from within Everbridge.
Our users are support staff, on-call resources, on-call leaders, incident commanders, communication managers. There are a couple of senior leaders who know how to use it, but it is mainly incident management and communication management.
Our deployment team was just a handful of folks. We needed a little bit of partnership from our ServiceNow folks to get the API into place. You could go with a half-dozen people on the integration. For the maintenance aspect, it's even less than that.
There are four of us who administrate the tool. I'm the communication management piece of it. My manager handles major incident and critical communications, so he's incident management as well, and he does a lot of admin work in it. Our project manager is the incident commander and communication management.
We have support staff who don't have the rights to kick off Everbridge to automatically engage people, but they'll still access the Everbridge Member Portal to manually look up resources and call them for lower-priority issues. We use it pretty heavily right now and we are definitely looking at other ways of utilizing the tool. We expect it to increase pretty substantially, as we go forward.
One of the big things that we're looking to do is integrating it with event monitoring. We're looking to further reduce the mean time to assemble for major incidents by bypassing the current process. Currently, event monitoring takes in a ticket and it gets assigned to a queue in ServiceNow where an agent will see it, and they'll call out the support person. That person will say, "Okay, well we need a bridge call for this." What we are trying to do is identify, with the various application support teams, among the events that are creating tickets, which ones are deemed "critical" that could be a precursor to a major incident. We want to identify those and create incident conditions in ServiceNow that will engage an Everbridge template to get the incident management team engaged right away, rather than waiting on those manual actions to happen. We're still in the early stages of that, and we are looking to increase that sort of usage for Everbridge to gain more efficiencies.
Some of them are live right now. We call them the "Everbridge critical alerts" and we have many that are already in production. We are looking to expand that even more.
I would rate Everbridge IT Alerting at eight out of ten, overall. It's a very powerful tool. We've made a lot of efficiency gains but there are definitely things that, from an enhancement standpoint, we would like to see added to the tool. The progress on that hasn't been as quick as we'd like. Its been pretty slow going.
With what we already have in place, it's enabling us to do a lot. I absolutely love having the tool. I would not rate it as a ten out of ten, but they're definitely heading in the right direction. From what I've seen, as far as what they are planning on having, I would say it could be a ten out of ten this time next year, if things go well in year-two. But for year-one, I would say it's an eight out of ten.