Tidal Automation is a better tool than Redwood for enterprise-level applications. It is a stable tool and easier to integrate with any product.
We have a lot of custom-made applications within and outside our company. The product helps us connect with all of the applications in a better way than Redwood. It has in-built plugins enabling easy synchronization.
Tidal Automation’s most valuable feature is customization. It can work and connect with any app.
The product’s UI is outdated. They should work on this particular area. There could be better accessibility for the iPhone and other software similar to Redwood.
We have been using Tidal Automation for five or six years.
I rate the product’s stability a seven out of ten.
We have to make minimal efforts to scale Tidal Automation. Its scalability is an eight or nine out of ten.
The technical support services are good, but they should respond faster.
Neutral
The initial setup process is straightforward.
The product generates a return on investment.
The price is reasonable in terms of the product’s functionality.
I rate Tidal Automation an eight out of ten. It is a very good container if you are looking for a single enterprise-level scheduler to control the entire operation.
The primary use case for Tidal Automation software will depend on the specific needs and goals of each organization. It includes tasks such as job scheduling, workload automation, and event-driven automation. It helps in batch processing, data transfers, and job scheduling.
The software can also be used to integrate with a wide range of enterprise systems and applications, including ERP systems, databases, and messaging systems. Tidal Automation software improves the efficiency and accuracy of business processes, reduces errors and delays, and optimizes resource utilization.
It has improved our organization in the following ways:
1. We have intricate workflows and business processes using Tidal Automation software, which has increased output, accuracy, and efficiency.
2. Tidal automation software has greater visibility and control over its business processes.
3. Tidal Automation software provides real-time monitoring and alerts, allowing organizations to respond quickly to issues and ensure that workflows are running smoothly.
4. Tidal Automation software can help reduce errors and ensure that tasks are completed with a high degree of accuracy.
Tidal Automation software provides real-time monitoring and alerts, allowing users to track job progress and identify potential issues before they cause delays or errors.
Users can create customized workflows that integrate with multiple systems and applications, allowing for end-to-end automation of complex business processes.
Tidal Automation software is a valuable tool for organizations looking to improve efficiency, productivity, and accuracy.
Tidal Automation software includes workload management features, such as job prioritization and distribution across multiple servers or platforms.
Areas where the product or service be improved are:
1. The software's performance and scalability could be improved, particularly when dealing with large-scale workloads or complex business processes.
2. Tidal Automation software could become even more valuable to organizations looking to automate and optimize their business processes.
These additional features should be included in the next release:
1. Adding machine learning capabilities to Tidal Automation software could help organizations to automate more complex workflows and processes, such as predictive maintenance and anomaly detection.
2. Adding more customization features or a more flexible API could help users tailor the software to their specific requirements.
3. Adding collaboration features such as shared workflows, team management tools, and commenting capabilities could help teams work more efficiently and effectively.
I've used the solution for one year.
We use it for a host of standard/general stuff, like batch workflow automation, in the front and back offices. We have also centralized all of our SQL Server maintenance that is running on it. Instead of having SQL Server maintenance plans or jobs running on 300 or 400 disparate servers, we run them through Tidal so we have consolidated administration and reporting that feeds straight into ServiceNow.
Last year, we made a step change with our DR recovery process. We had a bunch of people running manual scripts and different things where you have networks: Wintel, DBAs, or application support teams. They were running their own separate scripts to do application failover. This is different when it's active-active or active-passive replication. What we did was integrate it with different command line driven jobs, like PowerShell commands, to effectively failover applications and infrastructure into a sequenced set of dependant jobs. Therefore, if we need DR, we were not relying on a mix of SMEs saying, "Where was that script or how do we fail this over?" Instead we can just push a button and the thing fails over, which is beautiful.
Additionally we do compliance reporting from within Tidal and like many people we are regulated from PWC. Everyone has the technology control frameworks that they have to evidence. Instead of people taking screenshots, we will effectively find out what information PWC need and build the job using CLI which runs on either month or quarter end. The job will go off, collect that evidence, come back, and be formatted. Then, we just drop it in SharePoint or use Tidal to save it to a file share, sending an email off to say, "Your evidence is collected. You need to review it, then sent it onto audit."
We use it for a vast array of housekeeping jobs. It is not that Tidal is a monitoring tool, but automation is basically as far as your imagination can take you with anything that runs by a command line, which is virtually anything you can do.
We previously had a use case for it to give us a quick alert for when some of our infrastructure became unavailable. We just had it running every minute. Typically, it's not an enterprise monitoring tool, but if you have some deficiencies or things that you need to enhance, or give a different sort of dimension to, we've used it for that in the past. We also run it against our infrastructure using PowerShell to pull a whole host of reporting from our infrastructure daily, which is useful.
We use Tidal to run SQL Server and Windows. There is not really any Unix.
Since we start using it, they do more stuff in AWS. They now have a whole bunch of different cloud capabilities. We are moving towards private cloud. We're in the sandbox at the moment.
The product helps our company in the way that we've engineered it using bespoke jobs that we've written in a clever way. There's nothing directly at the moment. That might change as we move into the cloud, depending on which cloud we go with or on the adapters that they use, e.g., if they have native S3 adapters or events that can fire Lambda functions, which are a bit more interesting to us.
There are many valuable features. I would struggle to say that there is one more useful than another. Job Events and its email capabilities are good.
We have integrated Tidal with other automation platforms. You can integrate legacy platforms, as the integration is easy. Overall, we have good impressions of its ability to manage and monitor workloads.
They have a bit of work to do on the ServiceNow Adapter. At the moment with 6.2.1, we can send an SNMP Trap to ServiceNow in order to create an incident fail. However, there is so much scope for a CLA API interface between the Adapter and the stuff that you can do with it. I would have other use cases for different things within ServiceNow potentially if that was the case.
The reporting is kind of lacking and not super awesome. They have a product where the administrative overhead isn't that straightforward. Maybe, we're using it wrong.
The ability to express jobs as code is something I wanted for years now, especially as we move into the DevOps space. We have been doing one-touch deploys in terms of our CI/CD pipeline for a while and we have releases and code deployments that go through environments with a single tool for deploying. Therefore, SQL code, SSIS packages, and registry entries can install something all at once. Tidal can't do this for jobs, because they use a Transporter mechanism, which baffles me because the product is a SQL Server on the back-end. We would like it for a developer to be able to push a button saying "Script", which exports a script for the injection from one environment to another. This is what it needs instead of a clunky Transporter tool to take it from one environment to another. If they could just rip out the code that they were going to insert into the next phase, then we can express those jobs as code and dive into our consolidated release process. For me, in the DevOps space, expressing jobs as code would be the way to go.
The solution’s current drill-down functionality is alright because the Client Manager is an actual database. With the next version 6.5.3, they put that into a memory database. Therefore, you have no real ability to go through and have a look at it. I think there's a gap there.
10 years.
It has been super stable. There are no complaints on stability. We would not be using it if Tidal wasn't stable. You can't have an automation system that is unstable because it is too critical. If it's fallen over, everything is delayed in the morning. The business impact will be significant, because potentially your front office can't trade. If your automation platform doesn't work, you're in bad shape.
Two people are required maintenance.
We have had no scalability complaints. It is all pretty straightforward.
We're looking at rolling this out a bit more globally. We have some people in India, North America, and elsewhere. The rate that the skills get picked up can depend on the region, but it also depends on the skill sets that you already have. If you already have some knowledge of an automation tool or orchestration tools, then it's quite intuitive. However, if you have somebody who has never seen it before with no knowledge on the information system, then it might take them a bit longer.
We have about 100 DBAs, testers, business analysts, and automation developers using it. At one point, we had nine live environments.
I have been through many different iterations of the company. They used to be owned by Cisco, then Tidal was moved to somebody else. Now, it's with STA Group who seems very responsive and customer-driven, which is nice. They are making efforts to listen to their customers and see what they want, which is great. It's still in the early days to see how reactive they are in terms of development.
I've never called the technical support. My guys are the ones who have to speak to the tech support. I've not had any complaints.
We went from AutoSys (formerly CA) to Tidal. We switched because of CA's expensive licensing. They were also behind the curve.
The initial setup is fairly straightforward. There are a few nuances or a couple of bugs, but as soon as you report them, they are fixed as STA Group is fairly reactive.
We are in the process of an upgrade, but we have a whole lot of other work going on and are not under any pressure to get it done. We just took our time with it. Therefore, it's not like we're doing just this upgrade. Though, you could install an instance in a couple of days.
The amount of people involved in an upgrade or deployment depends on how your infrastructure stands up. If you have a small IT department and you have one guy who administers Tidal, builds the servers, does the installations, and has nothing else to work on, then it is pretty quick. If you work in a larger organization where you have teams working in silos where everyone is maxed out with BAU and projects, then you may have to wait three weeks for your servers and a bunch of other stuff. It depends on how siloed your infrastructure setup is. Once you have the servers, then you can install the thing with probably two or three guys. Though, it depends on how complex your setup is. E.g., if you're doing HA between different regions in AWS, then you will need more people from information security along with network specialists.
If you can automate things that people are doing, you will save time and resources because people can be doing more value-add work than manual stuff. Broadly speaking, if you start automating all of your clients' compliance evidence and collecting, it becomes standard, then the people who are doing that can do something more useful. If you extrapolate that, then that is time well spent and saved.
I have had no issues with the licensing.
The solution enables admins and users to see the information relevant to them, but this is bundled as an add-on that we would have to pay for. I am attending a webinar on this feature next week. It remains to be seen how much it costs and what the value is. It's touted as giving you all the analytics that you want. We have had it 10 years and got by without this feature. Instead, we have DBAs who can write queries to pull out whatever we need from our SQL database. There are ways around everything, as there are a million ways to do stuff.
We have evaluated other solutions.
I would rate the product as a seven (out of 10). I love the product. It's pretty good. There are more reporting analytics that I would like to do and see out-of-the-box. I would also like to not have to pay for it. Our implementation has been super stable, and it really kind of ticks all of the boxes.
The Adapters that they provided are quite good. We have SQL, Oracle, and other ones that we have used in the past. I'm looking forward to using two or three adapters and being able to do harsh cloud native capabilities with Lambda. These are particularly interesting as we go into the cloud space. I haven't used them yet.
I have three installs of Tidal: production, qual and dev. I have a portfolio of 12,000 unique job definitions in production, 13,500 definitions in qual, and about 8,000 in dev.
The Tidal adapters I use are for Windows and Linux agents, as well as Informatica, Cognos, and mSQL.
With the portfolio of jobs that we're talking about, it's continuing to grow. There is way more work being added to the system than there is work that is being retired from it. That's just the way the animal works. It's been able to handle, perfectly fine, the complexity of the interrelationships between the processes.
We actually ported off of Maestro. Maestro was the scheduler that we were using, enterprise-wide, and it was very inefficiently used when I got here. When we came up on Tidal, we didn't convert anything. We built all of the definitions that exist in Tidal. So over the 15 years, that portfolio has grown.
As a whole, we're trying to automate as many things as we can to alleviate the manual processes. One of the things that Tidal has helped us with, because it is cross-platform: We had a number of different schedulers in this organization and we've been porting everything that was running out of these other, unrelated schedulers into this scheduler. That has afforded us the ability to set up direct dependencies between processes that couldn't talk to one another before. Over the 15 years, we've definitely gained a lot from that. What had been manual controls have become automated controls, by using this tool to replace a number of schedulers.
The automation aspect of the solution is the most important. I'm able to construct groupings that have dependencies which automatically allow the proper jobs to run in the proper sequence. That's the strongest selling point of any scheduler.
As for the solution's ability to enable admins and users to see the information relevant to them, the security model that I use is fairly simple and straightforward. For developers and other folks, an inquiry-type access is more suitable for the production environment. I've added functionality for people in both the qual and the dev environments, based on their roles. But I haven't restricted anything, meaning that anyone who has an account can see everything. There is a lot of flexibility in the way that things can be configured with Tidal. You could restrict it down to the point of people only seeing those things that are applicable to them specifically. I found that that would be too restrictive, and result in a lot of overhead to manage. So I went with a much simpler model, but the flexibility is there.
There are certain things I can put in play, triggering events based on statuses. For instance, if I have a certain job type where a number of the jobs are going to "waiting on resource" in the middle of the night, I can configure alerts so that I can assess those and then determine if I have to raise the job limits on some of those resources to make sure that we're not having things held up on necessarily. By the same token, if we're having long-running processes, I may want to tailor that down so we don't have so many processes running concurrently. There's some flexibility in that. I haven't had to rely on it a lot, but there are some features there that can be tapped into.
From an administrative point of view, I wouldn't give really high marks to the solution. I actually entertained getting the JAWS application at one point. One of the shortcomings with the scheduler is the reporting capabilities. At least at the time, JAWS was the best that they had for a third-party integration. I think they've got things in the pipeline to help alleviate that gap.
Also, one of the things I'm concerned about is that, with the security we have, there's a hazard that somebody could go in and accidentally delete a master grouping of definitions out of Tidal. Right now, I don't have an easy way to recover from that. It looks like a couple of things that are in the pipeline with Tidal are going to allow for that kind of recovery. There should eventually be a replacement for the Transporter tool. That sounds like it's going to have the capability of doing copies out of Tidal. If I scheduled that once a week, it would give me a copy of definitions out of Tidal. If it turned out that one of the operators, who had the rights, accidentally deleted a grouping of definitions, I would have something that listed definitions that I could go back to and recover.
I've been using Tidal Workload Automation for about 15 years.
The stability has been fine.
In fact, we're going back to using the master and the fault monitor. We had it disabled for some time, but we've gone back to setting it up with the fault monitor and the master, and the backup. There was a problem with it. There was some kind of a fault status that kept getting triggered. The network person who was in charge convinced us to disable the redundancy that we had set up, and we've just recently gone back to it. And it's been working fine.
We haven't hit any roadblocks with volume, but I think we've been sized properly too, behind the scenes, with each upgrade that we've done. It's been scaling fine. That's the bottom line.
There are systems out there that are larger than ours. We try to get to the user conference, here in Boston once a year, to do some comparisons to other organizations and the way they're using the tools. It's an information-sharing session.
Whenever we go for an upgrade, we look for an assessment of whether we need to provide more horsepower or not. If any of the configuration has to change, we watch that carefully with each upgrade. There's a formula that Tidal provides on whether you should have a small, medium, or large installation, based on the number of definitions that you have. They help with calibrating that.
We consider Tidal to be an enterprise scheduling application, so any new process that comes along is first looked at to see if it can be run from Tidal, whether that would involve purchasing another adapter or whatever else would make it work from here. We want it to be an automated function as opposed to being run manually and not integrated with the scheduler.
The technical support is much improved. That's over the course of 15 years. Tidal has gone to great lengths, with the transition to STA, to strengthen its support capabilities and also strengthen the relationships it has with its clients. STA seems very interested in trying to focus on a direction, advertise that direction, and make the current clients comfortable. That, in turn, will help them take on new clients.
As I mentioned, we came off of Maestro. Back in 2004 or 2005, when we were looking at schedulers, Tidal was one of the solutions we demoed. Universally, we all decided that Tidal seemed to be the better candidate.
The setup was pretty flexible. We had to come up with our own ways of deciding how to group things and what our naming convention would be.
When we first came up on the product, one of the issues that we noted was that the default sort for all of the jobs was alphabetical. That complicates the ability of the operators to visualize the order jobs should run in. To overcome that, we came up with a naming convention that puts a prefix on all of the job names with a number. So when we create our groupings, within a grouping it will list the jobs in the order that they run. Half of Tidal's clients wanted to see things alphabetically listed and half wanted to see them listed numerically, in the order that they run. The vendor wasn't willing to modify the product to give the user a choice of one order or the other.
I don't remember the original installation taking that long. It took us a while to actually build all of the job definitions. That was a lot of work. It was done within about a week. Once the equipment had been spec'ed out we had an onsite install here in the computer facility.
We've had to train a number of new operators and I don't think it's been a terribly big learning curve for them to understand how it works. The developers, in fact, self-trained in their environments and they seem to be able to maneuver fairly well. There are times I have to explain things here and there, some ways of handling things that are more convention. Those are things they have learned over time. But they seem to do all right with it. There isn't that much of a learning curve.
The only people who need to have the training would be the operations staff. I think there was a beginner's and intermediate course that we originally took, when we came up on the product. And then we learned things as we went.
One of the things that would be beneficial though would be some training that incorporates best practices. You can go through the manual and it will tell you, "This feature does this," and, "these are the parameters that you need to put in," and then the delimiters, but it doesn't necessarily tell you the best use case for certain functionality. I've had a few people mention to me "Oh, you shouldn't do this, and you shouldn't do that." Well, where does it say that in the book? It doesn't. And that's the problem. There's a little difference between an instructional manual that gives you the nuts and bolts of how to do things, and something that's more tailored to best practices, or recommendations of things you should not do. And some of that has to do with the architecture behind the scenes. Users wouldn't necessarily know that unless there was some documentation expressing it.
I don't really have metrics for ROI. It's more of a feeling because we've been able to consolidate from all these separate scheduling products into this one scheduling tool, allowing us to have direct dependencies between things. That's an efficiency in itself, but I don't have any statistics to support the number of hours saved and the number of dollars saved. Overall, it has improved our business model with automation.
My experience was that it was very difficult to figure out the licensing cost on an annual basis. I don't know if they've changed the model, but I remember it would take a month to reconcile if we were being billed the proper amount because it was based on the number of CPUs; if they were test CPUs or production CPUs. I recall, and this was probably five years ago, that it was very difficult to reconcile the annual statement with what we had, and to verify that they were components we were using.
Our ability to budget for the solution is a fairly easy aspect of it. One of the difficulties that I have internally has to do with the specialized adapters. I don't think it's well known within my company that I can't just snap my fingers and get an adapter. There's a cost associated with it and the license key has to be updated after we've made the outright purchase of it. I don't think there's familiarity, within our company, of budgeting for the coming year if it involves these additional Tidal components. That's nothing to do with Tidal. That's just an internal struggle.
There were five solutions we looked at in total. Two were ruled out right away. When we went to do demos with the three of them, the third one couldn't even do the demo, so it came down to Maestro and Title.
One piece of advice I would have is that if you get into a product, try to keep it upgraded. It's to your benefit, support-wise to be, maybe not on the cutting or the bleeding edge, but close to the current version. That's been a pain point for Tidal, to try and get their clients up to speed.
Stay on the latest version because of the functionality. It's not only relevant to just this tool, but to many IT tools. It's just like the next generation of laptops that are coming out; they're coming out more quickly. The same thing is happening with the functionality that is being added to all of these products, including the scheduling application. It's important to go through the pains of staying up to date.
It's been a good product. We could have done a lot worse. This is a heck of a lot easier to use than some of the other schedulers that I've used in the past. But, then again, it's been proven as a solution, as well. Other solutions are all moving targets. Everybody is making changes in their products. At the time that we made the selection of Tidal, it was definitely constructed a lot better. It was easier to use than the other option.
In terms of the number of users in our organization, I honestly wouldn't mind if everybody in the company had an account to log into Tidal with inquiry access. But I think we've got around 300 accounts set up in each instance. They could be used by managers, developers, operators, and all the other IT folks who have accounts.
For deployment and maintenance of Tidal, since we're doing a 24/7 staff, we're talking about eight people, and three or four other people who are going to be part of production control and/or an IT server ops-type of functionality, because you need that level of support as well from time to time. So we have twelve or so people in one capacity or another maintaining Tidal.
I would give Tidal a solid eight out of 10.
We are mainly using it for triggering data jobs. It does a lot of ITIL stuff and data movement from systems into Hadoop. We use it because it has the capability of dependency triggering or dependency running. That's the main idea behind it. Also, it helps us to centralize and organize jobs across the organization.
We use Tidal to run Hadoop backup system, SAP HANA, and SAP BusinessObjects. We also trigger a lot of jobs into SnapLogic, Salesforce, ServiceNow, Workday, and Tableau, along with a couple of dashboards. We run a couple of batches from our Unix and Windows machines: the stuff that the developers are working on and want to run in ITIL. But, SAP is the main thing.
The main goal is to use Tidal for managing and monitoring cross-platform, cross-application workloads. The ability to manage those loads is what they do well. I can put a job to run in SAP, and once the job ends successfully, I can run that job in Hadoop. Or, I can run that job in Salesforce.
Tidal enables admins and users to see the information relevant to them. We have something like 50 different teams working on our Tidal platform. We segregate between them using work groups. When a user logs into Tidal, they only see what they have permission for, not other projects. Data engineers are the users of this solution.
A user who comes into Tidal, develops his job, and creates their job in Tidal, then triggers the job or sets up a schedule. An admin is someone who keeps the lights on, making sure the platform is up and running. They maintain the solution and configure it, doing upgrades.
If I just want to monitor my job, that is something that the solution does really well because there is some constant job activity that you can login and see what has happened every day and every minute. That is pretty good. An admin can drill down to processes and data, but I don't think they are doing that.
The solution has helped to eliminate weekend hours. In the past, we had to schedule a job every Saturday. Then, someone had to login and run the job. Now, Tidal has the capability of event-driven jobs. For example, if a job is failing, we can do something. Or, if a job is completed abnormally, we can rerun the job. So, all of these features that they offer help us not to come into the office on a Saturday. We don't need to have a human person do those weekend activities and treat them. They also thought a lot of about outages in the product. You can set up an outage to an adapter or connection, to say, "Between these hours on the weekend, I don't want to trigger any jobs." That works very well.
The job dependency is something that you cannot have in a regular, simple cron job or simple scheduler dependency. The event-driven jobs are core for us, as we really need that. Therefore, we really need Tidal with its ability to run thousands of jobs per day.
We started to deploy Azure, and it's still not fully baked. We are struggling with it. It is not something that has worked out-of-the-box. We haven't installed Tidal in the public or private cloud. We have a problem with security. While we can install the entire platform in the cloud to handle separate work or an entity, if we want to centralize it, then it's a little difficult.
They don't have good reporting capabilities. From the user perspective, I have 6,000 jobs running per day, and I would like to track them to know exactly what is going on. E.g. if a manager asks me, "Can you bring me this data or can you do a dashboard or report?" I need to take a lot of actions in order to do that. It's not easy to compute that data.
We are now testing version 6.5. The speed of this console is much better than 6.2, where the speed has not been sufficient for me.
Most of my users are doing customer service review these days. So, we are asking the customers what they think about Tidal and what the vendor needs to improve. The number one that we are exploring is the user experience (UX). It has a lot of features, which is one thing that is great. On the other hand, the user experience is a bit old. It is hard to find what you're looking for. The UX is not intuitive for all users. So, if I'm a user, it might take me some time to know where I need to find my stuff.
It takes a lot of time to learn the product. I have admins and developers who are working on the products for the last three to four years and still don't know all the functionalities. Tidal has really great things about it, but people are focused on their day-to-day job and the solution is not intuitive.
We have internal training where we do two weeks of training for three hours each day. So it's approximately 30 hours of training. I cannot say after that users know everything. It takes about six months to ramp up on Tidal to be really good and professional.
I have been using it for the last three years.
In version 6.5, it is very stable and works quickly. The UI works quickly too. Their services load pretty fast. If one of the servers reboot, they have a layout of the high availability. This means that from each component of the product you have two items. If one of them goes down, then the other one kicks in and starts to work. I really like that idea.
In terms of room for improvement, if one of the master goes down, then another one takes a minute to start. While it is not a big deal, when you commit on four nines, one minute is huge. So, I'm pushing the vendor all the time to be better on this.
They still haven't implemented the load balancing-oriented thinking. So, if I have two client managers, I cannot put them behind a load balancer. Or, I can put them there, but the load balancer will never have a health check. That is something that everyone is doing, building health checks, and they don't have health checks on clients for load balancing. Maybe this will come in the future. I submitted a request for having a health check for load balancers.
In version 6.2, they had a lot of problems. One of the problems is the Oracle support. We are using Oracle RAC and high availability on Oracle. If one of the databases would suddenly goes down, the entire system would crash. In version 6.5, we have tested this. They have done significant work and it's working perfectly. It's not crashing and working continuously without any issues. From this perspective, I am very happy with the new version.
If you have enough memory, it is scalable. We are running 20,000 jobs. We just increased our memory. It scales really well.
The North American technical support is very good. They go the extra mile for you all the time, and we are very happy them. We have had some problem in the past with the Asian support during IST time, while it is night in the North America. However, I think it's getting better. Overall, I'm very happy.
We used local solutions, like scheduling for each platform, such as SAP Scheduler, SnapLogic scheduler, and cron jobs. We didn't have a centralized place.
I was the architect of the initial setup. The initial setup was complex; it's not easy. They have a lot of settings and configuration that need to be done. There are a lot of small things that vary from environment to environment, and they fail to consider every situation.
The deployment takes a couple of days.
With our first environment, we tested it in a sandbox. I let my admin play with it to see how it behaved and what are the downsides. Then, we created a document. While I know that they have a document for installation, every time that we go to install, we are finding new issues.
I'm behind a firewall and we are in a limited environment. Our infrastructure is built differently from what they probably tested on their environment. So, it's a bit different from what I need to install. I first put it on the sandbox to see all the issues that we are facing, document step-by-step what we did, and then I go and do it in stage. Now, stage is the place where the developer come in and develop their jobs. Once they are ready, we move the jobs into production.
Stage is really almost production. If stage wasn't available, then the developer could not work nor deliver. We see if it works for at least three weeks. If we don't have issues during that time, then we deploy to production.
They do a better job in version 6.5, which we are testing now.
We have seen return on investment.
BMC is really expensive. The other solutions are about the same price. I think Tidal is even cheaper than the others, such as CA, Stonebranch, and JAMS.
Our licensing model for Tidal is on an annual basis. It is very good and works well for us. Tidal's licensing is very transparent and simple. It lets you know, for the amount you use, that's the price that you pay. So, we buy X number of licenses, and we know that this is where we are. I'm very happy with that. I saw the licensing modules on other platforms, and I didn't like them. Other companies and solutions would calculate the connections, adapters, and instances. I think that's the reason that BMC was pretty expensive: They just didn't understand what our needs are.
The solution has no hidden costs. It helps me to plan forward into the future. I know that I can add another 100 or a thousand jobs, and that's how much it will cost me today.
We did evaluate other schedulers. This was the best solution.
I was not the one who selected it in the first place. I was the one who asked to evaluate a replacement at some point. There was a time when Cisco was the owner and we felt like Cisco was not delivering the product like we wanted. We sought to move to a new solution and assessed different solutions: BMC, CA, Stonebranch, and JAMS. We installed all of them, running all our tests. It took us six months to do our evaluation. Eventually, we found out that they are very similar from the infrastructure side. I could not see any advantage using the other solutions.
We discovered that we are good with Tidal and what we have. Then, a new company acquired Tidal from Cisco and they promised a lot of things to be better. We felt that the solution was going to a better place. So, we decided to wait and see how much they invest on the stability. We have been happy with the results. They are really focused on the customer and our pain. They are trying to remediate everything that we have issues with. Therefore, we decided to stay with them for now.
Don't be afraid. Just do it. You will enjoy the features of it. It is a great tool.
You need to test Tidal many times. It's not straightforward. You need to test and learn it.
We have something that is not unique to many platforms. I have five guys who handle the platform. That's costly for us. We would like to see the platform more automated or straightforward. I would like to not need to hire so many people just to administrate and maintain the platform.
Our capacity has increased in terms of the number of jobs and integrations, but that is a natural thing. I don't think it's related to the solution. When you start to develop jobs, then year by year the number of jobs grow because the organization is growing.
I'm very happy with the product, but it's not a fully baked product. It requires babysitting. I have worked on other solutions and know what is there. This takes time for us to install, upgrade, and task because there are so many components to the product. If you do one little mistake, then you can screw the system.
I would rate the solution as an eight (out of 10).
For us, the calendaring system is very robust. Some of the teams have very specific requests for when they need jobs to run. That's been really valuable, because a lot of times, when people run scripts, if they run on a holiday, they're going to fail. We've even started adding some European holidays and other times when scripts should not run because they're going to fail, because they try to connect to external exchanges that are closed on a holiday. For things like that, things you can't do in a lot of built-in scheduling tools, Tidal has been very helpful. A couple of times a month it probably saves us work and the necessity of logging in from home and checking to make sure everything's okay.
Especially in the newer versions of Tidal, the segmentation of user permissions enables us to give people operator permissions for their jobs, to rerun jobs, but view-only for other groups' jobs. We're able to keep people from hurting themselves or other groups accidentally. The permissioning is really good. We have 20 different root-level job groups that hold all of the jobs for each team underneath it, in our shared space. I can set it so that the database group only sees database jobs, if that's all they want to see, so it's not cluttered with everyone else's jobs. But if there are teams that need to see all the spaces, we can do that as well. We can let them see only certain servers or certain users to run jobs. You can edit it too so that people don't see too much or don't get confused and lost in this sea of the thousands of jobs that they could be seeing, when they only need to see their own. That's been nice to set up over time.
In the past year, in particular, the client has gotten tremendously better. If you asked me three years ago, I would have said that the client was the biggest problem with Tidal. The backend was always really solid, but the client was pretty bad for a while. In the past year, with the new company taking over and putting a lot of development effort into the clients, especially the web client, it has really made people a lot happier when having to use the client and work with it. In the past, they begrudgingly used the client, but now they're happy to use it, which is a big change.
Because we've been with Tidal for so long, I can't compare it to the way things were before Tidal. Back before Tidal, there was much less electronic trading.
But an example of how we benefit from it is that we have Tidal jobs that load all of the trading symbols into our database every morning before trading opens. That's mission-critical in terms of getting ready for the traders to start trading on a specific day. If they don't have that updated information through the database, they can't trade.
There's a lot of overnight, big-data processing that happens, things that need to run all night. That's launched through Tidal and monitored as well. It's pretty much a 24/7 operation in terms of uptime, and we've definitely used Tidal to meet that goal.
The solution has increased productivity by saving staff hours. We have an operations team that's here 24/7. We have a runbook that says, "Okay, if this job fails, do this." I'd say 80 to 90 percent of the time the operations team is able to resolve a problem by following runbook and steps without having to contact someone overnight or on the weekends. But Tidal does save the person who owns the Tidal job from having to do work in off-hours especially.
What I like about the new company is that if you ask for something, and they feel like it would be a valid improvement, they're willing to push it out, even if it's a few months out. They make sure to provide it at some point. It doesn't just get lost in the mix.
I work for a financial trading company: stocks and options. The use cases depend on each group that is using it. We have a compliance group, HR group, and a bunch of trading groups and technologists. It's used for a thousand different things depending on the group. It's all to support a financial trading firm, and the processes that happen before the market opens and after the market.
We have a pretty good mix of Linux and Windows boxes, a good 60 percent Linux and 40 percent Windows. We launch trading scripts to start processes up, to stop processes, and to pull in data from third-party vendors; we have FTP jobs that do that. We run an Oracle backend.
From talking to the Tidal people, we have a lot of agents connected to masters, compared to most other firms. But we're probably middle-of-the-road in terms of how many jobs run per day. We're only slightly over 100,000 jobs per day, throughout the whole space.
The biggest problem for us was the Transporter tool that works through the API. It's like a GUI into the API where you can transfer and compare jobs between two Tidal spaces. Up until the last few months, the Transporter tool that was offered was not really good at all. It was hard to take a job in development and promote it to production. There was no really good tool to do that. They offered a tool, but it wasn't that good.
But they just put out the Tidal Explorer tool, which is basically a replacement for the Transporter. That looks promising. I haven't really gotten to use it yet, but it seems to be a better system. That's what people have been requesting for a while now: an easy way to promote and review changes; something like a script repository-type of system, where you can promote something or pull it down, compare it, and then, if you like it, push it. If it doesn't work, you can back it out to previous revisions. It looks like it offers all those features, but I really haven't had a chance to dig into it. I set it up and it does look promising for the future. It's probably something that we're going to try to integrate into the day-to-day processing once it gets released. I don't think it has even been released as general-availability yet. It's still in beta. But once it gets to be production-ready, we would definitely love to use it. It's something that's been on our radar for a while now.
Tidal also had a cache database, which was a copy of the master database, that the web client used. They got rid of that in the latest version, and that is something we had been asking for, for a long time. The way it had been set up didn't really seem optimal.
It looks like they're trying to put forth a better tool for certain places that were lacking.
On another topic, we have to set up ways to send a job event that finds a job that completes abnormally. What we do is send it to an SNMP trap that gets aggregated into one space and we can see those errors. We try not to use Tidal for monitoring, as much as for job launching and tracking. We have a Nagios setup so that if something fails, the error can be sent to Nagios and checked there. If a job is a long-running job, like an eight-hour job, we don't want that job active in Tidal for the whole time and taking up a job slot. We'll kick the job off in Tidal and it will show that it has completed normally. Then we'll hand it off to another tool to monitor that the process is running for the specified amount of time. I don't know if Tidal wants to get into the business of monitoring long-running jobs, but that could be a feature for the future: a job launching and monitoring tool. Using Tidal for monitoring doesn't seem like a good fit, but if they could offer something that did that as an add-on or include it, it might be helpful.
Finally, the solution is a little tough to learn. Talking to people who are new to using the Tidal interface, it's difficult. But I don't have anything to compare that to. They have said it's not as difficult as Control-M or some of the larger scheduling systems that people have used. It's not as hard as that. Tidal has worked to prevent new users, especially, who aren't exactly sure what they're doing, from hurting themselves too much, which is good. They've put a lot of restrictions in place to prevent people from doing things that weren't intended. There is a learning curve, but I don't think it's steeper than any other new scheduling system. In the past, we've downloaded some other options and they had a learning curve too. If you've never used it, there's always a curve, with the terminology, etc. But I don't think it's any harder than any of the others.
New users of Tidal need at least a month of working with it a little bit each day. I give people a three-hour introductory course. Every quarter I provide an overview for new users of how things are set up. Luckily, in our company, a lot of these new users are joining groups that already use Tidal on a daily basis. If they have any questions after the initial course, they can talk to their team. Over time, the teams that use Tidal are resources for the new employees. That takes a little bit of training off of my plate. Within a few months people are confident and moving along. It takes a few hours to pick up but to be fully confident it would take a few months to really feel that you know what you're doing in the space.
We've been using Tidal Workload Automation for 15 years.
I'm really confident in the stability.
Cisco owned the company for a few years and I felt that it was something of an afterthought for them; it wasn't really their business. They didn't really put the time and effort into it. It seemed like, for a couple of years, nothing was getting resolved and people were pretty unhappy. We ended up staying in a version that was years and years old, compared to what we should have been on because we were not confident in the solution that they were providing, to give us what we needed.
In the past two or three years, since the new company took over, we have much more confidence. People are much happier with the direction that Tidal is going and the features that they're releasing.
Our usage of Tidal goes up every year. That's not even from planning to increase usage. We have a few holdouts, people who still use Task Scheduler or cron, but over time they've all been folding into the Tidal space to have a better overview and a cross-platform way to see everything and rerun everything and be alerted. They've come to the conclusion that it's a better method, especially for overnight. We have an operations team that manages things overnight, so that if something fails in the middle of the night, that team can handle it, which they would not be able to do if it wasn't in Tidal, along with thousands of other jobs.
In terms of the number of jobs in Tidal, it's been increasing at between 10 and 20 percent a year. It's going up. It's definitely not going down. Initially, it was probably 50 percent a year because everyone was adopting it. Over the past five years, since it was already utilized by everyone, there has been a general 10 percent a year increase because of new jobs that need to be created and new processes that need to be started and stopped.
We're somewhere around 95 percent in terms of adoption of Tidal. There a few small groups that like to do their own thing and use open-source products, but those are groups that maybe only run Unix and that's it. They're happy with Jenkins or something open-source that only needs to run a few hundred jobs. It's only one platform, and it does what they need to do a little bit better than Tidal. But for groups that need an all-in-one solution, they've all gone to Tidal. If they need to do what Tidal offers, they're going through Tidal to do so. It's pretty accepted here.
Tidal was here when I got here. It's been around for a while. But over the past 15 years, I've been the one who researches the new patches and service packs and revisions and I've done all the upgrades.
The upgrading process is straightforward for me, but I've been here so long that it's just something I know. It has gotten much better with the new company. We're on a Unix backend, so a lot of times, with a simple hotfix or service pack, you can just run a shell script and it replaces all the files. It does everything it needs to do. It places everything in the right location, and then all you have to do is start and stop the backend process and it picks up the new revision. That's been really good. In the past, it was a more manual process. In the past couple of years it's gotten much easier in terms of being able to do things with one script.
The releases have been good with very few bugs or installation problems. There were some in the past, a few years ago, where you would try to run something and it wouldn't take into account your environment and it would fail. You'd have to tweak some of the script. That was a lot of manual work. The upgrade scripts, recently, have worked pretty seamlessly.
We have an enterprise contract, so if we want to add another agent, or if we want to add another master, we don't have any restrictions on those things. Other vendors don't have that flexibility. For me, as an admin, that makes it easy because I don't have to think about what a new master is going to cost or what a new agent is going to cost. If someone needs a new agent and they need to run a job from that agent, we just go ahead and do it. If we're in Dublin, Ireland and someone wants a new master because there's a group over there that wants to adopt Tidal, we can just say, "Sure, get a new license, create, and you're fine." For the license that we have, the flexibility is great.
I don't know if other people aren't happy with the licensing model because they have a non-enterprise license where they have to think about everything they change.
We're negotiating our new license now for April, which is when we have to renew. We've usually gone with a three-year license. The numbers that the new company has put forward haven't really changed significantly from our past renewal. People here are pretty happy with that. It's not like the new company came on and jacked the prices up exponentially. The new prices that we've received seem reasonable and comparable to the marketplace.
Because our environment is older, it's a little tough to integrate some of the newer features that they're offering. That's because of the way we had to configure our environment for older versions that didn't have these newer features. In terms of how you delegate permissions, how you set up calendars, who you give permissions to, my advice would be to figure out how the permissioning structure works before you set up your environment, and stick with a standard. A lot of the time, we're having to go backwards to make things standardized. If we started over right now, I know how I would set up a Tidal environment. It's hard to do that after the fact, and after things have been set up differently in the past. So try to develop the best system for standards and then keep that.
We don't use any of the Tidal adapters that they offer, just because we're heavy on development here. A lot of the people here, in the past, felt that they could write their own wrapper scripts to do the same thing that the adapter jobs do. That's ingrained in our environment now. We don't even look at the adapters too often, just because we have an in-house solution to those.
The vendor is starting to offer tools such as Tidal Repository, but that's going to be an add-on cost. I'm still evaluating whether it's something that we want to try to get a price on and use. It would allow us to see if certain jobs are running longer than they usually run. We could also see if queue levels are hitting their limits often and what we could do about that. The Repository seems like it's going to be a tool to gives you the drill-down information, like seeing how calendars are configured and a lot of the information that you're trying to get at. It's more like an admin dashboard where you can drill down. Right now, I just go directly to the master to search the logs, or we have all the master logs sent to a repository. I can search them there. We're doing things from our side with other monitoring tools we have and log aggregation tools.
We use it in our production environment. We use it to schedule and execute many jobs. It is used by multiple application teams within our organization, such as SQL, Unix, ETL platform, MFT, and our AWS team. Other application teams include front office, back office, and accounting. They all use the Tidal environment.
We use Tidal to connect to other resources and systems through many of its adapters, such as for S3, ServiceNow, and PeopleSoft. We use it to trigger jobs in those applications. The REST API is very easy. Our application users use it to create, rerun, or cancel their jobs. And if they want to update something in the database, they do it with the REST API. It's a good feature.
That direct integration to our applications means we don't need to do other integrations from other applications or create scripts for these integrations. The multiple adapters available for connecting to other software make it easy for users to use the solution.
With other tools, you do not have the ability to schedule jobs on their own. You need to create a group and then assign everything to that group. Only then will the job be able to execute. In Tidal, you can schedule a single job and there is no need to create a group. That's what I like the most.
There are other helpful features as well, like SLA monitoring and the data book so you don't need to maintain other documents.
We also use the Graphical Views feature because our end-users want to see how their jobs are being executed and to monitor flows. They want to see how a job flow is going or where it stopped.
The drill-down into details using the Graphical Views feature is a bit difficult and not that helpful. If you want to go into the details, you have to go to the Job Activity. Graphical Views is not that easy for getting that kind of information.
Also, the user interface could be much better. It's an old-looking UI. Tidal could be much more user-friendly and attractive. I think they are working on that.
Another change I would like to see is that when we face issues or bugs in Tidal, we don't get to the root cause. We are told by Tidal, "Just apply this fix pack and it will resolve the issue." But we need to know the root cause. What has caused that issue? That scenario can be improved and Tidal has to work on it. They should provide us with some root cause analysis about every issue because we have to provide the root cause to our organization. Without the root cause, it is difficult for us to identify the problem.
I started working with Tidal, as an administrator, in April, so I have been using it for six to seven months.
In our non-product environment, it is not as stable for us as it is in our production environment. I'm not sure why it is like that.
It is scalable. It has many features that other tools do not provide. It is very convenient to use and we can scale it up.
Tidal has been widely promoted in our company and we are getting multiple requests to onboard teams into Tidal. They are exploring it and giving us feedback. We pass the feedback to Tidal and then they provide us with enhancements. Our job counts are increasing, day by day.
We have 500 to 600 people using Tidal in our environment.
Technical support is good except for the root-cause issue I mentioned. But they are always willing to help. When we ask them to join a bridge, they usually join it and they try to support us. They have good product knowledge and help us troubleshoot things on the calls.
Neutral
We have two deployment models, making it a hybrid, because we have Tidal on-prem and Tidal for cloud as well. They are two different tools, but they are linked to each other. The cloud environment is based in different regions including US East and US West. We have Tidal in both.
The deployment was straightforward. It was not that difficult because we installed it in a Windows environment. It took three to four months because there was a process we needed to follow. We had a team of eight people involved.
There is some maintenance needed. Sometimes, we face issues with the Client Managers, which is the UI console for Tidal. We need to clear the cache monthly so that we do not face issues.
Our organization has been using Tidal for the past 10 to 15 years. That means it's a valuable tool.
It's cost-effective because it is available at a low cost. That saves us a lot of money compared to other tools.
I have worked with IBM Tivoli Workload Scheduler previously and Tidal has many more features. Tidal is a low-cost tool and not expensive in comparison to other tools.
The ease of use of Tidal for automating is moderate, if you have the proper knowledge and training on how to schedule a job. A new person will need some training on how to get a job scheduled in Tidal.
They have multiple new features coming up in the future. They are going to come up with repositories, for example. If you are going to use Tidal, I would recommend going through their documentation in-depth and only then start using it, so that you are aware of everything. When I joined, I was not experienced with Tidal, so I had to go through the documentation, and then I started working with our live production.
We're running jobs on a global scale. Being a global company, we're running scheduled jobs and ad hoc jobs across different regions. Jobs cover backend processing, financials, and the like. We're running on an SAP ERP system and we're also running Informatica for data warehouse. We're running BusinessObjects web reports as well as a lot of straight Windows and Unix command-line things. We run FTP processing, PGP encryption processing, and data services jobs. We're running about seven or eight of the different adapter types that Tidal has available.
We have it on-prem. Both our test and production environments are on fault-tolerant setups.
When I started here, they had already been on Tidal for about five years. So I'm not really sure where they were before Tidal. They did a lot of mainframe things in the past. From what I've heard from people here from the "old school," once they globalized and got everything into Tidal, the ability to see everything across the scheduling universe was a huge improvement. They didn't have to give different people different access to different systems and check four or five things, just to make sure something was running correctly.
The solution helped to reduce weekend and overtime hours. We're a 24 by 7 support model. Regarding the Tidal application, the one thing that we try to explain to anybody, from a support or monitoring standpoint, is that jobs trigger through Tidal, but not physically in Tidal. So if we have, hypothetically, an SAP job failure, it's not a Tidal failure, it's an SAP failure. So it goes right to SAP support, which saves time. In the environment I came from, they didn't have that mentality. So if, hypothetically, an ERP job failed, they'd call the Tidal person first instead of the ERP support. That type of understanding, as a whole, really helps from a support standpoint. The admins don't get a lot of calls unless there's an actual issue with the Tidal application itself.
In the time I've been here, we've definitely increased staff availability. From a business standpoint, we've started utilizing file monitors more, for what they call "file events" within the application. In the past, when an end-user would drop a file in SAP, for example, they'd contact our operations team, or send an email saying, "Run in this job." There isn't a real need for that in many cases. We've implemented a lot of file events that will actually only run jobs if they need to, if a file's available. Along the same lines, we had processes that would run a process in SAP, and even though it didn't create a file, there were other jobs downstream that would be hanging out and waiting for a file that never showed up. So not from just a staff availability point of view, but in terms of resource availability, it has definitely improved things a lot. From an operator standpoint, I would estimate Tidal is saving us 15 to 20 hours per week, just in manual interaction with inserting jobs on a request, since a lot of that stuff was implemented at our end.
Regarding job counts, we're pushing over seven million a year. That varies, obviously, depending on request jobs and other things. There are some processes that we shut down for year-end processing, so they stop running for a week or two. But from an expansion standpoint, we are constantly looking to see where else we can use Tidal, for new applications that are coming online or things that people are running on their own where they haven't even thought about Tidal's scheduling. In 2019, we did 7.7 million jobs. In 2018, we were at 7.1 million. In 2017, we were at 6.1 million. So with Tidal we're adding on the order of half-a-million jobs per year.
The feature that I find to be valuable, as I'm working with other folks, is the ability to cross-schedule across platforms, and the flexibility that comes with that. I'm kind of biased, as I've only used Tidal. I haven't used CA or IBM or any of the other scheduling platforms that are available on the market.
From a management standpoint, when using the solution for cross-platform, cross-application workloads, I've never had a problem with the application. It's very interactive, especially with the different security levels that they offer. We have two or three operators who are at a certain level where they can actually rerun jobs. If they fail, they don't actually have to get ahold of a Tidal administrator. The only thing they don't have access to is changing the master settings on the jobs. That flexibility of access is a big plus.
We do have a few developers who will actually set up processes within Tidal, but only in the test systems. They get a little bit more access that way, but they obviously have to have training prior to that, from me, on how to properly schedule things in Tidal. So the security and flexibility are valuable features.
They have a lot of pre-set stuff, but you can actually create something like: "Run the third Wednesday of every third month on a blue moon," going to the extreme. Their scheduling functionality is really advanced enough where we can create a lot of different kinds of customizations, based not only on a regular calendar year, but on fiscal calendars and regional calendars. We have jobs that process files for our EU operation and when they have a bank holiday over there we don't need to run the job. We can tie up those jobs that don't need to run on their local, European bank holidays.
The solution also enables admins and users to see the information that is relevant to them. The admins have super-user access, so they can actually adjust and transport different jobs from test to prod. Whereas the operators can adjust a job that's already scheduled if they need to, based on direction from support. They can change this variable, or change this setting, or change this text. But they don't have the access to actually change the master copy of that job. So, a one-off change is literally just that, a one-off change of the next compile scheduled. Otherwise, it's going to run as it's normally set up.
Another good thing that Tidal has is in regard to the history retention of job failures. Whereas our SAP ERP system usually has an eight-day history retention for jobs, Tidal can actually go back longer than that. So if somebody says, "Hey, why did this job fail three weeks ago?" we can bring up the failure message, which is something they can't do directly in SAP.
For the most part, the drill-down and the logging are really good. But if we take an Informatica job, for example: We have the ability, and the operators have the ability, to actually drill down and see, at a session level, where the failure is. There is, unfortunately, no way to extract that into an actual output email or failure email. It's not that that information is not available, but extracting it into an email would be a nice-to-have. It's minor, but it would definitely be a help. In the grand scheme of things though, you can drill down to session-level failures and get that error message to provide to support.
Another thing has to do with job events. A job event triggers when a job completes. It sends an email or reruns a job. Right now — and I've even talked to Tidal about this — it will run all the events at the same time. It doesn't provide the logic to say, "I want this job to rerun five times. If it fails on the fifth time, then send an email: 'Out for Failure.'"
The only other thing I would like to see is an easy way to flag jobs running longer than a certain percentage of the estimated time they should take. Right now, you can hard code in a max expected run-time and you can trigger a notification off of that. The unfortunate thing is, in a consumer product-related business such as ours, Q3 and Q4 jobs are going to run longer. So you can't really put a hard-coded expected run-time, because that's going to fluctuate. So it would be useful if we could specify something like "Flag this job if it runs 25 percent longer than estimated," which the solution does track for 30 or 35 days. That's what they usually recommend, out-of-the-box, for keeping track of history.
I have been using Tidal for about 13 years. I used it for about eight years at my previous company and then I came over to this company.
I came on about four-and-a-half years ago here and Tidal has been really solid. The high-availability and the fault monitoring they use is very good. I can think of twice, in the last four-and-a-half years where we've actually had to failover for one reason or another. And the bottom line was that it wasn't even a Tidal issue; it was something to do with patching. One of the patches from Microsoft was a little funky. From a stability and support standpoint, this is a rock-solid app, in my opinion.
It's very stable, especially for those who utilize what they call Fault Monitor or Fault Tolerance. When we do patching, the jobs, in and of themselves, automatically fail over from our primary to our backup. There might be a slight disconnect in the web UI that the operators use, but that maybe lasts a minute because of the cut-over time. But it picks up all of the backend PIDs, and the jobs just pick up where they left off. From a stability standpoint, this is a really good product.
From what I've seen, the scalability is very good. There are companies that I know that run millions of jobs a day. I've been through some user groups that have some people running nine different instances of Tidal, and they're running a lot of different things. So, the 7.7 million a year we run here, coming from where I was beforehand where we were running about 400,000 a year, seems like a lot. But we're still a small fish in the barrel compared to how other Tidal customers are using it.
So the scalability is phenomenal. We're always looking for that next hook and working on trying to tie into other things. We're keeping our versions updated as much as we can, in regard to OS compatibility. Take Informatica, as an example: We're making sure that we're as up-to-date as we can be with the versions that are out on the market.
In my previous company we used the Lawson ERP's internal job scheduler. There were Windows tasks that we had to check on. They were running a lot of VB6 stuff. In my current company, I came onboard years after they had already cut over to Tidal. I know they had some mainframe stuff in the past, but I don't think they converted from something like CA to Tidal. Tidal was their first choice.
I came in at the tail end of the initial setup when I first started with Tidal back in '07. The decision had been made on the application before I got the position of scheduler in the Tidal admin. In terms of the actual setup, I was on the periphery. Once it was set up, I got more involved. But I have been involved since then with the system upgrades and version upgrades.
Upgrades seem to be fairly straightforward. When it comes to hotfixes and partial, mid-version updates, it's pretty simple. You don't have to call the vendor in. When it comes to versioning upgrades, like when we'll go from 6.3 to 6.5 in a couple of years, we do utilize a third-party vendor to come in and assist, because they do a lot of backend database cleanup and scrubbing. We're running in a SQL database for Tidal, and I know just enough SQL to get me in trouble. So we do rely, especially because this is such an enterprise-based application here, on having a third-party come in and take over the upgrade part of it. We work in conjunction with them, making sure jobs are set and that the copies are good.
As for the learning curve, a lot of it depends on the individual's knowledge of the particular systems. Windows is fairly straightforward. If you know some Unix commands, you can help set them up really easily within the application, when you're setting up a job to run from the Unix command line. If you don't know SAP or whatever the ERP system of the company is, at least a little bit — enough so that you can navigate through it — there might be a little bit of a learning curve. But it's really not as big as one might think. Take the SAP ERP as an example. I came from a Lawson background. I came into the SAP environment here, which I was totally unfamiliar with. But within about a month, I was able to set up SAP jobs without an issue.
There are some little things involved in understanding how to up jobs if you want to overwrite certain variant settings. Learning to do that, and making people feel comfortable doing that, was probably the biggest learning curve.
The other thing is understanding using API hooks within Tidal to other processes. That's one thing they could improve on as far as their training materials go. I've talked about that with them during the past couple of user calls that I've been involved in. At this point it's still a little rough, but hopefully that will get better as time goes on.
The amount of training a new user needs in Tidal depends on the level they're at. We have a training program in place for our operators who do a lot of the manual reporting and failures, running jobs on request, etc. We'll start them with just an inquiry only so they can see everything that's happening, but they can't act on it. That way they can get a feel for the application. We'll give them that for about a week or so, and they'll work hand-in-hand with an operator who's been onsite and using the application. Then we can roll them out to a test version with test-operator access, for another week or so. By that time, they're through four weeks of Tidal acclimation and they're good to go with everything. Because of the operator's schedule — they work a four-on, three-off rotation, it's not like they're working five eight-hour days of straight Tidal — plus all the other things that are on their plate for their job requirements, they're not going to see every single potential issue that could come up. But they have a pretty good grasp at the end of that time.
We'll usually get a feel from not only the trainee, but also the person who is working with them, about how they are doing and if they feel that they're ready to start doing stuff in production. Generally, within a month, they're up and running as an operator, in both test and prod environments.
Developers are a different story because of all the different things that they have access to regarding scheduling and building schedules. We haven't brought on a lot of developers since I've been here. It would probably take a good two to three weeks for developer training, if someone wanted to know how to set up a job in Tidal. We'd really try to hand-feed them little things, so they don't inadvertently schedule a job, or an entire job group that runs hundreds of jobs, which could really bog things down from a systems standpoint.
The partner we use is a Tidal partner called BLUEHOUSE. They've always been very helpful and very flexible in terms of scheduling. The way we do it here is we'll have them come onsite to update our test system. We'll bring that up online and run that on the new version for two months or so. Then they'll come back and we'll do the production update. The whole time onsite, between test and prod together, is about four or five days. But they do a lot of the prep work for production, while we're doing the test upgrade. When we're ready to go to the production, they're only here for a day or a day-and-a-half at the most for the production cut-over. When it comes to initial support right after the fact, they're very receptive to fielding the questions.
I would say we have seen a return on investment by going with Tidal, and not only because of the volume of jobs we're running, but because of the variation of jobs that we're running. It gives us the ability to manually adjust processes on-the-fly, and having that visibility and quick reaction to failures has been a big plus for us.
At my previous company they looked at IBM, CA, and one other solution. The reason my old company went with Tidal back then, was that it was the only one that offered integration with Lawson.
As with any product you're looking at, first of all, don't get pigeonholed into it. Don't have a laser-focus on an individual product. But with Tidal, especially now that they're rebuilding the customer base, reach out and work with their salespeople, and network with current users. One thing I found, especially being on some of the network boards — they used to have a Yahoo Group for Tidal — people aren't afraid to say, "Hey, this works great and this doesn't." I'll be the first to tell you what works great and what still needs some work. And now that Tidal has put its own forum together, the company is monitoring and responding to concerns and questions a lot quicker than they used to when they were under Cisco's umbrella.
The biggest lesson I've learned from using Tidal is that it's always growing. In user calls that we've had since Tidal went back to its own environment, they're really looking to rebuild and invest in the application, and make sure that things are up to date and validated. They're working on making sure they're as current as they can be with certain connections.
It's like they have a renewed vision since Tidal was divested from Cisco. They seem to have a real yearning to get back into the way things used to be in the pre-Cisco days. I'm not trying to knock Cisco, but it is what it is, because I worked with Tidal before Cisco acquired the product. Now with the STA Group and a lot of the older Tidal developers and folks "back in the saddle," there seems to be a renewed interest in rebuilding, making it a lot easier, and opening up a lot more process availability for users and customers.
We've got a handful of developers, five or six people, who actually have the ability to create jobs in our test system. We have a team of six operators who have access to Tidal as well. They do the 24-hour monitoring and ad hoc jobs, etc. And we have two Tidal admins. We do have some other folks who have inquiry access into our production system. We'll give people who might be developers in our test system view-only access to prod. Overall we have 15 to 20 people who have access to the system, with varying security levels. I'm responsible for maintenance, upgrades, job migration, and production. I also work with people who don't have access to Tidal and on helping them get jobs set up properly. I also make sure we get the email notifications correct.
For what we're using it for, and what we have, it's very good.
