What is our primary use case?
We use multiple models here because we do full development. What we deploy on MySQL is from the Helm chart or it's a Dockerized deployment of MySQL. So we're using the Helm stable chart right now. That's sort of the easiest way to deploy it - to say just one command and it bootstraps your whole database within your classical means or cluster. You can do it locally with mini-crews or developers, for organizational use, or Kubernetes. It's single-node Kubernetes.
Also, you can just deploy MySQL locally with a Helm chart. Regarding production, we have a kind of automated process which is similar to what Spinnaker deploys, with a Helm chart as well as within the cluster. Some other solutions we don't run within the cluster, we use the Cloud version of the database which is Cloud SQL, Google Cloud, and AWS. Those are fully managed ones, of which there are two versions. We have our self-managed version which we run locally and with our DEV cluster and then there is production, as well.
We also use a self-managed version since every cloud provider offers MySQL, even AWS. It depends on the client's needs, how flexible the client is, and also how comfortable they are with MySQL.
We either go with our managed version or the Cloud version. Both are supported because today the Mica server that's actually accessing the database or the piece of software just needs a connection string, it doesn't care if it's running within the Synchronous Cloud. If it's running somewhere else in the Cloud, it's still a private connection on the same network.
So the only differences here are in terms of money costs and whether it's managed or not managed. So for local development, you don't want to have a managed database in the Cloud. You don't need to be tethered to the Cloud, you'd rather just deploy locally. And because we have the same deployment scripts that run locally in DEV and testing, we use the same Helm chart and the same Docker version with MySQL to distribute that through our DEV environment to test the bills and run the test and there is a full QA environment for teaching, as well.
What is most valuable?
I treat these products kind of as a throwaway versus what a DBA would be. From an organizational point of view, it's difficult to actually define the most valuable features because we have so many different databases. For some of them, Postgres for example, which uses MySQL, is just personal preference with is no real difference. Unless you get to really high volumes or through-putting. So in our case, because of legacy reasons, we started using MySQL, which was a popular database. It's not a bad database, so there was no reason for us to change it to the Postgres. We do use Postgres right now for other tools, but for a main database for application purposes, we still stick to MySQL. And I think it's just because of legacy, there's no real advantage of one over the other right now.
We built up the scripts already. Because once you start integrating the scripts into your company, your deployment scripts, test scripts, etc. you just leave it over time because it takes effort to change. But in terms of advantages or features, you can go feature by feature with any database and if you add up the totals, there's no real advantage here between Postgres or MySQL.
What needs improvement?
As for what can be improved, right now we don't use the MySQL cluster. There is a MySQL cluster that you can run in a standalone mode, like a single database or you can do it in a cluster master-slave implementation. The cluster is not the best when it comes to MySQL. That's why we switched to MariaDB. For that simple reason that the cluster there is better. It's more manageable and it's easier to work with.
We decide what to use depending on the needs. For example, if we need to mount something in a cluster mode, we use MariaDB, which again, is a Dockerized solution with a Helm chart as well, and it's very easy for us to deploy and manage, and also to scale when you just increase the number of slave versions. So MySQL doesn't have that great support when it comes to clusters. You can definitely use MySQL for that too, both support clustering, but the MariaDB is better.
Additional features that I would like to see included in the next release of this solution include better support for backups. Because if you go with the MySQL Percona version, it gives you the tools to back it up securely. The vanilla version of MySQL doesn't have that. It actually does have it, but it is just really poorly executed. I would improve the backup system as well as the encryption. To make it smoother right now takes too much work. It should be a little bit smoother to backup the encrypted data the way you want it and have the ability to push it anywhere you want. That is not part of it right now.
Now it is a database, so you don't know what you're going to do with it. It's difficult. You're just going to come up with solutions. But I think you can generalize here and come up with really simple solutions, which we have already in MySQL. That's probably the one thing that I would try and push right now for people to switch. But people are still not biting, because if you go with the managed version, then all the backups are taken care of for you by Amazon or Google or Microsoft. Then you really don't care. But for us, since we're doing it locally, self-hosted, we would like to have better tools for locking up the data.
Right now, one aspect that is also linked to backups is running things in a crosscheck with semi-managed solutions. This requires a bit of a context. Since we're running things within the clustered communities, we're kind of pushing the Cloud into the cluster. We also want to push some of the tools for the database into a cluster, as well. So these are what we call Kubernetes operators. And there's MySQL operators that were first developed by the community. Those kind give you the ability to backup data within the cluster. So now you have a fully managed solution running from your cluster. These are called MySQL Kubernetes operators.
We are looking into those right now to upgrade our solution, which would mean that we can just execute our backup natively within Kubernetes, not via special scripts. This would make it much easier to actually deal with any kind of MySQL issues within the cluster, because it would be cluster-native. That's what the operators are for.
I think Oracle just created a really good one. It surprised me that they have this. It's not because of Oracle, but they got pushed by the community and actually created the MySQL Operator for Kubernetes, and that's what we're moving towards.
This is going to give you an ability to have a cloud-managed solution within the cluster. And then you can ask the MySQL Operator for the database. They'll partition the database and give it to you. So it will change the nature from you deploying it to you just asking the cluster to give you a database. It's a fully managed solution right from the cluster.
So that's what we're heavily looking into right now. We'll be switching to using Kubernetes MySQL Operators. It's a high-availability cluster running within the Kubernetes cluster.
Right now we're pretty good with that. It's working fine. We're trying to find some time to actually release that globally everywhere. That's where I am right now.
But in terms of technology, if you give up Oracle, you just go to a MySQL operator. That's the one we're using, what we're actually looking at - to create, operate and scale mySQL and sell it within the cluster. This idea of having a cognitive MySQL becomes much easier to manage within the cluster, as well. So you don't have to go with the cloud solution with AWS or Google cloud or Amazon MySQL or the Microsoft version.
The Oracle SuperCluster is the Oracle MySQL operator. That's what we we are looking into a lot right now. Mainly because it does backups on demand - it's so easy to backup. You can just tell Kubernetes to backup and you don't have to run special scripts or special extra software or codes to back it up. You can make the backup as you would do anything else. Send a backup or some other data source or insert an Elasticsearch into it here. Just say "Kubernetes, back it up" and you know Oracle has this adapters within the cluster to back it up for you taking increments or different companies. So that makes it really nice and easy to use and to deploy.
With that kind of solution you can ask to class or petition the database how you want. So again, it changed the nature of the kind of push-to-pull second nature system. Are you pushing your containers to a cluster? You just say cluster, "give me a database" and the class gives you the base partition database, creates a database in a secure manner, gives the connection to the database, and you're done. Then you can back it up on a schedule on to any backup switches. It's much easier. So once this goes, it is going to be widely adopted, which it should be. But I think people might not have the tech skills right now. But once it's adaptive, maybe in a few more months, it's going to be the number one solution for everybody.
In terms of what I'd like to see in the next release, one thing that's always missing is dash boarding. There's no real BI tool for MySQL, like there is in Yellowfin and all the different tools that you get. They all have MySQL connectors, but there's no specific BI tool for MySQL. Open source projects have sprung up, but they're more general purpose, like Postgress, a MySQL kind of database, a relational database. I don't see any really nice tool like Cabana for elastic searches that I can tell clients to use because it would be too technical for them. They would have to have more technical engagement with writing the course, drag and drop, and creating a graph like in Power BI where you just connect with DIA.
So I'd like to see the grab and drag and drop tables, nice beautiful graphics, and pie charts. You don't necessarily have that with MySQL like you have other solutions, which are really cost prohibitive for some clients. It'd be nice to have an open source solution for that. Decent solutions. I mean decent that I can take to clients. It's so technical. They want to drag and drop.
For how long have I used the solution?
I have been using MySQL now for five or six years.
What do I think about the stability of the solution?
This specific version of this MySQL has been battle tested for a long time. Any issues are known issues and we pretty much don't have any problems when they're in production. So it's very stable. When we picked simple switch demand, again, these things came up. But it was quickly resolved. So I guess that's the benefit of going with open source and seeing all those problems ahead of time in source code and having the ability to fix them.
What do I think about the scalability of the solution?
Since we have MySQL specifically, and we have to use it in many different environments, dev, testing, and production. All those different people are using it. Developers, QAs, automated testings running against that. In production we have many different users, so we have different meaningful products that are already running. For example, gotoloans.com. It's a loan application site in Canada that is serving a lot of users daily and is backed by MySQL and Elastic SQL databases. So we're using it for high volume and low volume. We have it in many different projects and many different environments.
We use it in different environments, the production also, and many different products as well.
We do have plans to run everything as a cluster, and probably will slowly switch to MetaDB. That is something we're doing right now. We also have plans to switch it to the managed version as well for production deployments, for the simple reason that we're trying to offload as much as we can from the DevOps people. So if offloading that management database from them will help them, then we'll do it.
Also, there are clients that have preferences when it comes to where the database should be running. For example, one of our major clients wants to run specifically in our database because we built it for them and they're comfortable managing it. You're always more comfortable having a managed version. So if you have a small team with a managed DBA, even though it's more expensive and there's always some issues coming up with it, you can just let Amazon manage it for you, and you don't even have to think about it. You could do the backups and if something happens, they can restore it. And you can scale as much as you want, as well.
In terms of cost, there are different flavors of it. It depends on the solution. Locally, as I said, MySQL is going to stay the way it is right now. We're not going to have a cluster version, because for development we just need a database. You need to have a scalable database or clustered.
So MySQL is going to change. We're in the process of transitioning the production versions to cluster versions for some of the projects because they have more volume. We can see that because of the volume of users, and how many queries they do on a daily basis, they would benefit from having a cluster versus a SQL database. So you can have a master to master cluster, which you can have separately. You can actually manage your read and write separately, and then optimize. So you can give more power to people, to certain queries, spreading across the cluster. So all those sorts of things come with the cluster database. That's going to improve performance.
One of the things that we're doing is looking at the short version of MySQL, which is a new thing. This means a shared database. Elasticsearch is made up of shards. This is a different way of thinking about relational databases like MySQL. Traditionally, MySQL or relational databases, have been crafted by having an instance of equal slave to equal master. You have many slaves and many masters, as well. Now the sharding makes the database a little bit different, and it's more usable for us in terms of the way we deploy things. So we're looking right now at MySQL sharding as well, and a few of the different flavors of that so that we can scale it horizontally. Instead of actually creating an instance of MySQL, we can actually spread across multiple different shards across many instances.
And it's also cheaper as well. Once you start getting into the shard world, it's really cheaper to deal with some of these issues, like clustering issues. So it's more cost-effective.
How are customer service and technical support?
I have not been in touch with support because any issues that came up, we really just resolve them because it's open-source, so if you look at the code, then you can solve it. There's also lots of community engagement in these databases. There are millions and millions of forums online. So if there's a problem, everybody will be on it trying to fix it. So there are no real major issues here.
How was the initial setup?
The initial setup is straightforward because we're using Docker. So we Dockerize not only our database but the applications, as well, because it's really easy to play as a Docker container, and then tour them to the Kubernetes cluster. It's very easy for us to manage it. And also, we have backups on top of that. So we have a schedule, and a job running, always backing up the system with secure backups. So it's actually very straightforward to get it up and running.
Deployment takes seconds. That's why we can include some of the companies, because we figure a way to do it simply, and you don't have to deal with all the complicated SQL servers, and you can bend a lot into Microsoft.
So for us, deployment is trivial, especially when you use the cloud version. For example, for our database, or cloud SQL, again that's trivial, just a simple deployment. You're up and running within less than a minute, five minutes maximum. Locally, people just deploy those databases every day when they build stuff. Again it takes a few seconds to get that going.
What was our ROI?
Since we’re running it ourselves, it's our own flavor of MySQL, for dev, and QA, staging, production environments, that cost is basically a part of their running between this cluster. So I can't give you a fixed cost, but I can give you the cost of the entire cluster. There are many nodes in a cluster, and there's many different parts continuously running it. So to fully utilize the cluster, we put everything in it and just try to maximum each node.
So you can have a MySQL database beside a Java Microservice and Angular applications on the same node, and using the same kind of resources. So it would be difficult for me to kind of break it down. Obviously I'll do a deep dive, and I'll look at it, in terms of, what percentage of the CPU is being used by MySQL.
Now when it comes to the Cloud versions, obviously there's a fixed cost with that. So for example, one of the clients uses our database, they chose to go with the extra large version of the ECQ's, and there's a price for that. And you can just get a price quickly, and there's a whole chart of pricing there.
So that's based on clients and their comfort level. We can tell them exactly what performance we're requiring here, and then say, what is the minimal thing we need here, in terms of CPU resources and connections? So that's what you really need for just a cloud version of it. Once we define that, then we tell the client, this is what you really need. You can get away with a smaller version of the virtual machine by using something bigger. To be comfortable they decide to do it. So I'm dealing with the pricing, and the pricing is transparent.
I have all the separate pricing for the databases as well. And from that, you can figure out what the cost is.
There's no licensing fees here because it's open source. So the only fees are really just for using the Cloud resources, even if you go with managed or non-managed, you're still using the Cloud resources. You can be more frugal if you're running it yourself, versus what Google or Amazon will do for you. It'll be a little more pricey to go with them, but because it's a managed solution, you do have that peace of mind, because they're managing it for you. You just connect with it and just talk with it.
But in our cases, we deploy it, we manage it, we back it up, we do all that stuff. So there's more work that we have to do, but a lot of time we eat up the cost because it's not an expensive thing to do. So it can be more cost effective running within the Coud, than in a non-managed version, self-hosted version.
At the end of the day, Google and Amazon are still making money, because it doesn't matter if you're running it yourself or it's managed, it's still using the Cloud. It's the same CPU and same RAM.
What's my experience with pricing, setup cost, and licensing?
So we jumped from version 5.6 to 5.7. That's not the latest version. The latest version is 5.8. We didn't move to eight for the simple reason that there's lots of code-based on 5.7 and there's no incentive for us to change right now. So a lot in the industry have not migrated to version eight yet. Oracle is having difficulty committing people to actually go with that version right now.
MySQL has been battle-tested for years and years. So people were comfortable from 5.6 to 5.7. It wasn't just a minor change, it was actually a major change in terms of the databases. Now, once Oracle started managing MySQL, they didn't do a good enough job. That's when MariaDB was invented when they jumped from version five to eight.
There wasn't enough confidence in that. Because there's so much time invested in it. Because MySQL is not just MySQL, they give it in a cluster mode, when you have huge databases with lots of master-slave nodes. So it's just not a trigger for a DBA to move to a new version that hasn't been battle-tested like their 5.7.
So 5.7 is a good database. That's 1418 right now or something like that. I think that's the one we use in production. So for most DBAs it's difficult for them to change. Also with Google and Amazon, you can choose not to go back for 5.7. It is very easy to create a fully scalable solution with 5.7. So, there's no incentive for people to actually switch.
What other advice do I have?
The biggest lesson I would tell others is regarding the backups. Once you start doing it yourself, backing up becomes a thing. When we sign up the clients, we'll give them a set amount of backups daily and we always give them a little verbiage about how much data can be lost if the thing goes down.
Or for example, if you get hit somewhere, what is the last backup you did? How much are you willing to lose? Backups can become quite complicated, and that's something that you have to manage yourself. We have to come up with clever solutions to do runs within our Dockerized environments in production, which you usually don't get from the community. So we have to do it ourselves.
That became a thing quickly once we started going. But that was years ago. We resolved these issues on the way and we are still making them better over time - how we back up the data, the business, the compliance, where did the issue live, who should have access to that? All that stuff.
So backups are usually the thing that people don't think about. And that can bite you in the ass kind of quickly.
On a scale of one to ten, I'd probably give MySQL a seven. There's definitely room for improvement here in terms of tools that come with the product, the way we deploy it, and the way we back it up. In essence, it's a good beginner base. It's just, the tooling around the database needs a little bit more work. You just need to be fair because it is a good database. It's also an open-source database. You know you can get commercial products that Percona for a commercial version of MySQL or Aurora database MySQL. So if you go with that, then you would probably give a much higher score because you really don't see it at all. It's just close your eyes and click a button and it's there. You don't have to touch it at all.
For us, since we deal with it every day and try to compete with the companies, the small DevOps team tries to be as efficient as they can, and sometimes you have to build too many things around the solution.
The commercial products only have that because they put 20 to 30 people on JSON and they can give it to you faster. That's what Google can do because they're good at the tooling around the database. In the current requests of the work, MySQL Workbench is the default tool to interact with the database. Again, MySQL Workbench is an open-source tool that it gets directly from Oracle. It's okay. It's not the greatest. It gets the job done. It's not a finesse tool. It just gets the job done.
If you hide it behind a main service and you don't see it, it's great. You're good to go. People talk about Amazon RDS and how great it is. But that's a managed product. If you peel the layers and look at the SQL in there, they put a lot of work around that. It's fully scalable. The money used and the way they restructured that SQL database to actually give you that performance took a lot of work for the AWS people. So they're not going to share that IP with you. And they're definitely not going to release it because other people can pick it up, like Google. Then Google has Cloud SQL, as well. So they also have a MySQL version in there and they don't show you how the backup is, or how they actually manage it or scale it. You don't get that information.
So that's the trade-off between managed and non-managed or self-hosting. It's always that kind of battle, that fight. It depends on the money, depends on the client. If it's for a healthcare issue or one of the hospitals, you just have to decide what they want, what's the best for them and how they're going to be protected. So there are many variables that come into play. It depends on your use case. In general, it’s a good database, I have no problem with it.