There's room for improvement in how it handles data streaming capabilities. Our main challenge currently is that Matillion runs on an EC2 instance, limiting us to running only two processes simultaneously at the entry level. This constraint means we can run about sixteen jobs concurrently at the moment. However, once we transition to the SaaS offering, scalability will no longer be an issue. With the SaaS solution, we'll have the flexibility to run as many jobs as needed, making it a natural next step in our progression.
As someone new to the data industry and with limited experience in ETL tools, I'm not familiar with other options. My background was as a university professor until about a year ago, so I'm still getting acquainted with this field. I found some of the more complex aspects of ETL challenging, but I grasped the concepts fairly quickly.
Business Intelligence Manager & Data Analytics (Retail Business) at B.TECH
Real User
Top 10
2023-07-25T06:39:00Z
Jul 25, 2023
The product must enhance its near-real-time data capture feature. Also, if they enhance CDC functionality, we can replace multiple components with one component. Otherwise, we must purchase a different component for CDC functionality and another component for data transformation. This added feature will help the user do everything with a single component.
So the main thing I would like to see improved in Matillion are two things. Firstly, their ability to process concurrent workloads. Right now, the concurrency reaches a stalling point if too many things are added, and it gets stuck waiting for each one to finish. Secondly, Matillion needs an improvement in its backend integration and the way that it pushes things through. It is already good, but it could be cleaner. I will say that I think both of those issues are being addressed in the new platforms that are coming out. Matillion Unlimited Scale is the new answer to concurrent workloads, and Matillion Data Productivity Cloud is their new software-as-a-service version of a Matillion ETL provider, including a deeper git integration. So my concerns are being addressed, but those are the two things that stand out to me the most right now.
When using the SQL loader type there were not a lot of pre-processing features for the data. For example, if there is a table with twenty columns, but we only want to load ten columns. In that case, we can use a security script to select the specific columns needed. However, if we want to perform extensive pre-processing of the data, I faced some challenges with Matillion ETL. I did not encounter many challenges, but my overall experience is limited as I only have three years of experience. The solution could improve by adding support for instructed data types.
Director of IT Operations at Broadridge Financial Solutions, Inc.
Real User
Top 5
2023-02-21T19:25:00Z
Feb 21, 2023
Matillion’s on-premises capabilities don’t allow you to build something customized. I will give an example of tables to explain it. If we want to do a lookup, we have to copy the whole table with three million rows in it, every time. It is not cost-effective for me. We have these three million rows ported over in our S3 bucket. We have to pay for that and Snowflake as well. So, we have told them to build up a custom solution allowing us to bring over the data we care about, using the records and the drivers. But, that wasn’t out-of-the-box. So, the default way to work is to port everything over from on-premises to your AWS environment. For me, it is not cost-effective in the long run.
Data Architect at Old Mutual Life Assurance Company (South Africa) Limited
Real User
Top 20
2022-09-06T14:04:00Z
Sep 6, 2022
I am looking forward to seeing the expansion of the source range for their data loader product. However, I think they have done a very good job of incorporating a lot of different data sources.
Data & Analytics Practitioner (BIDW, Big Data) at Tech Mahindra Limited
Real User
2021-12-27T19:04:00Z
Dec 27, 2021
There are certain functions that are available in other ETL tools which are still not present in Matillion ETL. It would be good to have more features.
Managing Director at a tech services company with 51-200 employees
Real User
2021-02-15T11:57:38Z
Feb 15, 2021
It can have multi-environment support. We should be able to deploy it in different environments. Its integration with SAP connection is not so nice, which should be improved. It can also support an on-prem database.
Director of Data Architecture at a healthcare company with 201-500 employees
Real User
2018-12-11T08:31:00Z
Dec 11, 2018
It is not an end-to-end platform for ETL. It brings in the data. To complete the pipeline, they might want to include some connectors which would put the data into different platforms. This would be helpful. We are working with different platforms. Most of the connectors that we are looking for are included, but sometimes scripting is required. The good thing is we can reach out to support and have them create the custom connector for us. This can probably be improved.
* It could have better integrations with other databases and other services. * I would like to have more customization available. * Going forward, I would like them to add custom jobs, since we still have to run these outside of Matillion.
* Performance can be improved for efficiency, and it can be made faster. * Latency could be reduced. Sometimes, it takes longer to fetch data out of it. There are network issues because we experience a little lag. * While the UI is good, it could be improved in its efficiency and made easier to use. * It can be used by different consumers. So, I would recommend to the company to promote more, because people don't know much about it. If they promote more, they can sell it. They need more marketing.
Senior Engineer, Big Data/Data-Warehousing at a manufacturing company with 501-1,000 employees
Vendor
2017-03-16T10:34:00Z
Mar 16, 2017
Compared to the likes of traditional ETLs, like Informatica, SnapLogic, and Talend, or even raw Python scripts, this product needs no improvement, as it is so much better. Any new product like this has teething problems that get solved pretty quickly in the next release. Better user documentation with more examples would be helpful, especially in areas with run-time parameters or JavaScript inserts.
Matillion ETL is a powerful tool for extracting, transforming, and loading large amounts of data from various sources into cloud data warehouses like Snowflake. Its ability to load data dynamically and efficiently using metadata is a standout feature, as is its open-source ETL with good performance and high efficiency.
The solution has a graphical interface for jobs, is easily adjustable and extensible, and allows for scheduling and error reporting. Matillion ETL has helped...
There's room for improvement in how it handles data streaming capabilities. Our main challenge currently is that Matillion runs on an EC2 instance, limiting us to running only two processes simultaneously at the entry level. This constraint means we can run about sixteen jobs concurrently at the moment. However, once we transition to the SaaS offering, scalability will no longer be an issue. With the SaaS solution, we'll have the flexibility to run as many jobs as needed, making it a natural next step in our progression.
As someone new to the data industry and with limited experience in ETL tools, I'm not familiar with other options. My background was as a university professor until about a year ago, so I'm still getting acquainted with this field. I found some of the more complex aspects of ETL challenging, but I grasped the concepts fairly quickly.
The product must enhance its near-real-time data capture feature. Also, if they enhance CDC functionality, we can replace multiple components with one component. Otherwise, we must purchase a different component for CDC functionality and another component for data transformation. This added feature will help the user do everything with a single component.
So the main thing I would like to see improved in Matillion are two things. Firstly, their ability to process concurrent workloads. Right now, the concurrency reaches a stalling point if too many things are added, and it gets stuck waiting for each one to finish. Secondly, Matillion needs an improvement in its backend integration and the way that it pushes things through. It is already good, but it could be cleaner. I will say that I think both of those issues are being addressed in the new platforms that are coming out. Matillion Unlimited Scale is the new answer to concurrent workloads, and Matillion Data Productivity Cloud is their new software-as-a-service version of a Matillion ETL provider, including a deeper git integration. So my concerns are being addressed, but those are the two things that stand out to me the most right now.
The cost of the solution is high and could be reduced.
When using the SQL loader type there were not a lot of pre-processing features for the data. For example, if there is a table with twenty columns, but we only want to load ten columns. In that case, we can use a security script to select the specific columns needed. However, if we want to perform extensive pre-processing of the data, I faced some challenges with Matillion ETL. I did not encounter many challenges, but my overall experience is limited as I only have three years of experience. The solution could improve by adding support for instructed data types.
Matillion’s on-premises capabilities don’t allow you to build something customized. I will give an example of tables to explain it. If we want to do a lookup, we have to copy the whole table with three million rows in it, every time. It is not cost-effective for me. We have these three million rows ported over in our S3 bucket. We have to pay for that and Snowflake as well. So, we have told them to build up a custom solution allowing us to bring over the data we care about, using the records and the drivers. But, that wasn’t out-of-the-box. So, the default way to work is to port everything over from on-premises to your AWS environment. For me, it is not cost-effective in the long run.
I am looking forward to seeing the expansion of the source range for their data loader product. However, I think they have done a very good job of incorporating a lot of different data sources.
There are certain functions that are available in other ETL tools which are still not present in Matillion ETL. It would be good to have more features.
It can have multi-environment support. We should be able to deploy it in different environments. Its integration with SAP connection is not so nice, which should be improved. It can also support an on-prem database.
Ideally, I would like it to integrate with Secrets Manager as well as the AWS.
It needs integration with more data sources. I would also like a better UI because it is complicated to manage it.
It is not an end-to-end platform for ETL. It brings in the data. To complete the pipeline, they might want to include some connectors which would put the data into different platforms. This would be helpful. We are working with different platforms. Most of the connectors that we are looking for are included, but sometimes scripting is required. The good thing is we can reach out to support and have them create the custom connector for us. This can probably be improved.
* It could have better integrations with other databases and other services. * I would like to have more customization available. * Going forward, I would like them to add custom jobs, since we still have to run these outside of Matillion.
* Performance can be improved for efficiency, and it can be made faster. * Latency could be reduced. Sometimes, it takes longer to fetch data out of it. There are network issues because we experience a little lag. * While the UI is good, it could be improved in its efficiency and made easier to use. * It can be used by different consumers. So, I would recommend to the company to promote more, because people don't know much about it. If they promote more, they can sell it. They need more marketing.
In the next release, we would like to have connections to more databases.
Compared to the likes of traditional ETLs, like Informatica, SnapLogic, and Talend, or even raw Python scripts, this product needs no improvement, as it is so much better. Any new product like this has teething problems that get solved pretty quickly in the next release. Better user documentation with more examples would be helpful, especially in areas with run-time parameters or JavaScript inserts.