Matillion ETL Scalability

it_user628059
Senior Engineer, Big Data/Data-Warehousing at a manufacturing company with 501-1,000 employees
There are no issues with scalability if one strictly does all transformations in-database, using Redshift’s DDL/SQL. All the ‘heavy-lifting’ is done by Redshift, as it is MPP. Simply adding more nodes deals with scalability. It is worth noting that Matillion does not cost more if you add more Redshift nodes. If one uses Python components (as opposed to UDFs), one may encounter scalability issues. The CPU utilization in WatchTower, of Matillion’s single EC2 (it is not, itself, MPP), will peak. Therefore, it is best to keep a close watch over what your data engineers are doing with Python components. View full review »
Harpreet Singh
Director of Data Architecture at a healthcare company with 51-200 employees
The size of our environment is not big. We started six months back. Right now, we only use one node, which is moving the data onto our data warehouse in Snowflake. This node is also very small at this time. Eventually, we will grow big and quickly, because we just had our three drugs approved. Therefore, a lot of data is going to come over. We will be moving this over to our data warehouse, which will need to increase significantly. View full review »
SeniorIn7f6d
Senior Infrastructure Engineer at a tech services company with 51-200 employees
It scales okay. It takes up a good amount of computing resources while it is running, but we wouldn't have too much trouble giving it a little more if it needed it. We have a lot of data, therefore it is hard for me to guess how much we send through it. View full review »
Neel Vora
Application Developer at John Deere & Company
It can scale to a great extent. It can handle the load that we are putting on it, which is about 5TBs. View full review »
Luis Henriquez
Hewlett Packard Enterprise Solution Architect at a tech services company with 11-50 employees
It can scale up/down Snowflake warehouses. My client’s size is a mid-sized environment. View full review »
Engineer2076
Lead Software Engineer at a tech vendor with 10,001+ employees
We have about 4000 employees. View full review »

Sign Up with Email