IBM InfoSphere DataStage Review

Its parallel processing capability allows you to go through extremely large data sets in no time at all

What is our primary use case?

Complex data integration projects which require integration from multiple data sources.

How has it helped my organization?

I have worked during many implementations using DataStage. All of the projects that I worked on have been successful. This is due mainly to the strict discipline around best practices, and by following a set of standards and templates designed to reduce complexity and improve automation, including strong reference architecture.

What is most valuable?

  • Its parallel processing capability allows you to go through extremely large data sets in no time at all, if you do your job right. 
  • Highly customizable: Allowing you to handle multiple data latencies (scheduled batch, on-demand, and real-time) in the same job. 
  • High scalability: Start small and go big with the same job. You just need to adjust the configuration file, no need to recompile.
  • Strong metadata management: Business, technical, and process metadata can all be managed from a single place.
  • Ease of integration with other tool sets: Easily supports APIs (or build your own) to support data streaming (or batched) from other systems.
  • Data Quality Management from within the tool: Supporting data sampling, including profiling of data, directly from the development canvas.

What needs improvement?

High-cost of ownership: They could take a page from open source software, such as Talend.

Working with some of the big data components is good, but I can see improvements are needed, such as native support for Spark and HBase.

For how long have I used the solution?

More than five years.

What do I think about the stability of the solution?

No issues.

What do I think about the scalability of the solution?

No issues.

How are customer service and technical support?

Support is always good.

Which solution did I use previously and why did I switch?

Have used quite a few ETL tools in my job.

  • Ab Initio: Even pricier, but has a highly competent ETL tool. It is complete, but hard to use. 
  • Informatica: Not as flexible and does not support the same level of complexity in its maps.
  • Talend: It is a good tool suite, extensive, but can be cumbersome to cite all its pieces.
  • ODI: For the Oracle centric world.
  • SSIS: Week when compared to any of the above tool sets.

How was the initial setup?

Depends on type of environment that is being installed. I have seen fairly simple to overly complex initial setups due to the environment, not due to the tool.

What about the implementation team?

Both vendor and in-house team implementations:

IBM has top-notch support and tool services along with other partners as well. Depending on the partner, this can go from installation and configuration to solution development, etc.)

Most in-house teams that I have seen tend to have have good developers, but not always good architects. Like most every data integration project, if you do not have a strong architecture, your solution will eventually fail.

What was our ROI?

Depends on the project.

Which other solutions did I evaluate?

Have done many ETL tool evaluations based on client requirements. DataStage has always been in the top-three. It may not have been selected due to different weights being used for different sections of the evaluation for different clients, but it has always been in the top-three consistently.

What other advice do I have?

If you have the budget and your solution requires industrial/enterprise strength data integration, this product is always a good choice.

**Disclosure: I am a real user, and this review is based on my own experience and opinions.
Add a Comment