What is most valuable?
ARULES() function is the fastest implementation of the associations algorithm (a priori or tree) I have worked with. In comparison with the R associations package, it is +1000x faster. The issue with the R implementation, and R in general, is that the functions do not work on large sets of data. R would generate a 100,000 x 100,000 element matrix for some of the arules runs we do – very, very inefficient programming.
How has it helped my organization?
Any in db machine, learning functions help by eliminating the data transport issue. NZA goes one better by allowing you to utilize a tuned engine (software + hardware) to run the functions. The result is that we can run algorithms that would be impossible to run on other platforms.
What needs improvement?
We are using an older version. Some improvements are already implemented, such as the addition of neural networks. Other improvement would be to make the existing algorithms even faster, e.g. kmeans can actually be parallelized. If this reworking of the algorithm is done, it should leverage the MPP architecture of Netzza and run much faster.
Also, some algorithms are faster than almost any other method or system e.g. NZA.ARULES() function. Whereas others are not as fast and could probably be optimized for an MPP architecture like kmeans function.
For how long have I used the solution?
2-3 years on and off. 1 year for actual integration into another tool for mass consumption.
What was my experience with deployment of the solution?
For deployment, ensure you follow the instructions. The version of Netezza must be matched with the version of NZA. We tried to install a newer version of NZA not supported by the version of Netezza we had and it did not work.
How are customer service and technical support?
N/A. We did not engage IBM tech support, mainly due to past experiences where it is faster, easier and much less stressful to just figure things out yourself.
Which solution did I use previously and why did I switch?
Rapidminer, R, DB2 intelligent miner, SSAS data mining were all tested for association rules. Rapidminer and R did not pass. They use matrix math, and cause +100x performance degradations. DB2 intelligent miner, SSAS data mining, and likely also NZA follows the original a prori methods of using sets, not matrices.
How was the initial setup?
DBA set it up, but it seemed very simple. I helped with designing the overall architecture. If you stick with NZA as a “function repository DB” concept and do not put data on it (as recommended in the install manual) you will be ok.
What about the implementation team?
In-house. Don’t over think it. Just apply some base architectural principals and give it a try. If you find something not working make adjustments.
What's my experience with pricing, setup cost, and licensing?
I had nNo visibility to pricing. Netezza in general though, is very cost effective. If you compare to other MPP platforms like Hadoop. Hadoop requires a very “non-enterprise” culture to implement and a team of designers / engineers. Netezza requires maybe 1-2 admins.
What other advice do I have?
I would have to understand the specifics of the organization, the data size, the tasks, etc. to make any recommendations.