Melissa Data Quality Scalability
GM
GaryM
Data Architect at World Vision
We can run 9 million customer record exact matches in 10 minutes using 5 partitions/parallel dataflows. Survivorship takes another 50 minutes. I'm sure you could run faster with dedicated hardware and running more parallel dataflows. The tool starts to exponentially slow down once you pass about 2 million customers in a single dataflow so its best to keep it at or under that number although mileage will vary depending on the complexity of your matching. Its unfortunate that the vendor hasn't built in parallelism which would both eliminate the need to do this yourself. They should be able to auto-scale it based on # of CPU's your running.
Even with that limitation this tool is magnitudes faster than the last matching tool I used and it wasn't a simple plug-in to an ETL tool. I recently heard of a competing tool that takes longer to match just a few thousand customers than this tool takes to run millions of them.
Note:
We probably run higher volumes than many organizations. For B2B and daily matching you could probably process a delta in a matter of a few minutes with this tool.
Note: I suspect an essential ingredient when considering scalability is whether you're calling a web service for matching or just on-prem. Their SSIS component is only on-prem but they offer a web service as well which we have not tested.
Combining survivorship and matching in the same data flow slows performance. We got much better performance by running in two separate dataflows - the first for just matching and then another for just survivorship (re-using the previous grouping numbers in the first match) to make it perform to our requirements.
SV
Sam Varadarajan
Powerbuilder Consultant at a government with 10,001+ employees
No issues. In fact, we have increased our request volume in the last three years and they have been able to accommodate us easily and smoothly.
No issues.
View full review »Buyer's Guide
Data Quality
March 2024
Find out what your peers are saying about Melissa, Informatica, Experian and others in Data Quality. Updated: March 2024.
765,234 professionals have used our research since 2012.
We had found a bug that only appeared when trying to match over 94,000 records. The process hung and we could not identify without a lot of testing what was going on. Their support team worked with me to determine the issue.
View full review »DP
ITDirect78f0
IT Director
No issues. We do a very high volume of traffic in a condensed window during the September-October-November timeframe, and we have never had an issue with performance or scalability.
View full review »No scalability issues at all.
View full review »GG
Gavin Gibson
COO
No scalability issues.
View full review »No issues with scalability.
View full review »I haven't seen any problems yet. I do have to go through White Pages to try and get information about an owner, which is a different set. I'd love to see that incorporated in Melissa Data.
View full review »Never tried.
View full review »RW
Robert Wakefield
CTO at a comms service provider with 11-50 employees
Have not tested, but sure the calls can be scaled.
View full review »No issues.
View full review »Larger batches sometimes have failed batches.
We didn't encounter issues
View full review »Not applicable.
View full review »It depends on whether you can run extremely large lists on multiple servers. Any sort of dedupe or fuzzy match is processing intensive, regardless of the vendor.
View full review »No issues.
View full review »No scalability issues.
View full review »It has good capability. It probably has about as good a capability as what it's ever going to have.
No scalability issues.
View full review »No issues.
View full review »No issues.
View full review »No issues.
View full review »No issues.
View full review »It has been able to scale for our needs.
View full review »No issues.
View full review »No issues.
View full review »No scalability issues.
View full review »WK
Wendy Kennah
Director at a tech services company with 1-10 employees
No issues.
View full review »We did not encounter any issues.
No issues.
View full review »No issues.
View full review »No issues.
View full review »MatchUp seems to be single threaded, and limits the amount of data that can be processed automatically.
View full review »Buyer's Guide
Data Quality
March 2024
Find out what your peers are saying about Melissa, Informatica, Experian and others in Data Quality. Updated: March 2024.
765,234 professionals have used our research since 2012.