Developed automated data validation system in MS SQL

it_user358593 - PeerSpot reviewer
1 month project

Project Description

Developed automated data validation system in MS SQL Server.  The process imported more than 60 Excel spreadsheets worth of pass/fail SQL checks using SSIS into a validation table for every column in every table in each stage of the warehouse.  Each spreadsheet was then dynamically wrapped in a T-SQL procedure that could be called ad hoc or scheduled individually or as a group of checks.  The project required the procedures store the SQL body inside the procedure for rapid editing by developers with no more than basic SQL experience.  

On execution each check would populate a master table with a pass/fail record and on failure would track a detail record for each value that failed the check.  Hard failures due to syntax errors or other issues were then written to an error log that trapped the line number, error code, message, and failed check for rapid investigation.

Highlights

Ahead of schedule
Received recognition / award

Difficulties

Steep learning curve

Products Used

Technical Skills Used

  • T-SQL
  • Entity Relationship Diagramming
  • Madison (WI-US)43.0731-89.4012