Duration: 1 year Contract

Job Details

● Develop Test Strategy: Create a comprehensive test plan for the Lakehouse, focusing on Data Integrity, Accuracy, and Consistency.

● Automate Validation: Replace manual spot-checking with automated Python test suites that run as part of the CI/CD pipeline.

● Defect Analysis: Identify and document data anomalies, working closely with Data Engineers to perform root-cause analysis on Spark job failures.

● Regression Testing: Ensure that new PySpark code deployments do not impact existing Gold layer business logic or dashboard reporting.

Experience Requirements

● Total QA/Testing Experience: 5+ years.

● Data Testing Experience: 3+ years specifically in Big Data, Hadoop, or Cloud Data Warehouse environments.

● Databricks Experience: 1+ years of experience testing pipelines within a Databricks environment.

● Automation Focus: Proven track record of moving from manual SQL...