Associate Data Engineer Jobs in Exton, PA (NOW HIRING)

Job Description e&e is seeking a Lead Data QA for a hybrid contract opportunity in Philadelphia, PA. The Lead Data QA is responsible for defining and driving the overall Data Quality Assurance strategy for enterprise-scale data platforms. This role ensures that all data systems meet rigorous standards for accuracy, performance, integration, security, and compliance.

The Lead Data QA will provide leadership and mentorship to a team of data QA analysts and testers, establish quality frameworks for ETL/ELT pipelines, and integrate automation within Azure Data Factory (ADF), Databricks, and Snowflake environments. The ideal candidate possesses a deep understanding of data engineering, automation frameworks, and regulatory data compliance (HIPAA, CMS) within modern cloud architectures. Responsibilities: Leadership & Strategy Define and own the enterprise Data QA strategy encompassing functional, non-functional, integration, and performance testing.

Lead and mentor a distributed team of Data QA professionals across multiple programs and data initiatives. Establish and maintain data quality SLAs, KPIs, and dashboards for critical datasets. Collaborate with data governance, engineering, and architecture teams to embed QA best practices across the data lifecycle.

Data Testing & Validation Design and implement automated test plans, scripts, and frameworks for ELT/ETL pipelines. Validate complex payer datasets including claims, membership, provider, and clinical data. Conduct FHIR-based API testing for CMS interoperability and compliance standards.

Verify HEDIS measure calculations, healthcare quality metrics, and performance data accuracy. Log and track defects using appropriate QA tools; provide detailed feedback to engineering and architecture teams. Automation Strategy & Framework Develop and implement a data QA automation framework for Databricks (Delta Live Tables, Delta constraints) and ADF pipelines.

Utilize Great Expectations for reusable validation suites integrated into CI/CD workflows. Embed automated schema validation, reconciliation logic, and drift detection into data pipeline operations. CI/CD Integration Develop QA gates and automated quality checks within Azure DevOps pipelines for Databricks Jobs/DLT, SQL metadata, and ADF deployments.

Collaborate with DevOps and Engineering teams to embed QA automation into continuous integration and deployment processes. Technical Delivery Partner with ADF, Databricks, and Snowflake teams to ensure end-to-end data quality. Build and maintain automation frameworks leveraging Python, PySpark, and SQL.

Participate in code reviews, data model validation, and regression testing across environments. Work with business and data governance teams to identify, investigate, and remediate data quality issues. Performance & Compliance Design and execute automated load and stress tests for large-scale pipelines and dataflows.

Ensure all data QA processes align with HIPAA, CMS, and payer industry compliance standards. Support audits through proper documentation of QA processes, test results, and lineage verification. Requirements: Education: Bachelor's or Master's degree in Computer Science, Data Engineering, Information Systems, or a related field.

Experience & Skills: 10+ years of experience in Data QA/Testing, with at least 5 years in a leadership capacity. Strong proficiency with Azure Databricks (Delta Lake, Delta Live Tables, Unity Catalog). Hands-on experience with Azure Data Factory pipelines, monitoring, and CI/CD deployment.

Advanced skills in Python, PySpark, and SQL for test automation. Experience with Great Expectations, Azure DevOps, and data quality automation frameworks. Familiarity with data governance, PII compliance, and enterprise data quality frameworks.

Proven success integrating QA practices into DevOps pipelines within cloud data environments. Excellent communication, leadership, and cross-functional collaboration abilities. Experience in Agile/Scrum environments is a plus.

Preferred Qualifications: Experience with HL7/FHIR data models beyond payer use cases. Knowledge of Lakehouse and medallion architecture Familiarity with BI validation using Power BI or Tableau. Understanding of data governance platforms (e.g., Collibra)

Prior experience designing data QA automation frameworks for pipelines and regression testing. Certifications such as Microsoft Certified: Azure Data Engineer Associate or Databricks Certified Data Engineer.