Job Summary:
ClifyX is a company seeking an Azure Data Engineer with expertise in Databricks. The role involves designing, building, and orchestrating ETL/ELT pipelines, ensuring data quality and governance, and leading an offshore development team.
Responsibilities:
โข Act as the onsite lead for Azure Databricks data engineering initiatives.
โข Supervise and guide an offshore development team, assigning work, reviewing deliverables, and ensuring adherence to best practices.
โข Perform code reviews, design reviews, and solution walkthroughs for offshore-developed components.
โข Ensure alignment between offshore execution and onsite architecture, security, and business requirements.
โข Proactively identify risks, gaps, and improvement opportunities across the data platform.
โข Prior experience working in a hybrid onsiteโoffshore delivery model.
Qualifications:
Required:
โข 6+ Years of experience in data engineering.
โข Design, build, and orchestrate ETL/ELT pipelines using Azure Databricks.
โข Implement batch data ingestion and transformations using PySpark and Spark SQL.
โข Architect and maintain Lakehouse and analytical warehouse models (fact and dimension schemas) leveraging Delta Lake.
โข Ensure data quality, reliability, lineage, and governance across the data platform.
โข Collaborate with security and platform teams to enforce data access controls and best practices.
โข Experience on CI/CD pipelines using Git for Databricks.
โข Act as the onsite lead for Azure Databricks data engineering initiatives.
โข Supervise and guide an offshore development team, assigning work, reviewing deliverables, and ensuring adherence to best practices.
โข Perform code reviews, design reviews, and solution walkthroughs for offshore-developed components.
โข Ensure alignment between offshore execution and onsite architecture, security, and business requirements.
โข Proactively identify risks, gaps, and improvement opportunities across the data platform.
โข Prior experience working in a hybrid onsiteโoffshore delivery model.
โข Strong, proven expertise in Azure Databricks and Delta Lake.
โข Advanced proficiency in SQL with a solid understanding of distributed data processing concepts.
โข Demonstrated experience leading, mentoring, or supervising data engineering teams.
โข Strong experience performing code reviews and solution design validations.
โข Knowledge of data security, and access management (RBAC).
Preferred:
โข Experience with Azure Synapse / Fabric.
โข Experience with Azure Data Factory.
Company:
ClifyX provides innovative business solutions which satisfy requirements for mission-critical reliability, scalability, interoperations. Founded in 1998, the company is headquartered in South Plainfield, USA, with a team of 501-1000 employees. The company is currently Late Stage.