Job Summary:
Tata Consultancy Services is seeking an Azure Data Engineer specializing in Databricks. The role involves designing, building, and orchestrating ETL/ELT pipelines, ensuring data quality, and leading an offshore development team.
Responsibilities:
• Act as the onsite lead for Azure Databricks data engineering initiatives.
• Supervise and guide an offshore development team, assigning work, reviewing deliverables, and ensuring adherence to best practices.
• Perform code reviews, design reviews, and solution walkthroughs for offshore-developed components.
• Ensure alignment between offshore execution and onsite architecture, security, and business requirements.
• Proactively identify risks, gaps, and improvement opportunities across the data platform.
• Prior experience working in a hybrid onsite–offshore delivery model.
Qualifications:
Required:
• 6+ Years of experience in data engineering
• Design, build, and orchestrate ETL/ELT pipelines using Azure Databricks
• Implement batch data ingestion and transformations using PySpark and Spark SQL
• Architect and maintain Lakehouse and analytical warehouse models (fact and dimension schemas) leveraging Delta Lake
• Ensure data quality, reliability, lineage, and governance across the data platform
• Collaborate with security and platform teams to enforce data access controls and best practices
• Experience on CI/CD pipelines using Git for Databricks
• Act as the onsite lead for Azure Databricks data engineering initiatives
• Supervise and guide an offshore development team, assigning work, reviewing deliverables, and ensuring adherence to best practices
• Perform code reviews, design reviews, and solution walkthroughs for offshore-developed components
• Ensure alignment between offshore execution and onsite architecture, security, and business requirements
• Proactively identify risks, gaps, and improvement opportunities across the data platform
• Prior experience working in a hybrid onsite–offshore delivery model
• Strong, proven expertise in Azure Databricks and Delta Lake
• Advanced proficiency in SQL with a solid understanding of distributed data processing concepts
• Demonstrated experience leading, mentoring, or supervising data engineering teams
• Strong experience performing code reviews and solution design validations
• Knowledge of data security, and access management (RBAC)
Preferred:
• Experience with Azure Synapse / Fabric
• Azure Data Factory
Company:
Tata Consultancy Services is a business solutions company that specializes on information technology services and consulting. It is a sub-organization of Tata Group. Founded in 1968, the company is headquartered in Mumbai, IND, with a team of 10001+ employees. The company is currently Late Stage.