Job Summary:
McKinsey & Company is a global management consulting firm that drives lasting impact and builds long-term capabilities with its clients. In this role, you will be a core member of Periscope’s technology team, responsible for designing, developing, and maintaining scalable data pipelines and systems, while leading data engineering projects from inception to completion.
Responsibilities:
• design, develop, and maintain scalable data pipelines and systems using Databricks on Azure
• implement and optimize ETL processes with a focus on performance, cost-efficiency, and reliability
• utilize advanced Databricks technologies (DLT, Unity Catalog, Delta Sharing, SQL Warehouse) to enhance data management and sharing capabilities
• lead and execute data engineering projects from inception to completion, ensuring timely delivery and high quality
• continuously monitor, troubleshoot, and improve data pipelines and workflows to ensure optimal performance and cost-effectiveness
• collaborate with cross-functional teams to understand data needs and deliver solutions that meet business requirements
• mentor, and educate junior engineers on best practices, emerging technologies, and efficient data engineering techniques
• work with client facing teams to help solve and deploy client specific solutions that integrate into the client’s internal systems
Qualifications:
Required:
• Bachelor’s or master’s degree in computer science, engineering or a related field
• 6+ years of experience in data engineering, with a strong background in big data technologies
• Proficiency in Databricks on Azure, Spark, Python, and Spark SQL
• Extensive experience with Delta Live Tables, Unity Catalog, Delta Sharing, and SQL Warehouse (server and serverless)
• Experience in handling large data volumes and orchestrating automated ETL using Azure Data Factory, Airflow / data pipelines using CI/CD and Cloud Technologies
• Development efficiency leveraging AI framework technologies
• Demonstrated ability to optimize performance and cost in large-scale data environments
• Proven track record of leading projects independently and mentoring junior team members
• Experience with cloud platforms, particularly Azure, and a solid understanding of cloud-native architectures
• Experience adopting software development best practices (e.g., modularization, testing, refactoring, etc.)
• Experience with data governance, security, and compliance in a cloud environment
• Familiarity in understanding BI framework
Company:
McKinsey & Company is a global management consulting firm. We are the trusted advisor to the world's leading businesses, governments, and institutions. Founded in 2014, the company is headquartered in New York, USA, with a team of 10001+ employees. The company is currently Late Stage.