JOB SUMMARY
IT Data Engineer III is a senior-level position on the Enterprise Data team, responsible for leading the design, development, and support of complex enterprise data solutions across modern cloud platforms. This role plays a key leadership role in optimizing CDCN's data systems and processes for performance, scalability, and reliability while advancing our technical capabilities with cloud-native technologies. The IT Data Engineer III serves as a technical expert and mentor, guides ad hoc teams, and leads efforts to improve data workflows, architecture, and integration across the organization. This position requires deep technical expertise in Azure data platforms including Fabric and Synapse Analytics, strong collaboration skills, and the ability to communicate effectively with both technical and non-technical stakeholders.
JOB DUTIES
- Lead initiatives to design, develop, and maintain enterprise data warehouse systems and data integration pipelines used for analysis and reporting, leveraging Azure Fabric, Azure Synapse Analytics, and related Azure data platform technologies.
- Design and implement scalable ETL/ELT (extract, transform, load) solutions across enterprise systems, leveraging advanced data engineering tools and technologies including Azure Data Factory, Databricks, and serverless data processing platforms.
- Architect and optimize dimensional data models including fact tables, dimension tables, and slowly changing dimensions to support analytics and reporting requirements.
- Lead response efforts to stakeholder data needs and system issues, serving as the primary technical contact for complex or escalated requests and ensuring alignment with business requirements and regulatory compliance.
- Troubleshoot and resolve high-priority issues during regular business hours and as part of the after-hours on-call rotation, ensuring critical data systems remain operational and performing optimally.
- Guide ad hoc teams in analyzing and improving data processes, system performance, and cost-efficiency through technical innovation, process redesign, and optimization of cloud infrastructure costs.
- Provide technical mentorship and direction to peers and junior engineers across data projects and ongoing operations, fostering continuous learning and technical excellence.
- Collaborate with engineers, analysts, and business partners to ensure that solutions align with strategic business goals, regulatory requirements, and industry best practices.
- Implement and maintain data quality frameworks, validation rules, and monitoring mechanisms to ensure data accuracy, consistency, and reliability across enterprise systems.
- Lead technical design reviews and code reviews, establishing and enforcing data engineering standards, best practices, and architectural principles.
- Perform other duties as assigned, including leadership of special projects or enterprise-level initiatives.
QUALIFICATIONS
Education & Experience:
- Bachelor's degree in Computer Science, Information Technology, Data Analytics, Data Science, Mathematics, or a related field required; equivalent professional experience will be considered.
- 5-9 years of experience designing, developing, and supporting data platforms and enterprise data warehouse systems in production environments.
- Proven ability to lead and serve as a technical expert on small to medium-sized projects and cross-functional initiatives.
- Demonstrated experience mentoring junior engineers and contributing to team technical development.
Core Technical Skills:
- Expert-level experience with Azure Fabric including lakehouse architecture, notebooks, Delta Lake, data pipelines, and data warehouse design.
- Deep hands-on experience with Azure Synapse Analytics including dedicated SQL pools, serverless SQL pools, Synapse pipelines, and performance optimization.
- Expert-level SQL scripting with advanced proficiency in writing complex queries, stored procedures, query optimization, and performance tuning.
- Deep experience in ETL/ELT development and data pipeline architecture across enterprise systems.
- Advanced dimensional data modeling expertise including star schema, snowflake schema, fact tables, dimension tables, and slowly changing dimensions.
- Expert-level experience with cloud data platforms such as Azure (primary), AWS, or Oracle Cloud.
- Extensive hands-on experience with tools and technologies including Azure Data Factory, Azure Fabric, Azure Synapse Analytics, Databricks, Azure Data Lake Storage Gen2, Azure DevOps, SSMS, and Power BI.
- Expert-level understanding of data structures and data exchange formats including ANSI X12, JSON, XML, and industry-standard data formats.
- Proficiency in Python or PySpark for data processing and transformation tasks.
DevOps & Engineering Practices:
- Strong experience with version control systems (Git) and CI/CD practices for data engineering.
- Familiarity with infrastructure as code (Terraform, ARM templates, or Bicep).
- Experience implementing data quality frameworks and automated monitoring/alerting mechanisms.
- Knowledge of data governance, security best practices, and regulatory compliance requirements.
- Professional & Interpersonal Skills:
- Demonstrated ability to communicate complex technical concepts clearly to both technical and non-technical audiences.
- Strong problem-solving skills, attention to detail, and ability to manage multiple priorities under pressure.
- Able to work independently or as a team lead, taking full ownership of technical solutions from design through implementation and support.
- Proven ability to mentor peers and junior engineers, contributing to technical team development.
- Ability to work effectively on-call and respond to critical production issues with urgency and professionalism.
Machine Learning & AI (Preferred Qualifications):
- Experience with Azure Machine Learning including model training, deployment, and inference pipelines integrated into data workflows
- Familiarity with MLOps practices including model versioning, monitoring, retraining workflows, and model governance
- Understanding of feature engineering and the role of data pipelines in preparing data for machine learning applications
- Experience with Databricks ML features including MLflow and distributed model training
- Knowledge of AutoML and automated feature selection tools within the Azure ecosystem
- Experience implementing model monitoring and performance tracking in production environments
- Familiarity with responsible AI principles including bias detection, explainability, and ethical AI considerations
- Understanding of A/B testing frameworks and experimentation workflows in data-driven applications
Other Requirements:
- Must be able to successfully pass a background screening.
- Willingness to maintain and expand technical certifications relevant to Azure data platforms.
- Ability to work flexible and/or extended hours, if needed, to meet the job requirements.
- Employees must provide their own reliable internet if working a remote/hybrid position
The incumbent typically works in an office environment and uses a computer, telephone and other office equipment as needed to perform duties. The noise level in the work environment is typical of that of an office. Incumbent may encounter frequent interruptions throughout the workday. The employee is regularly required to sit, talk, or hear; frequently required to use repetitive hand motion, handle or feel, and to stand, walk, reach, bend or lift up to 20 pounds. Reasonable accommodations may be made to enable individuals with disabilities to perform the essential functions.