Job Summary:
CSpring is a purpose-driven consulting firm that helps organizations solve complex business and technology challenges through modern data, analytics, and engineering solutions. They are seeking a Senior Azure Data Engineer to design, build, and support a next-generation enterprise data platform on Microsoft Azure, focusing on developing scalable data pipelines and data products for analytics and reporting.
Responsibilities:
• Design and develop reusable, parameter-driven ingestion and transformation pipelines using Azure Data Factory, Synapse Pipelines, Databricks, and/or Microsoft Fabric Data Factory
• Build and maintain medallion architecture (Bronze / Silver / Gold) solutions using Azure Data Lake Storage Gen2, Delta Lake, Parquet, and structured streaming patterns
• Develop performant ELT workflows leveraging pushdown processing to platforms such as Synapse Dedicated SQL Pool, Azure SQL, and Teradata
• Create and optimize PySpark notebooks and distributed processing jobs in Azure Databricks or Synapse Spark
• Design dimensional data models using Kimball star and snowflake methodologies
• Implement data vault patterns, Slowly Changing Dimensions (Type 1/2/3), Change Data Capture, and late-arriving data strategies
• Optimize distributed SQL workloads in Synapse Dedicated SQL Pool and/or Fabric Warehouse environments
• Tune partitioning, indexing, and query performance for enterprise-scale datasets
• Implement CI/CD processes for data pipelines using Azure DevOps, YAML pipelines, ARM templates, Bicep, and/or Terraform
• Build monitoring, logging, and auditing solutions using Azure Monitor, Log Analytics, and KQL
• Support code reviews, branching strategies, release management, and engineering standards across environments
• Participate in troubleshooting and production incident response for critical data pipelines
• Lead or contribute to cloud modernization initiatives, including Informatica PowerCenter to Azure Data Factory migrations
• Support migration efforts from on-premises Teradata, Oracle, or SQL Server environments to Azure Synapse or Microsoft Fabric
• Assist with workload assessments, capacity planning, and cloud cost optimization initiatives
Qualifications:
Required:
• Deep hands-on expertise with Azure Data Factory, including pipelines, datasets, linked services, triggers, parameterization, mapping data flows, and Integration Runtime types (Azure, Self-hosted, and SSIS)
• Strong experience with Azure Databricks and PySpark
• Production experience with one or more of the following: Azure Synapse Analytics (Dedicated SQL Pools, Serverless SQL Pools, Spark Pools), Azure Databricks (Delta Lake, Unity Catalog), Microsoft Fabric (Warehouse, Lakehouse, OneLake)
• Strong understanding of Azure Data Lake Storage Gen2, including hierarchical namespace, RBAC/ACL security, lifecycle management, and governance
• Experience with Azure Key Vault, Azure AD / Entra ID, managed identities, service principals, and private networking concepts
• Experience monitoring and troubleshooting data solutions using Azure Monitor, Log Analytics, and KQL
• Advanced SQL skills including window functions, CTEs, query optimization, execution plan analysis, and performance tuning
• Strong Python skills for data engineering, including pandas, PySpark, REST API integration, and unit testing with pytest
• Proficiency with T-SQL and familiarity with Spark SQL, KQL, PowerShell, and Bash scripting
Preferred:
• 5+ years of enterprise data warehouse or data engineering experience
• 5+ years of data modeling experience using ERWIN or similar modeling tools
• 2+ years of experience with Azure Data Factory and Snowflake
• Experience working in healthcare or Medicaid environments
Company:
CSpring is a technology consulting company that delivers the systems and expertise needed to achieve technology-enabled business goals. Founded in 1996, the company is headquartered in Indianapolis, USA, with a team of 51-200 employees. The company is currently Growth Stage.
