Client is looking for a strong Azure Data Engineer to join their project.
Job Title: Azure Data Engineer
Location: Chicago, IL - must do one day week onsite
Length: 6 months
This will require an onsite interview or require the candidate to pick up their laptop in person
Requirements/Top Skills Needed:
ADF
Databricks
Python
SQL
Retail experience would be helpful, but not required
Job Title: Azure Data Engineer
Job Summary: ย We are seeking a highly skilled Azure Data Engineer to design, build, and optimize our next-generation cloud data platform. You will build scalable data pipelines, manage enterprise data warehouses, and transform massive datasets using Azure Cloud, ADF, Databricks, Spark, and PySpark.
Key Responsibilities
ยท Pipeline Orchestration: Design and implement scalable ETL/ELT pipelines using Azure Data Factory (ADF).
ยท Big Data Processing: Build high-performance, distributed data transformation workloads via Azure Databricks and Apache Spark/PySpark.
ยท Data Warehousing: Architect, model, and maintain cloud-native Data Warehouses and Data Lakes (ADLS Gen2).
ยท Architecture Implementation: Apply modern architectural patterns like the Medallion Architecture (Bronze, Silver, Gold layers) and Delta Lake formats.
ยท Performance Tuning: Optimize Spark jobs, cluster configurations, and SQL queries to minimize latency and cloud compute costs.
ยท Governance & Security: Implement security frameworks, data lineage, and user access policies using Unity Catalog.
ยท CI/CD Automation: Deploy, monitor, and automate workflows using Azure DevOps and automated alert configurations.
Required Technical Skills:
ยท Cloud Platform: Microsoft Azure Ecosystem (ADLS Gen2, Azure SQL, Synapse Analytics, Key Vault, Storages, Azure Functions, Logic Apps).
ยท Data Orchestration: Advanced proficiency in Azure Data Factory (ADF).
ยท Compute & Analytics: Deep hands-on experience with Azure Databricks.
ยท Big Data Frameworks: Strong mastery of Apache Spark and PySpark (including Spark SQL).
ยท Languages: Expert level in Python and SQL.
ยท Data Warehousing: Solid understanding of Data Modeling (Star/Snowflake schema) and warehousing concepts