Job Description:
· Cloud & Data Engineer with hands-on experience deploying and administering Azure resources, including compute, networking, storage, security, and monitoring components
· Proven ability to automate cloud provisioning using Terraform, ARM templates, Azure DevOps, GitHub Actions, and Jenkins to improve deployment speed and reliability.
· Experienced in designing and building distributed data pipelines using ADF, ADLS Gen2, Azure SQL, Synapse, Event Hubs, and Stream Analytics.
· Strong background in Databricks administration and PySpark development, delivering high-performance ELT pipelines, Delta Lake solutions, and workspace governance.
· Familiar with ML Ops frameworks, Azure Machine Learning (AML), MCP servers, and enterprise AI Search integration for production-grade AI workloads.
· Strong command of Python, PySpark, and SQL performance tuning for large-scale data processing and analytics workloads.
Responsibilities
- Identify, Clarify, Analyze and Confirm any requests for data sourcing or data changes for our ETL pipelines.
- Ensure that the requirements are clear, edge cases are considered, and impact of the changes are understood.
- Effectively manage our backlog prioritized and is a clear reflection of what the team is currently working on.
- Assist and support the developers from intake, to development, testing, and release of new features.
- Liaise between our Stakeholder, Product Owners, and upstream source systems.
- Ensure our documentation and our runbooks are up to date.
- Keep up to date with the latest industry trends and technologies related to data engineering
Preferred Qualifications
· 8+ years of relevant experience in a related field of job function.
· Azure certified, Databricks certified
· Experience with Jira, Confluence.
· · Experience with APIs, SQL.