Overview:Job Title: Senior Data Engineer (Databricks)Location: Chicago, IL / Remote
Job Type: Full-Time
Job SummaryWe are seeking a highly skilled
Senior Data Engineer with strong expertise in
Databricks, Apache Spark, SQL, and cloud-based ETL pipelines. The ideal candidate should possess strong software engineering fundamentals and experience building scalable, high-performance data engineering solutions in cloud environments.
Required Skills - Hands-on expertise in:
- Databricks
- Apache Spark
- SQL
- Cloud-based ETL pipelines
- CI/CD implementation
- Strong experience building scalable and distributed data engineering solutions
- Strong software engineering and problem-solving skills
- Experience with cloud platforms and modern data architectures
- Knowledge of performance optimization, monitoring, and cost management
Key Responsibilities - Design, develop, test, and deliver high-quality data engineering solutions aligned with business needs
- Build, optimize, and maintain scalable data pipelines using Databricks and Apache Spark
- Architect distributed data processing frameworks ensuring reliability, scalability, and performance
- Troubleshoot and resolve complex application and data issues through root cause analysis
- Implement code reviews, engineering standards, and best practices to ensure maintainability and quality
- Develop and support CI/CD pipelines, automated testing, and monitoring solutions for cloud data platforms
- Optimize cloud data platform performance and operational costs
- Mentor junior and mid-level engineers and contribute to continuous engineering improvements
Preferred Skills - Experience with Azure, AWS, or GCP data platforms
- Exposure to Delta Lake, Data Lakes, or Lakehouse architectures
- Familiarity with Agile/Scrum development methodologies