Trainee Databricks Data Engineer Jobs in Springfield, MA

Job Summary:
Travelers is a leading property casualty insurer with a commitment to innovation and collaboration. As a Senior Data Engineer, you will design and build production data pipelines, mentor team members, and leverage data solutions to support analytics and business intelligence. Your role will also involve modernizing legacy workloads and optimizing data processes for better performance and cost efficiency.
Responsibilities:
• Design and build production data pipelines across AWS, Snowflake, Databricks supporting both batch and near real-time analytics workloads.
• Establish reusable engineering patterns and frameworks - Parameterized, modular, idempotent pipeline templates that other engineers adopt, reducing duplicated effort and inconsistent implementations.
• Drive down lead time from commit to production by removing manual steps, leveraging AI, building self-service tooling, and standardizing the path to deployment; treat cycle time as a metric you actively own and improve.
• Champion SDLC discipline covering version control, peer code review, automated testing, environment promotion, change management, and documentation.
• Integrate AI coding tools into daily workflow to accelerate scaffolding, refactoring, test generation, code optimization, and documentation, with measurable and demonstrable impact on throughput and quality.
• Measure and demonstrate impact, tying AI-tool adoption to concrete outcomes such as reduced lead time, faster test coverage, and improved consistency, and sharing those results to drive broader adoption.
• Evaluate emerging tooling and make pragmatic recommendations on what engineers should adopt, standardize on, or avoid.
• DataOps- Blur the lines between data and software engineering practices. Employ CI/CD, automated testing, and apply trunk-based or short-lived branch development to data the same way it is to software.
• Modernize legacy workloads - Help manage and optimize Ab>Initio pipelines and when applicable, help migrate or re-platform Ab>Initio pipelines toward cloud-native, declarative, ELT-based patterns on Snowflake and Databricks where it delivers value.
• Embed data quality, observability, and lineage into pipelines as a default, not an afterthought, automated data tests, freshness/quality SLAs, and traceable lineage.
• Optimize for cost and performance across Snowflake compute, Databricks clusters, and storage, applying FinOps-aware engineering practices.
• Mentor and upskill engineers through code review, pairing, design guidance, and documented standards, acting as a technical multiplier for the team.
Qualifications:
Required:
• Bachelor’s Degree in STEM related field or equivalent
• Ten years of related experience building, designing and operating production data pipelines at scale.
• Demonstrable experience architecting, designing and building scalable, secure data solutions using AWS, Databricks, Snowflake and Ab>Initio or similar platforms.
• A track record of leveraging AI assistants, create skills/tools to augment data engineering practices throughout the development lifecycle.
• The ability to lead technical direction for data engineering initiatives across cloud and on-premises infrastructure.
• Willingness to run mentoring sessions and offer technical guidance to the 20-person admin team.
• Ability to manage infrastructure deployment and optimize cloud resources.
• Drive to learn, identify and set technical standards and influence engineering practices and data governance policies.
• Ability to lead and take action even when there is no clear owner, inspire and motivate others, and be effective at influencing team members.
• Technical Skills: Cloud: Proficiency with commonly used AWS services and architectures for data and analytics solutions
• Databricks: Workspace management, cluster configuration, open table formats (Iceberg, Delta Lake), Unity Catalog, building and tuning Spark/pySpark/SQL workloads.
• Data Warehousing: Strong understanding of data modeling, dimensional modeling (star/snowflake schemas) and medallion (bronze/silver/gold) architecture.
• AI Coding Assistants: Familiarity using Claude Code, Codex, Copilot, Cursor, etc. for day-to-day engineering tasks.
• AbInitio: Proficiency with GDE, Co>Operating System, and EME, including maintaining and optimizing existing graphs.
• CI/CD for data: Git-based workflows, branching strategy, and pipeline automation (e.g., GitHub Actions, GitLab CI, or Jenkins).
• Infrastructure as Code (e.g., Terraform): Familiarity with a configuration-driven, repeatable approach to environments
• Bachelor’s degree in computer science, related STEM field, or its equivalent in education and/or work experience.
• 6 additional years of data engineering experience.
Company:
Travelers provides insurance coverage to protect the things that are important to you – your home, your car, your valuables and your business. Founded in 1979, the company is headquartered in Walhalla, USA, with a team of 10001+ employees. The company is currently Late Stage.