Remote Databricks Data Engineer Jobs in New York

Good Things Start Here.

Good things are happening at Berkshire Hathaway GUARD Insurance Companies-an A+ (Superior) rated, nationwide Property & Casualty insurer backed by Berkshire Hathaway. With supportive leadership, collaborative teams, and opportunities to grow, GUARD is a place where people build meaningful, longterm careers.

Good Things You Can Count On.

• Hybrid schedule: 2 days remote / 3 inoffice
• Predictable hours (no nights, weekends, or holidays)
• Competitive pay + generous PTO
• Medical, dental & vision starting day one
• 401(k), tuition reimbursement & longevity bonuses

We are hiring ahands-on senior level Data Engineer to build trustworthy, well governed data productsand turn them into insights that drive underwriting, claims, product, distribution, and operations for the organization. You will collaborate directly with business leaders to translate questions into curated datasets, intuitive Power BI models, analytic insights and clear narratives that influence decisions. You'll context switch comfortably between data & analytic engineering and meeting withbusiness stakeholders to understand their processes and needs. 

What You'll Do:

• Collaborate closely with cross-functional teams; including Underwriting, Product, Claims, Finance, and Distribution/Operations- to transform business questions into clearly documented requirements and actionable data solutions.
• Model and publish curated, analysis ready datasets, semantic modelsin Power BI or analytics delivery that illuminate small commercial metrics. 
• Continuously learn and expand business knowledge to proactively produce automated analytic solutions to business challenges. 
• Design, develop, and maintain scalable ELT/ETL pipelines and lakehouse structures in Microsoft Fabric; Follow platform standards, optimize for performance, reliability, and cost. 
• Follow and implement documentation standards, governance and data quality controls across datasets. 
• Use Azure DevOps (ADO) for CI/CD and version control (branching, pull requests, release pipelines) to harden delivery and improve repeatability. 
• Document technical architecture and workflows to support onboarding and reduce risk
• Apply best practices in data modeling and engineering to create reusable, high-quality data products.
• Work in an Agile/Scrum environment, contributing to sprint planning, backlog refinement, and demos.
• Support development and scaling of data products, BI solutions, and AI-driven use cases across business areas.

Required

• Hands-on experience in data modeling (conceptual, logical, physical), data engineering, analytics engineering, SQL, and Python
• Bachelor's or Master's degree in Computer Science, IT, or related field
• 5+ years of data engineering experience, preferably in P&C insurance
• Experience with Data Mesh (domain ownership and decentralized data products)
• Strong experience with modern lakehouse platforms (e.g., Microsoft Fabric, Databricks, Snowflake, AWS)
• Experience building data solutions in cloud environments (Azure preferred; AWS a plus)
• Familiarity with Azure DevOps (ADO) and CI/CD practices
• Experience with data transformation, semantic modeling, and BI tools (Power BI, Tableau)
• Strong analytical and problem-solving skills
• Experience working in Agile/Scrum environments with product-based delivery
• Experience delivering end-to-end data, analytics, or AI solutions with business stakeholders
• Strong communication and collaboration skills

Preferred:

• Experience with distributed processing frameworks (e.g., Spark)
• Knowledge of data warehousing concepts and tools
• Additional programming experience (e.g., Scala)
• Understanding of data security and compliance
• Microsoft Fabric certifications (e.g., DP-600, DP-700)
• Experience with AI agents or multi-agent frameworks for business use cases

*Applicants must be authorized to work in the U.S. without current or future sponsorship*

Salary Range: 

$100,000 - $140,000.

In accordance with applicable pay transparency laws, the above range represents a good-faith estimate. Final compensation will be determined based on factors such as experience, credentials, geographic location, and other considerations permitted by law.

This role may be based out of any of our office locations, including:

Plano, TX, New York, NY, and/or Scottsdale, AZ

The successful candidate is expected to work in one of our offices 3 days per week and also be available for travel as required.