ZipRecruiter

Log In

1

Senior Databricks Data Engineer Jobs in Washington, DC

Berkley

Databricks Data Engineer

Manassas, VA · On-site

$114K - $137K/yr

The Databricks Data Engineer will help design, build, deploy, and maintain scalable and production grade data pipelines in modern cloud environments, enabling analytics, AI, ML, and decision ...

Berkley

Databricks Data Engineer

Manassas, VA

$114K - $137K/yr

The Databricks Data Engineer will help design, build, deploy, and maintain scalable and production grade data pipelines in modern cloud environments, enabling analytics, AI, ML, and decision ...

Berkley

Databricks Data Engineer

Manassas, VA

$114K - $137K/yr

The Databricks Data Engineer will help design, build, deploy, and maintain scalable and production grade data pipelines in modern cloud environments, enabling analytics, AI, ML, and decision ...

ECS

Senior Databricks Platform Engineer

Quantico, VA · On-site

$140K - $165K/yr

We are seeking a highly skilled Senior Databricks Platform Engineer to design, implement, and ... Implement data lineage, auditing, and compliance frameworks. * Design and enforce compute policies ...

ICF

Data Engineer (Databricks)

Reston, VA · Remote

$119K - $143K/yr

... Senior Databricks SME who is passionate about turning complex data into actionable solutions that ... You'll be joining a cross-functional team of full stack developers, data engineers, and data ...

ECS

Senior Databricks Migration Engineer

Arlington, VA · On-site

$130K - $158K/yr

ECS is seeking a Senior Databricks Migration Engineer to work onsite for the US Postal Services ... This role leads technical data migration efforts from legacy SQL systems, warehouses and servers to ...

ICF International, Inc.

Data Engineer (Databricks)

Reston, VA · On-site

$119K - $143K/yr

... Senior Databricks SME who is passionate about turning complex data into actionable solutions that ... You'll be joining a cross-functional team of full stack developers, data engineers, and data ...

ECS

Sr. Databricks Consultant

Fairfax, VA · On-site +1

$175K - $250K/yr

ECS is seeking a Sr. Databricks Consultant to work in our Arlington, VA office/remotely . Please ... Strong proficiency in Python and/or SQL for data engineering. * Experience with Apache Spark for ...

next page

Showing results 1-20

All JobsDatabricks Data Engineer JobsSenior Databricks Data Engineer JobsSenior Databricks Data Engineer Jobs in Washington, DC

Senior Databricks Data Engineer information

See Washington, DC salary details

$67.4K

$143.3K

$207.8K

How much do senior databricks data engineer jobs pay per year?

As of Aug 20, 2026, the average yearly pay for senior databricks data engineer in Washington, DC is $143,338.00, according to ZipRecruiter salary data. Most workers in this role earn between $118,400.00 and $162,500.00 per year, depending on experience, location, and employer.

What is a Senior Databricks Data engineer?

Senior Databricks Data Engineers are experienced professionals who design, develop, and optimize large-scale data processing pipelines using the Databricks platform. They work with big data technologies like Apache Spark, Delta Lake, and cloud platforms such as Azure or AWS. Their responsibilities include building and maintaining ETL processes, ensuring data quality, and collaborating with data scientists and analysts to deliver reliable, high-performance data solutions. As senior engineers, they also mentor junior team members and contribute to architectural decisions.

How does a Senior Databricks Data engineer typically collaborate with data scientists and analysts on large-scale projects?

A Senior Databricks Data Engineer works closely with data scientists and analysts to design, build, and optimize data pipelines that enable advanced analytics and machine learning initiatives. They often participate in cross-functional meetings to understand data requirements, translate them into scalable ETL processes, and ensure data quality and accessibility. Regular collaboration also involves troubleshooting data issues, optimizing Spark jobs for performance, and sharing best practices for data management. This close teamwork ensures that analytical teams have reliable, timely, and well-structured data to drive insights and decision-making.

What are the key skills and qualifications needed to thrive as a Senior Databricks Data engineer, and why are they important?

To thrive as a Senior Databricks Data Engineer, you need advanced expertise in data engineering concepts, big data technologies, and proficiency in programming languages like Python or Scala, usually backed by a bachelor's degree in computer science or a related field. Familiarity with Databricks, Apache Spark, cloud platforms (Azure, AWS, or GCP), and certifications like Databricks Certified Data Engineer are typically required. Strong problem-solving skills, effective communication, and the ability to collaborate across teams distinguish top performers in this role. These skills are essential to efficiently design scalable data solutions, optimize data workflows, and drive business insights in complex data environments.

What is the difference between Senior Databricks Data Engineer vs Data Engineer?

AspectSenior Databricks Data EngineerData Engineer
CredentialsTypically requires experience with Databricks, Spark, cloud platforms, and often certifications like Databricks Certified Data Engineer AssociateRequires knowledge of data pipelines, SQL, ETL tools, and often cloud platform experience, but less specialized in Databricks
Work EnvironmentWorks primarily within Databricks environment, focusing on big data processing and analyticsWorks across various data tools and platforms, including traditional ETL and cloud services
Industry UsageCommon in organizations leveraging Databricks for big data analytics and machine learningWidely used across industries for general data pipeline development and data management

The main difference is that a Senior Databricks Data Engineer specializes in using Databricks and Spark for big data solutions, often requiring specific certifications and experience. A Data Engineer has a broader focus on data pipeline development across various tools and platforms, with less emphasis on Databricks-specific skills.

Is a Senior Databricks Data Engineer in demand?

A Senior Databricks Data Engineer is in high demand due to the increasing adoption of cloud-based data platforms and the need for advanced data processing skills. Professionals with expertise in Spark, SQL, and cloud environments like Azure or AWS are particularly sought after in data-driven industries.

What are the most commonly searched types of Databricks Data Engineer jobs in Washington, DC?

The most popular types of Databricks Data Engineer jobs in Washington, DC are:

What are popular job titles related to Senior Databricks Data Engineer jobs in Washington, DC?

For Senior Databricks Data Engineer jobs in Washington, DC, the most frequently searched job titles are:

What job categories do people searching Senior Databricks Data Engineer jobs in Washington, DC look for?

The top searched job categories for Senior Databricks Data Engineer jobs in Washington, DC are:

Senior Databricks Data Engineer jobs near you
Infographic showing various Senior Databricks Data Engineer job openings in Washington, DC as of August 2026, with employment types broken down into 1% As Needed, 82% Full Time, 13% Part Time, and 4% Contract. Highlights an 87% Physical, 3% Hybrid, and 10% Remote job distribution, with an average salary of $143,338 per year, or $68.9 per hour.

Databricks Data Engineer

Berkley

Manassas, VA • On-site

$114K - $137K/yr

Other

Medical, Dental, Vision, Life, Retirement, PTO

Re-posted 28 days ago

Job description

Company Details
We started in early 2019 as a small group of technologists with a passion for making insurance better. Today we are working with a team of industry experts who run five different insurance brands and collectively control $1 billion in annual premiums.
We believe in an idea and execution meritocracy. In other words, a place where the best ideas win and the people who deliver the most value get the most opportunities.
As we grow our team, we are looking for inquisitive, entrepreneurial people who are excited to reimagine the insurance industry.
Insurance is too complex. Help us make it better.
Responsibilities
This position requires on-site work Monday-Thursday at either our Manassas, VA or Chesterfield, MO location.
The Databricks Data Engineer will help design, build, deploy, and maintain scalable and production grade data pipelines in modern cloud environments, enabling analytics, AI, ML, and decision advantage at scale. This role will work with cutting-edge tools like Databricks, Delta Lake, PySpark, and AI/BI genie to transform raw data into actionable insights. As a hands-on Databricks Data Engineer with deep expertise in Azure Databricks and MLOps, this role will have the opportunity to migrate and translate legacy SSIS ETL logic into scalable, cloud-native data pipelines in Databricks. This role will partner with data engineers, data scientists, and product manager to design features, train/evaluate models, and deploy them to production using MLflow, Databricks and Workflows-with rigorous observability, governance (Unity Catalog), and CI/CD automation.
Data Pipeline Engineering
  • Design, build, and maintain high-performance, scalable ETL/ELT pipelines using Azure Databricks, Delta Lake, and PySpark.
  • Convert and modernize existing SSIS package logic into cloud-native Databricks pipelines using PySpark notebooks, Delta Live Tables (DLT), and Databricks Workflows.
  • Implement reliable batch and streaming pipelines with robust data quality and validation frameworks.
  • Optimize pipeline performance using Photon, efficient file formats, partitioning, Z-ordering, and caching strategies.

Lakehouse Platform Development
  • Develop and manage datasets within Delta Lake, ensuring ACID reliability, schema evolution, versioning, and time travel.
  • Architect feature-rich data layers including:
    • Bronze (raw ingestion)
    • Silver (validated, conformed)
    • Gold (analytics-ready and ML-ready)
  • Implement data governance using Unity Catalog for fine-grained access control, lineage, auditability, and metadata management.

MLOps & ML-Enabled Data Pipelines
  • Partner with data scientists and data engineers to create feature pipelines, model training pipelines, and production scoring pipelines.
  • Deploy and operationalize models using MLflow, Databricks Model Registry, and Databricks Workflows.
  • Use Databricks built-in AI SQL functions such as ai_query, ai_forecast, ai_analyze_sentiment to generate actionable insight from large amount of unstructured or structured raw data
  • Implement monitoring for:
    • Pipeline failures
    • Data/feature drift
    • Model performance degradation
    • Operational SLAs/SLIs/SLOs
  • Build automated CI/CD workflows using GitHub Actions or Azure DevOps for notebook deployment, pipeline testing, and environment promotion.

Data Platform, Data Security & Data Governance
  • Collaborate with data engineers to design reliable data products on Delta Lake; leverage Delta Live Tables (DLT) for declarative pipelines when applicable.
  • Enforce Unity Catalog for lineage, permissions, and audit; manage secrets, tokens, and keys securely (e.g., Databricks secrets, Key Vault/Secrets Manager).

Collaboration & Leadership
  • Work closely with cross-functional teams: data engineering, data scientist, product manager, and business stakeholders.
  • Serve as a Databricks SME-championing best practices, code standards, governance, and reusable frameworks.
  • Document architecture, workflows, data models, runbooks, and operational procedures.

Qualifications
  • Minimum of 3 years of experience in Databricks, PySpark notebooks, Python, DevOps, software development, and data engineering.
  • Certified Databricks Data Engineer Associate or Professional is a plus.

Skills & Competencies
  • Proficient in designing, building, deploying, and maintaining high-performance, scalable ETL/ELT pipelines using Azure Databricks, Delta Lake, and PySpark Notebook.
  • Proficient in building, deploying, and operating production ML models such as supervised, unsupervised, and anomaly detection, including techniques for imbalanced datasets
  • Proficient with ML engineering and MLOps, including model versioning, CI/CD for ML, monitoring, drift detection, and automated retraining
  • Proficiency in Python including Pandas and PySpark Dataframes
  • Expert level of SQL skills including Stored Procedure, experience with SSIS, SSRS, Power BI is a plus.
  • Proficient with cloud data engineering platforms, such as Azure, Databricks, Spark, or SQL, and batch and streaming pipelines
  • Familiar with Databricks AI Built-In Functions such as AI_Query, AI_Gen, AI_Classify, AI_Forecast, AI_Analyze_Sentiment, able to use them to extract actionable insights from large amount of unstructured or structured raw data
  • Experience with Python and ML frameworks, such as PyTorch or TensorFlow
  • Experience improving data quality, lineage, and observability in enterprise data environments and operationalizing rules and model-driven scoring for prioritization, routing, or case selection
  • Experience with predictive analytics, machine learning and artificial intelligence desired.

Education
  • A Bachelor's degree in Computer Science, Management Information Systems, Engineering, Math, Physics, or a related quantitative field is required (4-year degree). Master's degree preferred
  • Experience in the commercial insurance industry is a plus.

Additional Company Details
The Company is an equal employment opportunity employer.
We do not accept any unsolicited resumes from external recruiting firms.
The company offers a competitive compensation plan and robust benefits package for full time regular employees.
Base salary & Benefits include Health, dental, vision, life, disability, wellness, paid time off, 401(k) and profit-sharing plans.
-Salary Range: $110k- $140k
The actual salary for this position will be determined by a number of factors, including the scope, complexity and location of the role; the skills, education, training, credentials and experience of the candidate; and other conditions of employment.
Additional Requirements
• Ability to travel locally and nationally up to 5% of the time
Sponsorship Details
Sponsorship not Offered for this Role

Apply