1

Overnight Databricks Data Engineer Jobs in Missouri

Data Engineer

Saint Louis, MO · On-site

$120 - $170/hr

Why This Role Exists The Data Scientist/Engineer supports USTRANSCOM's role as the Single Manager ... Develop and maintain data pipelines that ingest data into Databricks and support integration with ...

New

Data Engineer II

O Fallon, MO · Hybrid

$107K - $128K/yr

Title and Summary Data Engineer II Overview: Mastercard's Enterprise Credit Risk (ECR) team is ... Build and optimize ETL/ELT workflows using technologies such as SQL, Python, Databricks, and Spark.

Data Engineer

Saint Louis, MO · On-site

$110K - $133K/yr

The Data Engineer helps design, build, and maintain pipelines and platforms that power our data ... Expert with cloud-based data ecosystems, ideally Microsoft Azure (Data Factory, Synapse, Databricks ...

Data Scientist/Engineer On Site | Scott AFB, IL | Secret Clearance Required If you enjoy building ... What You Will Do • Develop and maintain data pipelines that ingest data into Databricks and ...

Lead Data Engineer

O Fallon, MO · Hybrid

$95K - $126K/yr

Title and Summary Lead Data Engineer Overview: Mastercard's Enterprise Credit Risk (ECR) team is ... Architect and implement scalable ETL/ELT frameworks using Databricks, Apache Spark, Delta Lake, and ...

New

$109K - $130K/yr

Data Engineer The candidate must be committed to move to St. Louis and live here 100% from day 1. ... Travel - Occasional overnight travel 1-3 nights per quarter for company meetings. Must have ability ...

Azure Solutions Architect Expert, Azure Data Engineer Associate, Snowflake Core, Snowflake Databricks Data Engineer Associate] is a plus - Proficient in Python and SQL - Experience with Docker and ...

Azure Solutions Architect Expert, Azure Data Engineer Associate, Snowflake Core, Snowflake Databricks Data Engineer Associate] is a plus - Proficient in Python and SQL - Experience with Docker and ...

Showing results 21-40

Overnight Databricks Data Engineer information

What is an overnight Databricks data engineer?

An Overnight Databricks Data Engineer is a professional who works primarily during night shifts to manage, design, and maintain big data pipelines and workflows using Databricks, a cloud-based data analytics platform. Their responsibilities often include developing and optimizing data processing jobs, ensuring data quality, and troubleshooting issues that arise during overnight data operations. This role is critical for organizations that require 24/7 data processing, continuous ETL jobs, or real-time analytics. Working overnight may also involve monitoring automated systems, performing scheduled data loads, and collaborating with global teams to ensure data availability and reliability.

What skills and qualifications are needed to thrive as an overnight Databricks data engineer?

To thrive as an Overnight Databricks Data Engineer, you need strong proficiency in data engineering, Python or Scala programming, and experience with big data technologies, typically supported by a relevant degree in computer science or a related field. Familiarity with Databricks, Apache Spark, cloud platforms (such as AWS or Azure), and certifications like Databricks Certified Data Engineer are highly valued. Attention to detail, problem-solving, and effective communication are essential soft skills, especially for troubleshooting and collaborating across shifts. These competencies ensure reliable data pipeline management and efficient resolution of issues during off-hours, maintaining seamless business operations.

What unique challenges do overnight Databricks data engineers face, and how can they be addressed?

Overnight Databricks Data Engineers often work with limited real-time support, which can present challenges when troubleshooting urgent data pipeline issues or system outages. To address this, it’s essential to develop strong problem-solving skills, document processes thoroughly, and leverage automated monitoring and alerting tools. Additionally, close collaboration with daytime teams during handoff periods ensures continuity and minimizes disruptions. Building a habit of proactive communication and maintaining detailed logs helps the entire team resolve issues efficiently and maintain data quality.

What is the difference between Overnight Databricks Data Engineer vs Data Engineer?

AspectOvernight Databricks Data Engineer
Work EnvironmentPrimarily remote or on-site, working overnight shifts to support global data operations
CertificationsDatabricks certifications, cloud platform credentials (AWS, Azure), data engineering certifications
Tools & TechnologiesDatabricks platform, Spark, cloud services, SQL, Python, ETL tools
Industry UsageTech, finance, healthcare, retail with 24/7 data needs

While both roles focus on data engineering, the Overnight Databricks Data Engineer specializes in managing data pipelines on the Databricks platform during overnight shifts, often supporting global operations. A Data Engineer may work across various platforms and shifts, with broader responsibilities in data architecture and pipeline development. The overnight role emphasizes specific platform expertise and shift timing, catering to organizations with continuous data processing needs.

What are the most commonly searched types of Databricks Data Engineer jobs in Missouri?

The most popular types of Databricks Data Engineer jobs in Missouri are:

Data Engineer

D&G Solutions

Saint Louis, MO • On-site

$120 - $170/hr

Other

Retirement, PTO

Posted 3 days ago

New


Key responsibilities

  • Develop and maintain data pipelines that ingest data into Databricks and support integration with enterprise platforms.

  • Design, develop, and maintain APIs that provide standardized access to enterprise data.

  • Review, analyze, and support data integration efforts across various data sources and systems.


Job description

On Site | Scott AFB, IL | Secret Clearance Required
If you enjoy building modern data solutions, integrating complex data environments, and supporting mission-critical decision making for the Department of Defense, we would love to talk.

Why This Role Exists

The Data Scientist/Engineer supports USTRANSCOM's role as the Single Manager for Global Bulk Fuel Management and Delivery (GBFMD) by helping establish and maintain foundational data capabilities for the Joint Petroleum Enterprise (JPE). This role is responsible for designing, developing, and maintaining data pipelines, APIs, and integration solutions that enable the collection, management, and consumption of bulk fuel data across the Department of Defense. Supporting the development of a Global Common Operating Picture and other critical fuel management systems, this position plays a key role in advancing data-driven decision making across the enterprise.

What You Will Do
  • Develop and maintain data pipelines that ingest data into Databricks and support integration with Palantir Ontology and other enterprise platforms.
  • Collaborate with cross-functional teams to support Common Operating Picture (COP) development and related data integration efforts.
  • Review and analyze Joint Petroleum Enterprise data sources, including applications, websites, programs, and operational systems to identify opportunities for data integration and enhancement.
  • Design and implement scalable data ingestion processes that transform source data into structured formats suitable for enterprise consumption and API development.
  • Develop, publish, and maintain APIs that provide standardized access to enterprise data across the Joint Petroleum Enterprise.
  • Collaborate with stakeholders to manage and maintain enterprise data dictionaries, metadata repositories, and data governance artifacts.
  • Administer data structures, data topics, and data engineering processes while implementing industry best practices.
  • Participate in technical working groups and stakeholder meetings to support metadata requirements, data modeling, architecture decisions, and tool selection.
  • Support the preparation of presentations and technical materials for USTRANSCOM, Joint Staff, and Office of the Secretary of Defense governance forums.
  • Provide technical recommendations and support for enterprise data modernization initiatives.
  • Complete additional projects and tasks as assigned by program leadership.
What We Are Looking For

Required:

  • Bachelor's degree in Computer Science, Data Engineering, Information Systems, or a related field.
  • 5 or more years of experience in data engineering, software development, or a related technical discipline.
  • Strong proficiency with SQL and Python.
  • Experience developing and maintaining data pipelines using Apache Spark, Kafka, or similar technologies.
  • Experience working with cloud platforms such as AWS, Microsoft Azure, or Google Cloud Platform.
  • Experience designing and implementing data integration solutions.
  • Ability to communicate effectively with both technical and non-technical stakeholders.
  • Active Secret clearance with eligibility to obtain and maintain a TS/SCI.
  • Experience with Databricks.
  • Experience with Palantir Ontology.
  • Experience with data warehousing, ETL processes, and enterprise data management.
  • Experience with API development and RESTful web services.
  • Experience with data modeling techniques and metadata management.
  • Experience with Maven Smart Systems or similar logistics and supply chain platforms.
  • Experience supporting the Department of Defense, Defense Logistics Agency (DLA), or USTRANSCOM.
  • Experience working in Agile development environments.
Eligibility Requirements
  • Secret with the ability to obtain and maintain a TS/SCI.
  • Ability to travel within CONUS up to approximately 10%.
  • Restricted to U.S. persons as defined under 8 U.S.C. 1324b(a)(3).
Who We Are

At D&G Solutions, people come first; our teammates, customers, and the communities we serve.

We are one of the fastest growing companies in the Washington, DC area and proud to be recognized nationally as a Best Place to Work. As an 8(a), Service Disabled Veteran Owned, and Women Owned Small Business, we bring ownership, accountability, and care to everything we do. We also deeply believe in giving back and proudly support organizations such as Homes for Our Troops, SemperK9, and Wreaths Across America.

Our teams are known for being welcoming, collaborative, and deeply committed to mission success. We believe great work happens when people feel supported, respected, and trusted, and we work hard to create that environment every day.

What We Offer
  • Competitive salary
  • Benefits
  • 401(k)
  • Bonus and profit sharing
  • Flexible hours
  • Education reimbursement
  • PTO
Final Note

If you want to work for a company that cares about its people, contribute to a mission that truly matters, and be part of a team where you are valued for who you are as well as what you do, we would love to hear from you.

#J-18808-Ljbffr