ZipRecruiter

Log In

1

Overnight Data Pipeline Engineer Jobs in Indiana

SEP

Data Engineer (in person)

Westfield, IN · On-site

$100 - $125/hr

Build and maintain data pipelines and transformations (batch and streaming) * Implement data models ... Experience with analytics engineering tools like dbt and data catalog tools (Unity Catalog ...

DPDHL

Data Engineer I

Whiteland, IN · On-site

$114K - $137K/yr

As a Data Engineer I, you'll design and optimize data pipelines that ensure reliable, accessible information across DHL Supply Chain. You'll develop and maintain data flows, integrate systems using ...

AAA Hoosier Motor Club

Data Engineer

Indianapolis, IN · On-site

$109K - $131K/yr

Build, maintain, and expand data pipelines and cloud warehouse infrastructure supporting analytics and reporting. * Integrate and onboard data from a variety of platforms and source systems into the ...

AAA Hoosier Motor Club

Data Engineer

Indianapolis, IN · On-site

$109K - $131K/yr

Build, maintain, and expand data pipelines and cloud warehouse infrastructure supporting analytics and reporting. * Integrate and onboard data from a variety of platforms and source systems into the ...

SEP

Data Engineer (in person)

Westfield, IN · On-site

$109K - $131K/yr

Build and maintain data pipelines and transformations (batch and streaming) * Implement data models ... Experience with analytics engineering tools like dbt and data catalog tools (Unity Catalog ...

SEP

Data Engineer (in person)

Westfield, IN · On-site

$109K - $131K/yr

Build and maintain data pipelines and transformations (batch and streaming) * Implement data models ... Experience with analytics engineering tools like dbt and data catalog tools (Unity Catalog ...

The AES Group

Microsoft Fabric Data Engineer

Indianapolis, IN · On-site

$109K - $131K/yr

Design, build, and maintain data pipelines that integrate laboratory, quality, design, and partner ... API & Platform Engineering * Develop and maintain RESTful API and instrument data connectors with ...

SEP

Data Engineer (in person)

Westfield, IN · On-site

$109K - $131K/yr

Build and maintain data pipelines and transformations (batch and streaming) * Implement data models ... Experience with analytics engineering tools like dbt and data catalog tools (Unity Catalog ...

DHL

Data Engineer I

Whiteland, IN · On-site

As a Data Engineer I, you'll design and optimize data pipelines that ensure reliable, accessible information across DHL Supply Chain. You'll develop and maintain data flows, integrate systems using ...

Eli Lilly and Company

Data Engineer

Indianapolis, IN · On-site

$109K - $131K/yr

Responsibilities Data Engineering & Pipeline Architecture * Design, build, and maintain the data pipeline architecture behind the unified CMC Operational Data Hub * Apply strong data modeling ...

Coastal

Data Platform Engineer (Data 360)

Indianapolis, IN · On-site

$109K - $131K/yr

Pipeline Engineering: * Ingestion amp; Streaming Pipelines: Develop, deploy, and maintain real-time streaming and batch data pipelines for ingesting, transforming, and loading high-volume enterprise ...

Cesco

Data Engineer

Jeffersonville, IN · On-site

$109K - $131K/yr

As a Data Engineer at Crescent Electric Supply, you will be responsible for the architecture ... Pipeline Architecture & Development: Design, build, and maintain scalable ETL/ELT data pipelines ...

Cesco

Data Engineer

Noblesville, IN · On-site

$107K - $129K/yr

As a Data Engineer at Crescent Electric Supply, you will be responsible for the architecture ... Pipeline Architecture & Development: Design, build, and maintain scalable ETL/ELT data pipelines ...

CSpring

Cloud Data Engineer

Indianapolis, IN · On-site

$109K - $131K/yr

As a Cloud Data Engineer , you will build the modern Azure data platform that powers analytics, AI ... The pipelines and datasets you build directly support programs that serve real people, not abstract ...

next page

Showing results 1-20

All JobsData Pipeline Engineer JobsOvernight Data Pipeline Engineer JobsOvernight Data Pipeline Engineer Jobs in Indiana

Overnight Data Pipeline Engineer information

What is an overnight data pipeline engineer?

Overnight Data Pipeline Engineers are professionals responsible for managing, monitoring, and optimizing data pipelines during overnight hours. They ensure that data ingestion, transformation, and loading processes run smoothly while most business operations are offline. Their work is crucial for maintaining up-to-date data for analytics and reporting by the start of each business day. These engineers often troubleshoot issues, perform maintenance, and implement improvements to ensure data reliability and efficiency.

What are the key skills and qualifications needed to thrive as an overnight data pipeline engineer?

To thrive as an Overnight Data Pipeline Engineer, you need expertise in data engineering, ETL processes, and programming skills in languages like Python or Scala, often supported by a degree in computer science or a related field. Familiarity with data pipeline tools such as Apache Airflow, Spark, and cloud platforms (AWS, GCP, or Azure), as well as knowledge of database systems, is typically expected. Strong problem-solving abilities, attention to detail, and effective communication are crucial soft skills, especially for resolving issues during off-hours. These skills ensure the reliability, efficiency, and timely delivery of data critical for business operations that depend on overnight data processing.

What are some common challenges faced by overnight data pipeline engineers, and how can they be addressed?

Overnight Data Pipeline Engineers often encounter challenges such as monitoring and troubleshooting data flows during off-hours when fewer team members are available. This requires strong problem-solving skills and the ability to work independently to quickly resolve issues that might impact downstream analytics or operations. Effective communication with both day and night teams is crucial, as is maintaining detailed documentation to ensure smooth handoffs. Proactively setting up automated alerts and robust error handling can help minimize disruptions and ensure data reliability during overnight runs.

What is the difference between Overnight Data Pipeline Engineer vs Data Engineer?

AspectOvernight Data Pipeline EngineerData Engineer
Primary FocusDesigning, building, and maintaining data pipelines that run overnight or during off-peak hoursDeveloping and managing overall data architecture, including pipelines, databases, and data models
Work ScheduleTypically overnight shifts or off-hoursStandard business hours, with flexibility for project deadlines
Skills & CertificationsSQL, ETL tools, cloud platforms, scripting languages, certifications like AWS or GCPData modeling, programming, cloud skills, certifications in data management or cloud platforms

While both roles involve data pipeline development, the Overnight Data Pipeline Engineer specializes in managing pipelines during off-hours, focusing on operational stability. Data Engineers have a broader scope, including designing overall data systems and architecture. Both roles require similar technical skills and certifications, but their work schedules and specific responsibilities differ.

What are the most commonly searched types of Data Pipeline Engineer jobs in Indiana?

The most popular types of Data Pipeline Engineer jobs in Indiana are:

What cities in Indiana are hiring for Overnight Data Pipeline Engineer jobs?

Cities in Indiana with the most Overnight Data Pipeline Engineer job openings:

Overnight Data Pipeline Engineer jobs near you

Data Engineer (in person)

SEP

Westfield, IN • On-site

$100 - $125/hr

Other

Re-posted 14 days ago

Job description

SEP is looking for a data engineer to join our growing data practice and deliver data solutions for our clients. We’ve been helping companies build great software products since 1988 in a wide range of industries and technologies, and more and more of these products are powered by data. Our clients want us to apply the same engineering rigor we bring to software development to their data pipelines and data platforms. You’ll spend your days building, testing, and shipping data systems that real clients depend on.

What we have to offer
  • Variety on every axis — tools, technologies, market sectors, methodologies
  • Flexible, reasonable work schedules
  • Extensive opportunities to learn and develop yourself
  • A community of friendly, talented, and effective peers
  • Opportunities to try out different roles with minimal risk
  • Gorgeous facilities
What you’ll be doing
  • Build and maintain data pipelines and transformations (batch and streaming)
  • Implement data models for analytics use cases
  • Write data quality checks and tests for data pipelines
  • Configure orchestration and workflow tooling to support delivery
  • Implement infrastructure as code for data platforms
  • Investigate and resolve pipeline failures and performance issues
  • Participate in peer code review and contribute to team documentation
  • Communicate progress, blockers, and risks to the team and stakeholders
  • Support client meetings in a technical capacity
Key attributes for applicants
  • A passion for great products, software development, and learning
  • Strong SQL skills with working proficiency in Python; experience with Spark, Scala, R, or C# is a plus
  • Practical experience in data pipeline development, ideally in both batch and streaming patterns
  • Understanding of data modeling patterns for analytics (dimensional modeling, normalized data models, lakehouse concepts)
  • Hands‑On experience with at least one cloud data platform (Azure preferred, AWS and GCP experience also valued)
  • Familiarity with modern data platforms such as Databricks, Snowflake, Microsoft Fabric, or Redshift
  • Familiarity with orchestration and workflow management (Apache Airflow, Databricks Workflows, Temporal, or similar)
  • Experience writing tests and data quality checks for pipelines
  • Exposure to infrastructure as code tooling (Terraform, ARM/Bicep) and CI/CD pipelines (GitHub Actions or similar) is a plus
  • Experience with analytics engineering tools like dbt and data catalog tools (Unity Catalog, Microsoft Purview) is a plus
  • Applies software engineering best practices to data engineering (source control, automated testing, code review, CI/CD)
  • Comfortable with ambiguity; can clarify requirements through conversation
  • Interest in mentoring and developing less experienced engineers
  • Professional data engineering experience (2+ years desired)
  • Must be legally authorized to work in the United States
  • Must not require visa sponsorship or have work authorization based on OPT or CPT
  • Must be able to work from our office in Westfield, IN without relocation financial assistance

We are an equal opportunity employer as to all protected groups, including protected veterans and individuals with disabilities.

#J-18808-Ljbffr
SEP logo

About SEP

Sourced by ZipRecruiter

Industry

Software development

Company size

51 - 200 Employees

Headquarters location

Carmel, IN, US

Year founded

1988

View All SEP Jobs

Apply