1

Overnight Data Pipeline Engineer Jobs in Indiana

Sr. Data Engineer

Indianapolis, IN · On-site

$109K - $131K/yr

Design, develop, and maintain data pipelines and workflows using modern tools and frameworks ... of experience in data engineering or data development * Strong hands-on experience with:

ETL/Data Engineer

Indianapolis, IN · On-site

$107K - $129K/yr

You will own end-to-end delivery of data pipelines and data products that power analytics ... This role is ideal for an engineer with strong hands-on depth in Azure Data Factory, Azure Synapse ...

Data Engineer

Indianapolis, IN · On-site

$109K - $131K/yr

The Data Engineer plays a key role supporting our organization's data ecosystem, contributing to the development, maintenance, and optimization of data pipelines and systems that power sales ...

Data Engineer

Indianapolis, IN · On-site

$109K - $131K/yr

The Data Engineer plays a key role supporting our organization's data ecosystem, contributing to the development, maintenance, and optimization of data pipelines and systems that power sales ...

Data Engineer

Indianapolis, IN · On-site

$85 - $120/hr

The Data Engineer collaborates closely with the Information Technology, Analytics & Digital ... Design and manage ETL/ELT pipelines, data migrations, and data validation processes including data ...

New

Fabric Data Engineer

Fort Wayne, IN · On-site

$104K - $125K/yr

This role focuses on building reliable, performant data pipelines and analytical data products that drive meaningful business insights across the organization. The Fabric Data Engineer brings a ...

Fabric Data Engineer

Fort Wayne, IN · On-site

$104K - $125K/yr

This role focuses on building reliable, performant data pipelines and analytical data products that drive meaningful business insights across the organization. The Fabric Data Engineer brings a ...

Act as the technical bridge across the ES Office of Data pillars--collaborating with Data Governance to embed security rules into pipelines, engineering the high-performance upstream data layers that ...

Sr Databricks Data Engineer

Indianapolis, IN

$109K - $131K/yr

As a Databricks Engineer in our AI & Data practice, you will design, build, and optimize cloud ... Pipeline Ownership: Oversee the design, development, and maintenance of robust data pipelines and ...

Azure Data Engineer

Indianapolis, IN

$109K - $131K/yr

Build monitoring, alerting, and error handling into pipelines from the start, and respond when ... Requirements * 3+ years of hands-on data engineering experience, including 2+ years building on ...

Showing results 41-60

Overnight Data Pipeline Engineer information

What is an overnight data pipeline engineer?

Overnight Data Pipeline Engineers are professionals responsible for managing, monitoring, and optimizing data pipelines during overnight hours. They ensure that data ingestion, transformation, and loading processes run smoothly while most business operations are offline. Their work is crucial for maintaining up-to-date data for analytics and reporting by the start of each business day. These engineers often troubleshoot issues, perform maintenance, and implement improvements to ensure data reliability and efficiency.

What are the key skills and qualifications needed to thrive as an overnight data pipeline engineer?

To thrive as an Overnight Data Pipeline Engineer, you need expertise in data engineering, ETL processes, and programming skills in languages like Python or Scala, often supported by a degree in computer science or a related field. Familiarity with data pipeline tools such as Apache Airflow, Spark, and cloud platforms (AWS, GCP, or Azure), as well as knowledge of database systems, is typically expected. Strong problem-solving abilities, attention to detail, and effective communication are crucial soft skills, especially for resolving issues during off-hours. These skills ensure the reliability, efficiency, and timely delivery of data critical for business operations that depend on overnight data processing.

What are some common challenges faced by overnight data pipeline engineers, and how can they be addressed?

Overnight Data Pipeline Engineers often encounter challenges such as monitoring and troubleshooting data flows during off-hours when fewer team members are available. This requires strong problem-solving skills and the ability to work independently to quickly resolve issues that might impact downstream analytics or operations. Effective communication with both day and night teams is crucial, as is maintaining detailed documentation to ensure smooth handoffs. Proactively setting up automated alerts and robust error handling can help minimize disruptions and ensure data reliability during overnight runs.

What is the difference between Overnight Data Pipeline Engineer vs Data Engineer?

AspectOvernight Data Pipeline EngineerData Engineer
Primary FocusDesigning, building, and maintaining data pipelines that run overnight or during off-peak hoursDeveloping and managing overall data architecture, including pipelines, databases, and data models
Work ScheduleTypically overnight shifts or off-hoursStandard business hours, with flexibility for project deadlines
Skills & CertificationsSQL, ETL tools, cloud platforms, scripting languages, certifications like AWS or GCPData modeling, programming, cloud skills, certifications in data management or cloud platforms

While both roles involve data pipeline development, the Overnight Data Pipeline Engineer specializes in managing pipelines during off-hours, focusing on operational stability. Data Engineers have a broader scope, including designing overall data systems and architecture. Both roles require similar technical skills and certifications, but their work schedules and specific responsibilities differ.

What are the most commonly searched types of Data Pipeline Engineer jobs in Indiana?

The most popular types of Data Pipeline Engineer jobs in Indiana are:

What cities in Indiana are hiring for Overnight Data Pipeline Engineer jobs?

Cities in Indiana with the most Overnight Data Pipeline Engineer job openings:

Overnight Data Center Operations Manager, DCC Communities

Amazon

New Carlisle, IN • On-site

$152K/yr

Full-time

Posted 8 days ago


Amazon rating

7.4

Company rating: 7.4 out of 10

Based on 7,147 frontline employees who took The Breakroom Quiz

5th of 39 rated national retailers


Job description

Manage a team that maintains cloud infrastructure for global data center operations.
AWS is looking for technology managers with experience in people management, strong technical understanding and the drive and vision to take AWS Data Center Operations to the next level. The successful candidate will have experience creating/devising strategies, mentoring people at all levels, sponsoring projects, and proposing technical solutions.

This position involves leading teams in hardware and network diagnostics followed by physical repair and includes participation in an on-call rotation.
AWS Infrastructure Services owns the design, planning, delivery, and operation of all AWS global infrastructure. In other words, we're the people who keep the cloud running.

We support all AWS data centers and all of the servers, storage, networking, power, and cooling equipment that ensure our customers have continual access to the innovation they rely on. We work on the most challenging problems, with thousands of variables impacting the supply chain - and we're looking for talented people who want to help.
You'll join a diverse team of software, hardware, and network engineers, supply chain specialists, security experts, operations managers, and other vital roles.

You'll collaborate with people across AWS to help us deliver the highest standards for safety and security while providing seemingly infinite capacity at the lowest possible cost for our customers. And you'll experience an inclusive culture that welcomes bold ideas and empowers you to own them to completion.
Key job responsibilities
The successful candidate will be operationally responsible for a Data Center.

Some high-level responsibilities include:
- Prioritize and assign trouble tickets to data center technicians and operators
- Manage the performance of a diverse team(s) of data center operation professionals with varying skills across different locations
- Developing Career Paths for the employees and to make updates in our internal tools.
- Recruit and train data technicians to ensure appropriate staffing levels
- Ensure effective and efficient management of day to day data center operations including queue management, 7/24 shift arrangement and hardware logistics
- Fast learn or act as the subject matter expert across all aspects in data center operations
- Ensure all operational KPIs and metrics are being measured and met
- Inspire and guide improvement in team process, technology innovation and automation
- Manage Large Scale Events (outages) and act as the call leader
- Manage and improve the work-flows and through-put for data centers operations
- Recommend, document, and oversee policies and procedures to meet industry best practices and to meet required SLAs
- Maintain the on-call schedule coordinating absence and vacations
- Be passionate about the quality and quantity of services being provided, and continuously strive to improve our Customer Experience
This role supports data centers in various operational stages. The position may be assigned to a facility still under development, which could require travel to established locations for training until your designated site becomes fully operational.
Essential Requirements:
- Ability to travel to or commute between data center locations as needed
- Willingness to temporarily work at alternative sites during training periods or until assigned facility is operational
- Travel frequency will vary based on business needs and operational status of assigned facility
Candidates must be able to travel (over 50 miles) or commute (less than 50 miles) to active data center sites as required by business needs


A day in the life
Amazon Web Services (AWS) is a fast paced technology company and a leader in the world of data centers. The position will help ensure overall availability and reliability to meet or exceed defined service levels of data center operations. We are currently expanding our infrastructure management team and are looking for technology managers with experience in people management, strong technical understanding and the drive and vision to take AWS Data Center Operations to the next level.


The successful candidate will have experience creating/devising strategies, mentoring people in all levels, sponsoring projects and proposing technical solutions. This position involves leading teams in hardware and network diagnostics followed by physical repair and includes participation in an on-call rotation. All candidates should be willing to work both independently and managing their teams effectively.

Work prioritization, organizational skills, effective communication, and the ability to react quickly are critical to being successful. In addition to strong knowledge in Data Centers, hardware and network repair, candidates will create documentation, innovate solutions, participate in site-level project work, and fix complex problems within the Data Center Space. This team works in an environment that operates 24/7.


What Amazon employees say

Pay

Benefits

Hours and flexibility

Workplace

Get the full story on Breakroom


Amazon logo

About Amazon

Sourced by ZipRecruiter

Amazon.com, Inc., commonly known as Amazon, is an American multinational technology company. It was founded by Jeff Bezos in 1994 and initially started as an online marketplace for books. Since then, Amazon has expanded its operations and become one of the largest e-commerce companies in the world. Amazon's primary business is its online retail platform, where customers can purchase a vast array of products, including electronics, clothing, books, home goods, and much more. The company offers a convenient and user-friendly shopping experience, with features such as fast shipping, customer reviews, and personalized recommendations. In addition to its e-commerce platform, Amazon has diversified its business into various other areas. One of its notable ventures is Amazon Web Services (AWS), a comprehensive cloud computing platform that provides services such as storage, compute power, and database management to individuals and businesses. AWS has become a leader in the cloud computing industry, powering many websites and applications worldwide. Amazon has also developed its own consumer electronics, including the popular Amazon Kindle e-reader, Fire tablets, Fire TV streaming devices, and the Alexa-powered Echo smart speakers. The Alexa voice assistant, integrated into these devices, allows users to interact with their devices using voice commands, perform tasks, and access information. Furthermore, Amazon has expanded into media and entertainment. It operates Prime Video, a streaming service that offers a wide range of movies, TV shows, and original content. Amazon Music provides a platform for streaming and purchasing digital music, while Audible offers audiobooks and other audio content. The company's commitment to customer satisfaction and convenience is demonstrated by its membership program, Amazon Prime. Prime members receive various benefits, including free two-day shipping, access to streaming services, exclusive deals, and more.

Industry

It services, book publishers, retail, real estate, computer and electronic product manufacturing and software development

Company size

10,000+ Employees

Headquarters location

Seattle, WA, US