1

Overnight Data Pipeline Engineer Jobs in Indiana

Senior Data Platform Engineer

Carmel, IN · On-site

$67.25 - $89.75/hr

Design and implement sophisticated data pipeline architectures from various ERP and source systems, establishing patterns and best practices for the Data Hub. * Partner with the Data Architect on ...

Data Systems/Solutions Engineer

Indianapolis, IN · On-site

$109K - $131K/yr

The Engineer applies modern software engineering and data engineering practices to ensure data ... Design, build, and maintain data platforms, pipelines, and services that support research ...

Cloud Data Engineer

Indianapolis, IN · On-site

$109K - $131K/yr

As a Cloud Data Engineer , you will build the modern Azure data platform that powers analytics, AI ... The pipelines and datasets you build directly support programs that serve real people, not abstract ...

Cloud Data Engineer

Indianapolis, IN · On-site

$109K - $131K/yr

As a Cloud Data Engineer , you will build the modern Azure data platform that powers analytics, AI ... The pipelines and datasets you build directly support programs that serve real people, not abstract ...

Cloud Data Engineer

Indianapolis, IN

$109K - $131K/yr

As a Cloud Data Engineer , you will build the modern Azure data platform that powers analytics, AI ... The pipelines and datasets you build directly support programs that serve real people, not abstract ...

Data Engineer

Indianapolis, IN · On-site +1

$109K - $131K/yr

About The Role We are seeking a Data Engineer to design, build, and maintain the data pipelines and infrastructure that power K1x's machine learning initiatives and future analytics-driven product ...

Data Engineer

Indianapolis, IN · On-site +1

$109K - $131K/yr

About The Role We are seeking a Data Engineer to design, build, and maintain the data pipelines and infrastructure that power K1x's machine learning initiatives and future analytics-driven product ...

Data Engineer

Carmel, IN · On-site

$108K - $130K/yr

Utilize CI/CD pipelines (Azure DevOps, Git-based workflows, Fabric deployment pipelines, etc.) to ... Develop scalable data models for investment reporting, portfolio analytics, and capital markets ...

Data Engineer

Carmel, IN

$108K - $130K/yr

Utilize CI/CD pipelines (Azure DevOps, Git-based workflows, Fabric deployment pipelines, etc.) to ... Develop scalable data models for investment reporting, portfolio analytics, and capital markets ...

Data Engineer

Indianapolis, IN · On-site

$109K - $131K/yr

About The Role We are seeking a Data Engineer to design, build, andmaintainthe data pipelines and infrastructure that power K1x's machine learning initiatives and future analytics-driven product ...

New

Sr. Data Engineer

Indianapolis, IN · On-site

$109K - $131K/yr

Delta Faucet Company is seeking a Senior Data Engineer to lead the design, development, and scaling of enterprise data pipelines that power omni-retail analytics, advanced modeling, and AI/ML ...

Data Engineer

Fort Wayne, IN · On-site

$104K - $125K/yr

Overview We are seeking an experienced Data Engineer to join the Flat Roll Group IT team, which ... In this role, you will design and manage the data infrastructure - pipelines, databases, and ...

Data Engineer

Fort Wayne, IN

$104K - $125K/yr

Overview We are seeking an experienced Data Engineer to join the Flat Roll Group IT team, which ... In this role, you will design and manage the data infrastructure -- pipelines, databases, and ...

Data Engineer

Fort Wayne, IN · On-site

$104K - $125K/yr

Overview We are seeking an experienced Data Engineer to join the Flat Roll Group IT team, which ... In this role, you will design and manage the data infrastructure - pipelines, databases, and ...

Data Analytics Engineer

Kokomo, IN · On-site

$101K - $121K/yr

This role is responsible for designing, implementing, and maintaining data pipelines, analytics systems, and reporting solutions. The analytics engineer also models raw data into clean, tested, and ...

Showing results 21-40

Overnight Data Pipeline Engineer information

What are some common challenges faced by overnight data pipeline engineers, and how can they be addressed?

Overnight Data Pipeline Engineers often encounter challenges such as monitoring and troubleshooting data flows during off-hours when fewer team members are available. This requires strong problem-solving skills and the ability to work independently to quickly resolve issues that might impact downstream analytics or operations. Effective communication with both day and night teams is crucial, as is maintaining detailed documentation to ensure smooth handoffs. Proactively setting up automated alerts and robust error handling can help minimize disruptions and ensure data reliability during overnight runs.

What is the difference between Overnight Data Pipeline Engineer vs Data Engineer?

AspectOvernight Data Pipeline EngineerData Engineer
Primary FocusDesigning, building, and maintaining data pipelines that run overnight or during off-peak hoursDeveloping and managing overall data architecture, including pipelines, databases, and data models
Work ScheduleTypically overnight shifts or off-hoursStandard business hours, with flexibility for project deadlines
Skills & CertificationsSQL, ETL tools, cloud platforms, scripting languages, certifications like AWS or GCPData modeling, programming, cloud skills, certifications in data management or cloud platforms

While both roles involve data pipeline development, the Overnight Data Pipeline Engineer specializes in managing pipelines during off-hours, focusing on operational stability. Data Engineers have a broader scope, including designing overall data systems and architecture. Both roles require similar technical skills and certifications, but their work schedules and specific responsibilities differ.

What are the key skills and qualifications needed to thrive as an overnight data pipeline engineer?

To thrive as an Overnight Data Pipeline Engineer, you need expertise in data engineering, ETL processes, and programming skills in languages like Python or Scala, often supported by a degree in computer science or a related field. Familiarity with data pipeline tools such as Apache Airflow, Spark, and cloud platforms (AWS, GCP, or Azure), as well as knowledge of database systems, is typically expected. Strong problem-solving abilities, attention to detail, and effective communication are crucial soft skills, especially for resolving issues during off-hours. These skills ensure the reliability, efficiency, and timely delivery of data critical for business operations that depend on overnight data processing.

What is an overnight data pipeline engineer?

Overnight Data Pipeline Engineers are professionals responsible for managing, monitoring, and optimizing data pipelines during overnight hours. They ensure that data ingestion, transformation, and loading processes run smoothly while most business operations are offline. Their work is crucial for maintaining up-to-date data for analytics and reporting by the start of each business day. These engineers often troubleshoot issues, perform maintenance, and implement improvements to ensure data reliability and efficiency.

What are the most commonly searched types of Data Pipeline Engineer jobs in Indiana?

The most popular types of Data Pipeline Engineer jobs in Indiana are:

What cities in Indiana are hiring for Overnight Data Pipeline Engineer jobs?

Cities in Indiana with the most Overnight Data Pipeline Engineer job openings:

Senior Data Platform Engineer

Allegion

Carmel, IN • On-site

$67.25 - $89.75/hr

Full-time

Medical, Dental, Vision, Life, Retirement, PTO

Re-posted 5 days ago


Allegion rating

8.3

Company rating: 8.3 out of 10

Based on 46 frontline employees who took The Breakroom Quiz

67th of 538 rated manufacturers


Job description

Creating Peace of Mind by Pioneering Safety and Security

At Allegion, we help keep the people you know and love safe and secure where they live, work and visit. With more than 30 brands, 12,000+ employees globally and products sold in 130 countries, we specialize in security around the doorway and beyond.Additionally, in 2024 we were awarded the Gallup Exceptional Workplace Award, which recognizes the most engaged workplace cultures in the world.

Senior Data Platform Engineer

This position serves as the senior technical expert for Data Hub processes within the Global Data & Analytics Team, owning the design and implementation of data ingestion, transformation, and transmission workflows for the Operational Data Hub. The Senior Data Platform Engineer is the go-to authority for Data Hub architecture and processes, while actively contributing technical expertise to enterprise data structure design in partnership with the Data Architect. This role mentors mid-level engineers, collaborates with Lead Engineers and cross-functional stakeholders, and balances deep technical expertise (80%) with informal leadership and mentorship (20%).

Qualified candidates must be legally authorized to be employed in the United States. The company does not intend to provide sponsorship for employment visa status (e.g., H-1B, TN, etc.) for this employment position.

What You Will Do:

  • Own and architect Data Hub processes and workflows, including data ingestion pipelines, transformation logic, transmission patterns, and integration orchestration.
  • Design and implement sophisticated data pipeline architectures from various ERP and source systems, establishing patterns and best practices for the Data Hub.
  • Partner with the Data Architect on enterprise data structure design, contributing Data Hub process expertise, feasibility assessments, and technical requirements.
  • Define Data Hub technical standards and patterns for pipeline development, error handling, monitoring, and operational workflows.
  • Collaborate with the Data Architect to ensure alignment between Data Hub processes and enterprise data models, providing input on how data structures impact processing efficiency.
  • Lead technical design sessions for Data Hub initiatives, making architectural decisions on ingestion patterns, transformation approaches, and integration strategies.
  • Contribute technical expertise to data structure discussions, advising on pipeline performance implications, transformation complexity, and operational considerations.
  • Provide technical mentorship to mid-level Data Platform Engineers on Data Hub architecture, patterns, and implementation best practices.
  • Drive complex problem-solving for Data Hub operational issues, including performance optimization, pipeline troubleshooting, and root cause analysis.
  • Implement advanced transformation and enrichment frameworks that leverage enterprise data structures designed by the Data Architect.
  • Own end-to-end architecture of Data Hub components, including serverless compute, orchestration, monitoring, and operational tooling.
  • Evaluate and prototype emerging technologies for Data Hub capabilities, providing recommendations on tools and platforms for data processing.
  • Establish data quality frameworks for Data Hub processes, implementing validation, testing, and monitoring strategies in collaboration with the Data Architect.
  • Design and implement Data Hub governance controls, including security, compliance, and operational best practices for data processing workflows.
  • Optimize Data Hub performance and cloud resource utilization, identifying opportunities for efficiency improvements and cost savings.
  • Document Data Hub architectures and technical patterns comprehensively, creating runbooks, design guides, and knowledge base articles.
  • Participate in on-call rotation for Data Hub production support, leading incident resolution and conducting technical post-mortems.

What You Need to Succeed:

  • 5-7 years of experience designing and implementing data hub solutions, operational data platforms, or similar data processing architectures.
  • Expert-level experience owning data pipeline architectures, including event-driven patterns, real-time processing, and batch workflows.
  • Advanced expertise architecting serverless solutions with Azure Function Apps or equivalent compute services for data processing at scale.
  • Deep experience with data hub processes including ingestion patterns, transformation frameworks, orchestration, and operational monitoring.
  • Advanced experience designing and optimizing Azure SQL Databases or similar platforms for operational data storage within hub architectures.
  • Expert SQL skills with proven ability to optimize queries, design efficient processing logic, and contribute to data modeling discussions.
  • Strong understanding of enterprise data architecture principles, with ability to contribute technical expertise to data structure and modeling decisions.
  • Proven ability to collaborate with Data Architects, providing Data Hub process perspective on data structure designs and enterprise models.
  • Demonstrated expertise in data integration patterns involving ERP systems, APIs, and enterprise application data flows.
  • Experience with Azure Data Lake, Azure Synapse Analytics, or similar platforms, understanding how Data Hub processes interact with enterprise data layers.
  • Proven track record of owning complex Data Hub architectures and delivering technical solutions independently.
  • Strong mentorship capabilities with experience guiding engineers on Data Hub patterns, pipeline development, and operational best practices.
  • Excellent problem-solving skills for Data Hub operational challenges, including performance tuning, troubleshooting, and reliability improvements.
  • Experience with CI/CD pipelines and Infrastructure as Code for automating Data Hub deployments and configuration management.
  • Strong understanding of data security and compliance requirements, implementing controls within Data Hub processes.
  • Excellent technical communication skills to collaborate with Data Architects, document architectures, and explain Data Hub technical tradeoffs.
  • Experience contributing to architectural discussions across data structures and processes, providing balanced technical perspectives.
  • Self-motivated with strong organizational skills, able to own Data Hub initiatives while contributing to broader architectural decisions.
  • Deep experience with DataOps/DevOps practices, including monitoring, logging, observability, and operational excellence for data systems.
  • Knowledge of cloud cost optimization for data processing workloads and efficient resource utilization.
  • Bachelor's degree in computer science, Information Technology, Engineering, or related technical discipline, or equivalent practical experience.

Why Work for Us?

Allegion is a Great Place to Grow your Career if:

  • You're seeking a rewarding opportunity that allows you to truly help others. With thousands of employees and customers around the world, there's plenty of room to make an impact. As our values state, "this is your business, run with it".
  • You're looking for a company that will invest in your professional development. As we grow, we want you to grow with us.
  • You want a culture that promotes work-life balance. Our employees enjoy generous paid time off, because at Allegion we recognize that you have a full life outside of work!
  • You want to work for an award-winning company that invests in its people. Allegion is proud to be a recipient of the Gallup Exceptional Workplace Award for the second year in a row, recognizing our commitment to employee engagement, strengths-based development, and unlocking human potential.

What You'll Get from Us:

  • Health, dental and vision insurance coverage, helping you "be safe, be healthy".
  • A commitment to your future with a 401K plan, offering a 6% company match and no vesting period
  • Tuition Reimbursement
  • Unlimited PTO
  • Employee Discounts through Perks at Work
  • Community involvement and opportunities to give back so you can "serve others, not yourself"
  • Opportunities to leverage your unique strengths through CliftonStrengths testing and coaching

Apply Today!

Join our team of experts today and help us make tomorrow's world a safer place!

Not sure if your experience perfectly aligns with the role?Studies have shown that some people are less likely to apply to jobs unless they meet every single qualification and every single preferred qualification of a job posting. At Allegion, we are dedicated to building a diverse, inclusive, and authentic workplace. So, if you're excited about this role but your past experience doesn't align perfectly with every item in the job description, we encourage you to apply anyway. You may be just the right candidate for this role.

We Celebrate Who We Are!

Allegion is committed to building and maintaining a diverse and inclusive workplace. Together, we embrace all differences and similarities among colleagues, as well as the differences and similarities within the relationships that we foster with customers, suppliers and the communities where we live and work. Whatever your background, experience, race, color, national origin, religion, age, gender, gender identity, disability status, sexual orientation, protected veteran status, or any other characteristic protected by law, we will make sure that you have every opportunity to impress us in your application and the opportunity to give your best at work, not because we're required to, but because it's the right thing to do. We are also committed to providing accommodations for persons with disabilities. If for any reason you cannot apply through our career site and require an accommodation or assistance, please contact our Talent Acquisition Team.

Allegion plc, 2023 | Block D, Iveagh Court, Harcourt Road, Dublin 2, Co. Dublin, Ireland

REGISTERED IN IRELAND WITH LIMITED LIABILITY REGISTERED NUMBER 527370

Allegion is an equal opportunity and affirmative action employer

Privacy Policy


What Allegion employees say

Pay

Benefits

Hours and flexibility

Workplace

Get the full story on Breakroom