1

Overnight Databricks Data Engineer Jobs in Wisconsin

Data Engineering: Develop scalable, welldocumented ETL/ELT pipelines using TSQL, Python, Azure Data Factory/Fabric Data Pipelines, and Databricks; implement bestpractice patterns for performance ...

Data Engineering: Develop scalable, welldocumented ETL/ELT pipelines using TSQL, Python, Azure Data Factory/Fabric Data Pipelines, and Databricks; implement bestpractice patterns for performance ...

... Engineer - Senior Associate, you will focus on designing and building data infrastructure and ... Databricks Unified Data Analytics Platform for advanced data analytics and visualization ...

Sr. Data Engineer

Madison, WI · On-site

$115K - $138K/yr

As a Senior Data Engineer, this seasoned professional will demonstrate competence and creativity in ... Spark on Snowflake or Databricks. * Python, Scala, SQL development. * ETL data pipelines.

New

The Senior Data Engineer is responsible for software development and data engineering projects ... Expert at developing complex software and data pipelines in Python, Airflow, DBT, Databricks, C# ...

Sr. Data Engineer

Madison, WI · On-site

$115K - $138K/yr

As a Senior Data Engineer, this seasoned professional will demonstrate competence and creativity in ... Spark on Snowflake or Databricks. * Python, Scala, SQL development. * ETL data pipelines.

New

Data Strategy-Manager

Milwaukee, WI · On-site

$99K - $232K/yr

... Databricks Certified Data Engineer / Data Analyst / ML - Proven leadership in data-driven strategies - Experience in defining data governance frameworks - Understanding of modern cloud data ...

Software Engineer, Data

Stevens Point, WI · On-site

$111K - $133K/yr

Software Engineer, Data Austin, Texas (Hybrid: Four Days in Office) Note: This is a full ... Our data storage layer includesBigQuery, Databricks, Azure, and Postgres. Across our broader tech ...

Data Governance- Manager

Milwaukee, WI · On-site

$99K - $232K/yr

... Databricks Certified Data Engineer / Data Analyst / ML Travel Requirements Up to 80% Job Posting End Date The salary range for this position is: $99,000 - $232,000. Actual compensation within the ...

Solutions Architect (Cloud)

Madison, WI · On-site

$155K - $175K/yr

Partner with engineering teams to implement Infrastructure as Code, CI/CD pipelines, and cloud ... Familiarity with modern data platforms, including Databricks, data lakes, data warehouses, and ...

next page

Showing results 1-20

Overnight Databricks Data Engineer information

What is the difference between Overnight Databricks Data Engineer vs Data Engineer?

AspectOvernight Databricks Data Engineer
Work EnvironmentPrimarily remote or on-site, working overnight shifts to support global data operations
CertificationsDatabricks certifications, cloud platform credentials (AWS, Azure), data engineering certifications
Tools & TechnologiesDatabricks platform, Spark, cloud services, SQL, Python, ETL tools
Industry UsageTech, finance, healthcare, retail with 24/7 data needs

While both roles focus on data engineering, the Overnight Databricks Data Engineer specializes in managing data pipelines on the Databricks platform during overnight shifts, often supporting global operations. A Data Engineer may work across various platforms and shifts, with broader responsibilities in data architecture and pipeline development. The overnight role emphasizes specific platform expertise and shift timing, catering to organizations with continuous data processing needs.

What are some unique challenges faced by Overnight Databricks Data Engineers, and how can they be addressed?

Overnight Databricks Data Engineers often work with limited real-time support, which can present challenges when troubleshooting urgent data pipeline issues or system outages. To address this, it’s essential to develop strong problem-solving skills, document processes thoroughly, and leverage automated monitoring and alerting tools. Additionally, close collaboration with daytime teams during handoff periods ensures continuity and minimizes disruptions. Building a habit of proactive communication and maintaining detailed logs helps the entire team resolve issues efficiently and maintain data quality.

What are the key skills and qualifications needed to thrive as an Overnight Databricks Data Engineer, and why are they important?

To thrive as an Overnight Databricks Data Engineer, you need strong proficiency in data engineering, Python or Scala programming, and experience with big data technologies, typically supported by a relevant degree in computer science or a related field. Familiarity with Databricks, Apache Spark, cloud platforms (such as AWS or Azure), and certifications like Databricks Certified Data Engineer are highly valued. Attention to detail, problem-solving, and effective communication are essential soft skills, especially for troubleshooting and collaborating across shifts. These competencies ensure reliable data pipeline management and efficient resolution of issues during off-hours, maintaining seamless business operations.

What is an Overnight Databricks Data Engineer?

An Overnight Databricks Data Engineer is a professional who works primarily during night shifts to manage, design, and maintain big data pipelines and workflows using Databricks, a cloud-based data analytics platform. Their responsibilities often include developing and optimizing data processing jobs, ensuring data quality, and troubleshooting issues that arise during overnight data operations. This role is critical for organizations that require 24/7 data processing, continuous ETL jobs, or real-time analytics. Working overnight may also involve monitoring automated systems, performing scheduled data loads, and collaborating with global teams to ensure data availability and reliability.
What are the most commonly searched types of Databricks Data Engineer jobs in Wisconsin? The most popular types of Databricks Data Engineer jobs in Wisconsin are:
What cities in Wisconsin are hiring for Overnight Databricks Data Engineer jobs? Cities in Wisconsin with the most Overnight Databricks Data Engineer job openings:
Threat Detection & Automation Engineer (with focus on Data Engineering)

Threat Detection & Automation Engineer (with focus on Data Engineering)

Northwestern Mutual Life Insurance Company

Milwaukee, WI • On-site

Full-time

Posted 16 hours ago

Posted today


Northwestern Mutual rating

8.0

Company rating: 8.0 out of 10

Based on 73 frontline employees who took The Breakroom Quiz

145th of 281 rated insurance


Job description

About the Job:

We are looking for a highly motivated Security Automation Engineerwith experience in Data Engineeringto join NM'sSecurity Automation & Data Analyticsteam. As a member ofourteam, you willplay a pivotal role indesigning,building andmaintainingcontinuous and sustainableautomatedpipelineswith the goal of enhancing the efficiency, effectiveness and accuracy ofcybersecurity teams'capabilitiesreducing manual tasks fortheIRCteamsand enable data-driven cybersecurity operations.This role requires a mix ofmodern data engineering practices, experience in database structures and tooling,coding, clouddevelopment,and security domainexpertise.

What You'll Do:

  • Design, develop, andmaintainscalable data pipelines, datasets, and analytics products supportingIRCdepartment'sreporting and analyticscapabilities.
  • Developandmaintainenterprise-ready ETL/ELT processes and data transformation frameworksto support data-driven initiatives.
  • Design, buildand supportcloud-based data pipelinesand data lake architecturesusing software and data engineering best practices.
  • Ensure data quality, lineage, integrity, and governance standards are implemented throughout the data lifecycle.
  • Optimizeand troubleshoot existing data pipelines for performance and reliability, ensuring data quality and integrity across various data sources.
  • Identifyopportunities to automate workflows/processesand improve efficiency across cybersecurity teamsthrough SOAR playbooksandKubernetes-hostedand/orcloud-hosted pipelines.
  • Assistwith the design,engineering, deployment, andmaintenance ofcustom automation tools(primarilyinPython and PowerShell)
  • Develop integrationswithsecurity toolsand APIs.
  • Evaluate and implement emerging technologies to improve cybersecurity and analytics capabilities
  • Assistwith the developmentand supportofdashboards,metrics,andmonitoringalerts related totheteam's and the department'soperations.
  • Develop andmaintaindocumentation related totheteam's automated workflows/products.
  • Partner withdata scientists,analysts, engineers, andotherstakeholderstoidentifyrequirementsanddeliverhigh-quality solutions and insights.
  • Provide mentorship and coaching to junior team members, fostering skill development and encouraging continuous learning.

What You'll Bring to the Role:

  • 5+years of professional development experienceusingSQLandPython.
  • 5+years of professionalexperiencebuilding datapipelines(batch and/or streaming)and datagovernancepractices.
  • Hands-on experience with modern datawarehousesand large-scale data platforms; comfortable working across large-scale,structuredand unstructured data.
  • Deep understanding of database structures principles and best practices
  • Deep understanding of data modeling,ETL/ELT design,data engineering best practices,including dataintegrationpatterns, replication,eventstreaming,and pipeline performance tuning.
  • Hands-on experience withsecurity automation and orchestration tools and/or software development technologies.
  • Comfortable with REST API best practices and usage.
  • Hands-on experience deploying contentand/or systems through CI/CDenvironments, and tooling such as GitLab, GitLab CI, Terraform, Ansible,Kubernetesetc.
  • Hands-on experience with AWS and cloud services configuration and development.
  • Excellent written andverbalcommunicationsskills.
  • Ability to communicatewithtechnical and non-technical audiences.

Preferred Skills:

  • Bachelor's degree in computer science, Cybersecurity, ComputerEngineering,Data Engineering,Software Engineering, or related fields
  • Relevant industry certifications are preferred, such as AWS Certified Data Engineer - Associate, Databricks Certified Data Engineer Associate, or Databricks Certified Data Engineer Professional, demonstrating expertise in modern cloud-based data engineering, analytics, and data platform technologies.
  • Experience withDatabricks
  • Experience with Splunk SOAR (Phantom) and Splunk SIEM.
  • Experience with Splunk SPL queries, dashboards, and alerts.
  • Experience with ServiceNow developmentand ServiceNow data lake
  • Practical experience with automation techniquesusingPowerShell, Bash,JavaScript,API calls, and regular expressions
  • Additionalcoding experience in one or more programming languages - Java, C++, Go
  • ServiceNowdevelopment experience
  • Experience with network and/or systems administration
  • Experience working in Security Operations, Incident Response, or Penetration Testing
  • Hands-on experience with security technologies such as SIEM, firewalls, IDR/IPS, EDR, IAMprinciplesand others.

What Sets you apart:

  • Adaptive Communication - Adaptive Communication: Formulates strategies to be used to convey complex information about services, products, systems, or processes to targeted audiences; communicates and liaises between technical and non-technical audiences.
  • Data Architecture - Data Architecture: Designs and maintains the structure of logical and physical data assets and data management resources; develops and governs policies and processes for data collection, storage, arrangement, integration, and usage of data.
  • Data ETL - Data ETL: Extracts, transforms, and loads data from multiple sources to a data warehouse or unified data repository.
  • Databases & Data Platforms - Databases & Data Platforms: Utilizes knowledge of databases to access, manage, and update information, typically containing aggregations of data records or files; includes understanding of different types of databases.
  • Engineering Expertise & Practices - Engineering Expertise & Practices: Applies specialized experiences in different facets of engineering, including data, applications, cyber, systems, operations, product, security, and testing, along with technical aptitude to adapt new expertise as they become relevant through an understanding of underlying engineering principles.
  • Programming Languages - Programming Languages: Demonstrates proficiency in one or more programming languages to execute activities, tasks, practices, and deliverables associated with writing and modifying programs and scripts that comprise an application system; designs, codes, tests, and installs complex computer programs and maintains detailed documentation of programming tasks.
  • SQL - SQL: Applies clear, logical codes for small and large-scale projects using the language constructs and object-oriented approach of SQL to manage data in a database management system.

#LI-Hybrid

Compensation Range:

Pay Range - Start:

$130,800.00

Pay Range - End:

$196,200.00

Geographic Specific Pay Structure:

Structure 110:

$143,920.00 USD - $215,880.00 USD

Structure 115:

$150,400.00 USD - $225,600.00 USD

We believe in fairness and transparency. It's why we share the salary range for most of our roles. However, final salaries are based on a number of factors, including the skills and experience of the candidate; the current market; location of the candidate; and other factors uncovered in the hiring process. The standard pay structure is listed but if you're living in California, New York City or other eligible location, geographic specific pay structures, compensation and benefits could be applicable, click here to learn more.


Grow your career with a best-in-class company that puts our clients' interests at the center of all we do. Get started now!


Northwestern Mutual is an equal opportunity employer that welcomes talented individuals of all backgrounds. We are committed to creating and maintaining an environment in which each employee can contribute creative ideas, seek challenges, assume leadership and continue to focus on meeting and exceeding business and personal objectives.



What Northwestern Mutual employees say

Pay

Benefits

Hours and flexibility

Workplace

Get the full story on Breakroom


Northwestern Mutual logo

About Northwestern Mutual

Sourced by ZipRecruiter

Northwestern Mutual has been helping families and businesses achieve financial security for over 160 years through a distinctive planning approach that integrates risk management with wealth accumulation, preservation, and distribution. With more than $290 billion in assets, $30 billion in revenues and more than $1.9 trillion worth of life insurance protection in force, Northwestern Mutual delivers financial security to more than 4.6 million clients. People are the power behind Northwestern Mutual, and diversity makes us better. We are committed to reflecting and serving the marketplace. We do so by attracting and improving the engagement of those who bring their outstanding perspectives, ideas, and beliefs.

Industry

Finance and insurance

Company size

5,001 - 10,000 Employees

Headquarters location

Milwaukee, WI, US