ZipRecruiter

Log In

1

Overnight Databricks Data Engineer Jobs in Pennsylvania

Synergy Staffing

Data Engineer - Fabric

Wexford, PA · On-site

$108K - $130K/yr

Synergy Staffing is seeking a Databricks Data Engineer to develop and support data pipelines and analytics environments in Azure cloud-based data lake. The role involves translating business ...

System One

Data Engineer - Databricks

Pittsburgh, PA · On-site

$107K - $128K/yr

Data Engineer - Azure Databricks Contract-to-Hire Pittsburgh, PA - Onsite Job ID J0726-0569 Visa : (No Sponsorship) Position Overview * Seeking an experienced Data Engineer to join a high-performing ...

System One

Data Engineer - Databricks

Pittsburgh, PA · On-site

$111K - $133K/yr

Data Engineer - Azure Databricks Contract-to-Hire Pittsburgh, PA - Onsite Job ID J0726-0569 Visa : USC, GC, EAD (No Sponsorship) Position Overview * Seeking an experienced Data Engineer to join a ...

System One

Data Engineer - Databricks

Pittsburgh, PA · On-site

$111K - $133K/yr

Data Engineer - Azure Databricks Contract-to-Hire Pittsburgh, PA - Onsite Job ID J0726-0569 Visa : USC, GC, EAD (No Sponsorship) Position Overview * Seeking an experienced Data Engineer to join a ...

System One

Data Engineer - Databricks

Pittsburgh, PA · On-site

$111K - $133K/yr

Data Engineer - Azure Databricks Contract-to-Hire Pittsburgh, PA - Onsite Job ID J0726-0569 Visa : USC, GC, EAD (No Sponsorship) Position Overview * Seeking an experienced Data Engineer to join a ...

System One

Data Engineer - Databricks

Pittsburgh, PA · On-site

$111K - $133K/yr

Data Engineer - Databricks Duration : Contract to Hire Location : Pittsburgh, PA Work Mode : 5 Days Onsite Join a high performing data engineering team responsible for building modern, cloud native ...

System One

Data Engineer - Databricks

Pittsburgh, PA · On-site

$111K - $133K/yr

Data Engineer - Azure Databricks Contract-to-Hire Pittsburgh, PA - Onsite Job ID J0726-0569 Visa : USC, GC, EAD (No Sponsorship) Position Overview * Seeking an experienced Data Engineer to join a ...

SysMind Tech

Data Engineer - Databricks

Pittsburgh, PA · On-site

$108K - $130K/yr

Role: Data Engineer Location: Warrendale, PA / Pittsburgh, PA (Onsite) Job Type: Contract Role Overview We are seeking an experienced Data Engineer with strong expertise in Databricks, Python, and ...

Amerihealth Caritas

Data Engineer

Newtown Square, PA · Hybrid

$109K - $131K/yr

Role Overview The Azure Data Engineer designs, builds, and optimizes large scale cloud data ... Tune Databricks jobs and clusters (cluster sizing, autoscaling, caching, partitioning, shuffle ...

Amerihealth Caritas

Data Engineer

Newtown Square, PA · Hybrid

$109K - $131K/yr

Role Overview The Azure Data Engineer designs, builds, and optimizes large scale cloud data ... Tune Databricks jobs and clusters (cluster sizing, autoscaling, caching, partitioning, shuffle ...

QVC

Data Engineer

West Chester, PA · Remote

$108K - $130K/yr

Experience using Azure tools (Databricks, Data Factory, Delta Lake, Synapse, ADLS, PySpark, SQL ... g., Azure DevOps). * Experience in data warehouse design (e.g., slow changing dimensions ...

Qurate Retail Group

Data Engineer

West Chester, PA · On-site

$108K - $130K/yr

Experience using Azure tools (Databricks, Data Factory, Delta Lake, Synapse, ADLS, PySpark, SQL ... g., Azure DevOps). * Experience in data warehouse design (e.g., slow changing dimensions ...

next page

Showing results 1-20

All JobsDatabricks Data Engineer JobsOvernight Databricks Data Engineer JobsOvernight Databricks Data Engineer Jobs in Pennsylvania

Overnight Databricks Data Engineer information

What is the difference between Overnight Databricks Data Engineer vs Data Engineer?

AspectOvernight Databricks Data Engineer
Work EnvironmentPrimarily remote or on-site, working overnight shifts to support global data operations
CertificationsDatabricks certifications, cloud platform credentials (AWS, Azure), data engineering certifications
Tools & TechnologiesDatabricks platform, Spark, cloud services, SQL, Python, ETL tools
Industry UsageTech, finance, healthcare, retail with 24/7 data needs

While both roles focus on data engineering, the Overnight Databricks Data Engineer specializes in managing data pipelines on the Databricks platform during overnight shifts, often supporting global operations. A Data Engineer may work across various platforms and shifts, with broader responsibilities in data architecture and pipeline development. The overnight role emphasizes specific platform expertise and shift timing, catering to organizations with continuous data processing needs.

What unique challenges do overnight Databricks data engineers face, and how can they be addressed?

Overnight Databricks Data Engineers often work with limited real-time support, which can present challenges when troubleshooting urgent data pipeline issues or system outages. To address this, it’s essential to develop strong problem-solving skills, document processes thoroughly, and leverage automated monitoring and alerting tools. Additionally, close collaboration with daytime teams during handoff periods ensures continuity and minimizes disruptions. Building a habit of proactive communication and maintaining detailed logs helps the entire team resolve issues efficiently and maintain data quality.

What skills and qualifications are needed to thrive as an overnight Databricks data engineer?

To thrive as an Overnight Databricks Data Engineer, you need strong proficiency in data engineering, Python or Scala programming, and experience with big data technologies, typically supported by a relevant degree in computer science or a related field. Familiarity with Databricks, Apache Spark, cloud platforms (such as AWS or Azure), and certifications like Databricks Certified Data Engineer are highly valued. Attention to detail, problem-solving, and effective communication are essential soft skills, especially for troubleshooting and collaborating across shifts. These competencies ensure reliable data pipeline management and efficient resolution of issues during off-hours, maintaining seamless business operations.

What is an overnight Databricks data engineer?

An Overnight Databricks Data Engineer is a professional who works primarily during night shifts to manage, design, and maintain big data pipelines and workflows using Databricks, a cloud-based data analytics platform. Their responsibilities often include developing and optimizing data processing jobs, ensuring data quality, and troubleshooting issues that arise during overnight data operations. This role is critical for organizations that require 24/7 data processing, continuous ETL jobs, or real-time analytics. Working overnight may also involve monitoring automated systems, performing scheduled data loads, and collaborating with global teams to ensure data availability and reliability.
What are the most commonly searched types of Databricks Data Engineer jobs in Pennsylvania? The most popular types of Databricks Data Engineer jobs in Pennsylvania are:
What cities in Pennsylvania are hiring for Overnight Databricks Data Engineer jobs? Cities in Pennsylvania with the most Overnight Databricks Data Engineer job openings:
Overnight Databricks Data Engineer jobs near you

Data Engineer - Fabric

Synergy Staffing

Wexford, PA • On-site

$108K - $130K/yr

Full-time

Re-posted 28 days ago

Job description

Job Summary:
Synergy Staffing is seeking a Databricks Data Engineer to develop and support data pipelines and analytics environments in Azure cloud-based data lake. The role involves translating business requirements into data engineering solutions and ensuring data quality and management in an enterprise-scale platform.
Responsibilities:
• Design, build, and optimize scalable data solutions using Databricks and Medallion Architecture.
• Manage ingestion routines for processing multi-terabyte datasets efficiently for multiple projects simultaneously, where each project may have multiple Databricks workspaces.
• Integrate data from various structured and unstructured sources to enable high-quality business insights. Proficiency in data analysis techniques for deriving insights from large datasets.
• Implement effective data management strategies to ensure data integrity, availability, and accessibility. Identify opportunities for cost optimization in data storage, processing, and analytics operations.
• Monitor and support user requests, addressing platform or performance issues, cluster stability, Spark optimization, and configuration management.
• Collaborate with the team to enable advanced AI-driven analytics and data science workflows.
• Integrate with various Azure services including Azure Functions, Storage Services, Data Factory, Log Analytics, and User Management for seamless data workflows.
• Experience with the above Azure services is a plus.
• Provision and manage infrastructure using Infrastructure-as-Code (IaC)
• Apply best practices for data security, data governance, and compliance, ensuring support for federal regulations and public trust standards.
• Proactively collaborate with technical and non-technical teams to gather requirements and translate business needs into data solutions.
Qualifications:
Required:
• BS degree in Computer Science or related field
• 5+ years of experience developing and designing Ingestion flows (structured, streaming, and unstructured data) using cloud platform services with data quality
• Databricks Data Engineer certification and 2+ years of experience maintaining Databricks platform and development in Spark
• Ability to work directly with clients and act as front line support for requests coming in from clients. Clearly document and express the solution in form of architecture and interface diagrams.
• Proficient at Python, Spark and R are essential.
• Knowledge and experience with data governance, including metadata management, enterprise data catalog, design standards, data quality governance, and data security.
• Experience with Agile process methodology, CI/CD automation, and cloud-based developments (Azure, AWS).
Preferred:
• Experience with the above Azure services is a plus.
• .NET based development is a plus.
Company:
Synergy Staffing is a recruiting and placement company. Founded in 2007, the company is headquartered in Pittsburgh, USA, with a team of 11-50 employees. The company is currently Early Stage.

Apply