1

Overnight Databricks Data Engineer Jobs in Pleasanton, CA

Data Engineer

Bay Point, CA · On-site

$125K - $151K/yr

Databricks Data Engineer Location: [Bay Area, CA] Duration : 12+ Months Need experience with Databricks Job Summary: We are seeking a skilled Data Engineer with hands-on Databricks experience to ...

Databricks Engineer Experience 4-8 years Location As per business requirement Employment Type ... Integrate data from various sources including relational databases, APIs, cloud storage, and ...

Databricks is looking for a Principal Data Scientist to serve as the statistical voice of the Data ... Partner with engineering VPs, product leaders, and executive staff to embed a data-driven decision ...

Data Engineer

Santa Clara, CA · On-site

$134K - $161K/yr

Proven hands-on expertise in data engineering with Snowflake and Databricks, including building and optimizing large-scale data pipelines. Strong understanding of data architecture fundamentals such ...

Data Engineer

Mountain View, CA · On-site

$135K - $162K/yr

Job Role: Data Engineer Location: Mountain View, CA (Hybrid 3days onsite, 2 days remote ... Leverage advanced tools like Databricks for big data processing, ensuring the data is accessible ...

Data Engineer III

San Ramon, CA · On-site +1

$128K - $153K/yr

Lead architecture and design of complex data pipelines on Databricks lakehouse architecture (Unity Catalog, Delta Lake, Structured Streaming) * Define technical approach for data engineering ...

Data Engineer III

San Ramon, CA · On-site +1

$128K - $153K/yr

Lead architecture and design of complex data pipelines on Databricks lakehouse architecture (Unity Catalog, Delta Lake, Structured Streaming) * Define technical approach for data engineering ...

Data Engineer

Mountain View, CA · On-site

$136K - $163K/yr

Intellyk Inc. is seeking a Data Engineer to join their team in Mountain View, CA. The role involves ... background • Databricks (nice to have) • AWS experience • AI/GenAI (nice to have ...

Snowflake Data Engineer

Santa Clara, CA · On-site

$134K - $161K/yr

Proven hands-on expertise in data engineering with Snowflake and Databricks , including building and optimizing large-scale data pipelines. * Strong understanding of data architecture fundamentals ...

They are seeking a highly skilled Senior Data Engineer / Data Architect with extensive experience in Azure and Databricks to build large-scale data platforms and deliver high-quality data products in ...

As a leading product engineering firm based in Silicon Valley, we provide full-cycle product ... CI/CD for Databricks workloads Big Data & Processing Frameworks * Apache Spark with advanced ...

next page

Showing results 1-20

Overnight Databricks Data Engineer information

What is an overnight Databricks data engineer?

An Overnight Databricks Data Engineer is a professional who works primarily during night shifts to manage, design, and maintain big data pipelines and workflows using Databricks, a cloud-based data analytics platform. Their responsibilities often include developing and optimizing data processing jobs, ensuring data quality, and troubleshooting issues that arise during overnight data operations. This role is critical for organizations that require 24/7 data processing, continuous ETL jobs, or real-time analytics. Working overnight may also involve monitoring automated systems, performing scheduled data loads, and collaborating with global teams to ensure data availability and reliability.

What skills and qualifications are needed to thrive as an overnight Databricks data engineer?

To thrive as an Overnight Databricks Data Engineer, you need strong proficiency in data engineering, Python or Scala programming, and experience with big data technologies, typically supported by a relevant degree in computer science or a related field. Familiarity with Databricks, Apache Spark, cloud platforms (such as AWS or Azure), and certifications like Databricks Certified Data Engineer are highly valued. Attention to detail, problem-solving, and effective communication are essential soft skills, especially for troubleshooting and collaborating across shifts. These competencies ensure reliable data pipeline management and efficient resolution of issues during off-hours, maintaining seamless business operations.

What unique challenges do overnight Databricks data engineers face, and how can they be addressed?

Overnight Databricks Data Engineers often work with limited real-time support, which can present challenges when troubleshooting urgent data pipeline issues or system outages. To address this, it’s essential to develop strong problem-solving skills, document processes thoroughly, and leverage automated monitoring and alerting tools. Additionally, close collaboration with daytime teams during handoff periods ensures continuity and minimizes disruptions. Building a habit of proactive communication and maintaining detailed logs helps the entire team resolve issues efficiently and maintain data quality.

What is the difference between Overnight Databricks Data Engineer vs Data Engineer?

AspectOvernight Databricks Data Engineer
Work EnvironmentPrimarily remote or on-site, working overnight shifts to support global data operations
CertificationsDatabricks certifications, cloud platform credentials (AWS, Azure), data engineering certifications
Tools & TechnologiesDatabricks platform, Spark, cloud services, SQL, Python, ETL tools
Industry UsageTech, finance, healthcare, retail with 24/7 data needs

While both roles focus on data engineering, the Overnight Databricks Data Engineer specializes in managing data pipelines on the Databricks platform during overnight shifts, often supporting global operations. A Data Engineer may work across various platforms and shifts, with broader responsibilities in data architecture and pipeline development. The overnight role emphasizes specific platform expertise and shift timing, catering to organizations with continuous data processing needs.

What are the most commonly searched types of Databricks Data Engineer jobs in Pleasanton, CA?

The most popular types of Databricks Data Engineer jobs in Pleasanton, CA are:

What cities near Pleasanton, CA are hiring for Overnight Databricks Data Engineer jobs?

Cities near Pleasanton, CA with the most Overnight Databricks Data Engineer job openings:

Data Engineer

Bay Point, CA • On-site

$125K - $151K/yr

Contractor

Re-posted 5 days ago


Job description

Hello There,

Wish you a Happy Monday,

My name is Yashmita, and I am a Technical Recruiter at Kanak IT Services LLC. I am reaching out to you regarding the following job opportunity. If you are interested, kindly reply to this email with your updated resume.  

Position : Databricks Data Engineer
Location: [Bay Area, CA]

Duration : 12+ Months
Need experience with Databricks

Job Summary:
We are seeking a skilled Data Engineer with hands-on Databricks experience to design, build, and optimize large-scale data pipelines and analytics solutions. You will work with cross-functional teams to enable scalable data processing using the Databricks Lakehouse Platform on Azure.

Key Responsibilities:
• Design and implement ETL/ELT pipelines using Databricks, Delta Lake, and Apache Spark
• Collaborate with data scientists, analysts, and stakeholders to deliver clean, reliable, and well-modeled data
• Build and manage data workflows with Databricks Jobs, Notebooks, and Workflows
• Optimize Spark jobs for performance, reliability, and cost-efficiency
• Maintain and monitor data pipelines, ensuring availability and data quality
• Implement CI/CD practices for Databricks notebooks and infrastructure-as-code (e.g., Terraform, Databricks CLI)
• Document data pipelines, datasets, and operational processes
• Ensure compliance with data governance, privacy, and security policies
Qualifications:
• Bachelor’s or Master’s in Computer Science, Data Engineering, or a related field
• 5+ years of experience in data engineering or a similar role
• Strong hands-on experience with Databricks and Apache Spark (Python, Scala, or SQL)
• Proficiency with Delta Lake, Unity Catalog, and data lake architectures
• Experience with cloud platforms (Azure, AWS, or GCP), especially data services (e.g., S3, ADLS, BigQuery)
• Familiarity with CI/CD pipelines, version control (Git), and job orchestration tools (Airflow, DB Workflows)
• Strong understanding of data warehousing concepts, performance tuning, and big data processing
Preferred Skills:
• Experience with MLflow, Feature Store, or other machine learning tools in Databricks
• Knowledge of data governance tools like Unity Catalog or Purview
• Experience integrating BI tools (Power BI, Tableau) with Databricks
• Databricks certification(s) (Data Engineer Associate/Professional, Machine Learning, etc.)

Feel free to reach out yashmita@kanakits.com