1

Overnight Databricks Data Engineer Jobs in Buffalo, NY

AI Engineer

Buffalo, NY

$50K - $112K/yr

Certifications aligned to data engineering, machine learning, and cloud platforms, including AWS, Google Cloud, Microsoft Azure, Databricks, Snowflake, or related data and AI credentials - Building ...

Knowledge of ML frameworks and libraries, such as Kubeflow, Databricks, KServe and so on ... Experience designing evaluation frameworks for complex visual data * Experience leading technical ...

Knowledge of ML frameworks and libraries, such as Kubeflow, Databricks, KServe and so on ... Experience designing evaluation frameworks for complex visual data * Experience leading technical ...

Machine Learning Engineer IV, Data

Buffalo, NY · On-site

$54 - $71.50/hr

Knowledge of ML frameworks and libraries, such as Kubeflow, Databricks, KServe and so on ... Experience designing evaluation frameworks for complex visual data * Experience leading technical ...

Knowledge of ML frameworks and libraries, such as Kubeflow, Databricks, KServe and so on ... Experience designing evaluation frameworks for complex visual data * Experience leading technical ...

Knowledge of ML frameworks and libraries, such as Kubeflow, Databricks, KServe and so on ... Experience designing evaluation frameworks for complex visual data #LI-AM3 Our Values Trust ...

Experience in multiple programming languages, including R and Python. * Deep understanding of GLMs ... Experience with Databricks or similar big data platforms is highly desirable. * Strong ...

Showing results 21-40

Overnight Databricks Data Engineer information

What is an overnight Databricks data engineer?

An Overnight Databricks Data Engineer is a professional who works primarily during night shifts to manage, design, and maintain big data pipelines and workflows using Databricks, a cloud-based data analytics platform. Their responsibilities often include developing and optimizing data processing jobs, ensuring data quality, and troubleshooting issues that arise during overnight data operations. This role is critical for organizations that require 24/7 data processing, continuous ETL jobs, or real-time analytics. Working overnight may also involve monitoring automated systems, performing scheduled data loads, and collaborating with global teams to ensure data availability and reliability.

What skills and qualifications are needed to thrive as an overnight Databricks data engineer?

To thrive as an Overnight Databricks Data Engineer, you need strong proficiency in data engineering, Python or Scala programming, and experience with big data technologies, typically supported by a relevant degree in computer science or a related field. Familiarity with Databricks, Apache Spark, cloud platforms (such as AWS or Azure), and certifications like Databricks Certified Data Engineer are highly valued. Attention to detail, problem-solving, and effective communication are essential soft skills, especially for troubleshooting and collaborating across shifts. These competencies ensure reliable data pipeline management and efficient resolution of issues during off-hours, maintaining seamless business operations.

What unique challenges do overnight Databricks data engineers face, and how can they be addressed?

Overnight Databricks Data Engineers often work with limited real-time support, which can present challenges when troubleshooting urgent data pipeline issues or system outages. To address this, it’s essential to develop strong problem-solving skills, document processes thoroughly, and leverage automated monitoring and alerting tools. Additionally, close collaboration with daytime teams during handoff periods ensures continuity and minimizes disruptions. Building a habit of proactive communication and maintaining detailed logs helps the entire team resolve issues efficiently and maintain data quality.

What is the difference between Overnight Databricks Data Engineer vs Data Engineer?

AspectOvernight Databricks Data Engineer
Work EnvironmentPrimarily remote or on-site, working overnight shifts to support global data operations
CertificationsDatabricks certifications, cloud platform credentials (AWS, Azure), data engineering certifications
Tools & TechnologiesDatabricks platform, Spark, cloud services, SQL, Python, ETL tools
Industry UsageTech, finance, healthcare, retail with 24/7 data needs

While both roles focus on data engineering, the Overnight Databricks Data Engineer specializes in managing data pipelines on the Databricks platform during overnight shifts, often supporting global operations. A Data Engineer may work across various platforms and shifts, with broader responsibilities in data architecture and pipeline development. The overnight role emphasizes specific platform expertise and shift timing, catering to organizations with continuous data processing needs.

What are the most commonly searched types of Databricks Data Engineer jobs in Buffalo, NY?

The most popular types of Databricks Data Engineer jobs in Buffalo, NY are:

What cities near Buffalo, NY are hiring for Overnight Databricks Data Engineer jobs?

Cities near Buffalo, NY with the most Overnight Databricks Data Engineer job openings:

$50K - $112K/yr

Full-time

Medical, Dental, Vision, Retirement, PTO

Re-posted 9 days ago


PwC rating

8.3

Company rating: 8.3 out of 10

Based on 77 frontline employees who took The Breakroom Quiz

26th of 72 rated business consultants


Job description

Industry/Sector

Not Applicable

Specialism

IFS - Information Technology (IT)

Management Level

Associate

Job Description & Summary

The Opportunity
As an AI Engineer, you will be at the forefront of transforming raw data into actionable insights, enabling informed decision-making and driving business growth. Within our Internal Firm Services practice, you will apply data, algorithms, and software engineering to build and deploy software and platform systems that create Artificial Intelligence and Machine Learning-based solutions at scale. Your work will involve designing AI systems, data wrangling, and software implementation to enable the AI models to be useful and scalable.
As an Associate, you will focus on learning and contributing to projects while developing your skills and knowledge to deliver quality work. You will engage with different stakeholders to build meaningful connections, learn how to manage and inspire others, and grow your personal brand by deepening your technical knowledge of firm services and technology resources. In increasingly complex situations, you will build acumen to anticipate the needs of your teams and internal stakeholders, embrace ambiguity, ask questions, and use these challenges as opportunities for growth.
In this role, you will take ownership and consistently deliver quality work that drives value for our clients and success as a team. You will be part of a dynamic environment where every experience is an opportunity to learn and grow, opening doors to more opportunities within the firm.
Responsibilities
- Designing and implementing AI systems to transform raw data into actionable insights
- Developing scalable machine learning models using Python and TensorFlow
- Integrating data from various sources to create unified views for analysis
- Building and maintaining data pipelines to support AI model deployment
- Applying complex data analysis techniques to discern patterns and trends
- Collaborating with team members to enhance AI solutions and drive business growth
- Utilizing natural language processing tools like NLTK for text analytics and sentiment analysis
- Implementing neural networks and deep learning methods for advanced AI applications
- Managing data quality and infrastructure to support reliable AI operations
- Engaging in continuous learning to adapt to new technologies and methodologies in AI engineering
What You Must Have
- At least a Bachelor's degree or, in lieu of a degree, demonstrating in addition to the minimum years of experience required for the role, three years of specialized training and/or progressively responsible work experience in Engineering with AI and Machine Learning for each missing year of college is required
- At least 1 years of experience
What Sets You Apart
- In at least one of the following fields of study: Computer and Information Science, Computer Engineering, Computer Management, Management Information Systems, Information Technology
- At least one of the following: Certifications aligned to data engineering, machine learning, and cloud platforms, including AWS, Google Cloud, Microsoft Azure, Databricks, Snowflake, or related data and AI credentials

- Building and orchestrating AI agent workflows using frameworks such as LangGraph to automate multi-step reasoning, integrate tools and APIs, and deliver scalable, context-aware solutions
- Applying generative AI techniques, including prompt engineering, LLM evaluation, and fine-tuning, to develop production-ready applications powered by foundation models
- Developing automated evaluation frameworks, including LLM-as-judge pipelines, regression testing, and adversarial benchmarking, to assess reasoning, tool-calling reliability, and output groundedness
- Optimizing open-weight language models, including LLaMA, Mistral, and Gemma, for local and cloud deployment using quantization, inference acceleration, and model-routing techniques
- Designing agent harnesses and implementing context engineering, memory management, retry logic, and structured output validation to support reliable, multi-step AI workflows
- Demonstrating proficiency in Python and TensorFlow for AI projects
- Utilizing machine learning libraries like Scikit-Learn for data analysis
- Engaging in complex data analysis and pattern recognition
- Implementing AI solutions using open-source software
- Applying natural language processing techniques in real-world applications

Travel Requirements

Up to 20%

Job Posting End Date

The salary range for this position is: $50,500 - $112,500. Actual compensation within the range will be dependent upon the individual's skills, experience, qualifications and location, and applicable employment laws. All hired individuals are eligible for an annual discretionary bonus. PwC offers a wide range of benefits, including medical, dental, vision, 401k, holiday pay, vacation, personal and family sick leave, and more. To view our benefits at a glance, please visit the following link: https://pwc.to/benefits-at-a-glanceAs PwC is anequal opportunity employer, all qualified applicants will receive consideration for employment at PwC without regard to race; color; religion; national origin; sex (including pregnancy, sexual orientation, and gender identity); age; disability; genetic information (including family medical history); veteran, marital, or citizenship status; or, any other status protected by law.PwC does not intend to hire experienced or entry level job seekers who will need, now or in the future, PwC sponsorship through the H-1B lottery, except as set forth within the following policy: https://pwc.to/H-1B-Lottery-Policy.Learn more about how we work: https://pwc.to/how-we-workFor only those qualified applicants that are impacted by the Los Angeles County Fair Chance Ordinance for Employers, the Los Angeles' Fair Chance Initiative for Hiring Ordinance, the San Francisco Fair Chance Ordinance, San Diego County Fair Chance Ordinance, and the California Fair Chance Act, where applicable, arrest or conviction records will be considered for Employment in accordance with these laws. At PwC, we recognize that conviction records may have a direct, adverse, and negative relationship to responsibilities such as accessing sensitive company or customer information, handling proprietary assets, or collaborating closely with team members. We evaluate these factors thoughtfully to establish a secure and trusted workplace for all.Applications will be accepted until the position is filled or the posting is removed, unless otherwise set forth on the following webpage. Please visit this link for information about anticipated application deadlines: https://pwc.to/us-application-deadlines

What PwC employees say

Pay

Benefits

Hours and flexibility

Workplace

Get the full story on Breakroom


pwc logo

About pwc

Sourced by ZipRecruiter

We know that the future success of our firm is contingent on equitable experiences for our people. From recruitment to partnership, we’re working hard to give every person an equitable opportunity to grow and to thrive as part of our community of solvers. We understand that establishing and maintaining a fair, equitable and welcoming environment for all people requires building a culture of belonging: a shift from awareness to empathy — while demonstrating inclusive leadership that cultivates trust among our people and our clients. PwC is committed to advancing diversity, equity and inclusion (DEI) through an evidence-based strategy designed to achieve well-defined and meaningful aspirational goals. Our aim is to solve problems for the long term, as that is how we build trust and continue to build on our culture of belonging. At the core of this endeavor are stated goals and a series of linked programs enabling targeted interventions at key moments in our employees’ career trajectories.

Industry

Finance and insurance

Company size

10,000+ Employees

Headquarters location

London, London, UK