O Fallon, MO · On-site
$45 - $49/hr
Senior Data Engineer with Spark Scala Location : O'Fallon, MO - onsite (LOCAL ONLY) Duration : 6 ... and flow management • Strong SQL skills and experience working with structured and semi ...
O Fallon, MO · On-site
$45 - $49/hr
Senior Data Engineer with Spark Scala Location : O'Fallon, MO - onsite (LOCAL ONLY) Duration : 6 ... and flow management • Strong SQL skills and experience working with structured and semi ...
$90K - $180K/yr
Feature Engineering & Model Improvement: Engineer meaningful and explainable features using ... Experience in MLOps and deploying/managing machine learning models. * Exceptional analytical and ...
$90K - $180K/yr
Feature Engineering & Model Improvement: Engineer meaningful and explainable features using ... Experience in MLOps and deploying/managing machine learning models. * Exceptional analytical and ...
$90K - $180K/yr
Feature Engineering & Model Improvement: Engineer meaningful and explainable features using ... Experience in MLOps and deploying/managing machine learning models. * Exceptional analytical and ...
$90K - $180K/yr
Feature Engineering & Model Improvement: Engineer meaningful and explainable features using ... Experience in MLOps and deploying/managing machine learning models. * Exceptional analytical and ...
They are seeking an AI Data Engineer - Manager to lead data architecture and engineering delivery for AI/ML/GenAI solutions, ensuring data integrity and scalability while managing a team and ...
They are seeking an AI Data Engineer - Manager to lead data architecture and engineering delivery for AI/ML/GenAI solutions, ensuring data integrity and scalability while managing a team and ...
$90K - $180K/yr
Feature Engineering & Model Improvement: Engineer meaningful and explainable features using ... Experience in MLOps and deploying/managing machine learning models. * Exceptional analytical and ...
$90K - $180K/yr
Feature Engineering & Model Improvement: Engineer meaningful and explainable features using ... Experience in MLOps and deploying/managing machine learning models. * Exceptional analytical and ...
Wildwood, MO · On-site
$107.40K - $129K/yr
Senior Data Warehouse Developer RGA is looking for results oriented, Senior Data Warehouse Developer to independently design, develop, test, implement, document, and support data warehouses ...
Wildwood, MO · On-site
$107.40K - $129K/yr
Senior Data Warehouse Developer RGA is looking for results oriented, Senior Data Warehouse Developer to independently design, develop, test, implement, document, and support data warehouses ...
$90K - $180K/yr
Description As a Data Engineer on the Global Data Foundation team, you will play a vital role in ... Design, manage, and govern data schemas throughout their lifecycle. * Collaborate with cross ...
$90K - $180K/yr
Description As a Data Engineer on the Global Data Foundation team, you will play a vital role in ... Design, manage, and govern data schemas throughout their lifecycle. * Collaborate with cross ...
$90K - $180K/yr
Description As a Data Engineer on the Global Data Foundation team, you will play a vital role in ... Design, manage, and govern data schemas throughout their lifecycle. * Collaborate with cross ...
$90K - $180K/yr
Description As a Data Engineer on the Global Data Foundation team, you will play a vital role in ... Design, manage, and govern data schemas throughout their lifecycle. * Collaborate with cross ...
Kansas City, MO · On-site
$103.40K - $140.50K/yr
About Us American Century Investments ® is a leading global asset manager with over 65 years of ... The Senior Data Quality Engineer will report to the Manager, Software Engineering and play a key ...
Kansas City, MO · On-site
$103.40K - $140.50K/yr
About Us American Century Investments ® is a leading global asset manager with over 65 years of ... The Senior Data Quality Engineer will report to the Manager, Software Engineering and play a key ...
Kansas City, MO · Hybrid
$103.40K - $140.50K/yr
About Us American Century Investments is a leading global asset manager with over 65 years of ... The Senior Data Quality Engineer will report to the Manager, Software Engineering and play a key ...
Kansas City, MO · Hybrid
$103.40K - $140.50K/yr
About Us American Century Investments is a leading global asset manager with over 65 years of ... The Senior Data Quality Engineer will report to the Manager, Software Engineering and play a key ...
$90K - $180K/yr
Description As a Data Engineer on the Global Data Foundation team, you will play a vital role in ... Design, manage, and govern data schemas throughout their lifecycle. * Collaborate with cross ...
$90K - $180K/yr
Description As a Data Engineer on the Global Data Foundation team, you will play a vital role in ... Design, manage, and govern data schemas throughout their lifecycle. * Collaborate with cross ...
Maryland Heights, MO · On-site
$110.60K - $132.90K/yr
Senior Data Engineer- Responsibilities: • Design, develop, and optimize scalable data pipelines ... Apex Systems Vendor Management. MSA: Blanket Approval Received Client Letter: Will Provide ...
Maryland Heights, MO · On-site
$110.60K - $132.90K/yr
Senior Data Engineer- Responsibilities: • Design, develop, and optimize scalable data pipelines ... Apex Systems Vendor Management. MSA: Blanket Approval Received Client Letter: Will Provide ...
Saint Louis, MO · On-site +1
$60 - $65/hr
Senior Data Engineer - ETL/SQL/Matillion Location: St. Louis, Missouri Duration: Contract Job ID ... Preferred: * Bachelor's degree in Computer Science, Computer Information Systems, Management ...
Saint Louis, MO · On-site +1
$60 - $65/hr
Senior Data Engineer - ETL/SQL/Matillion Location: St. Louis, Missouri Duration: Contract Job ID ... Preferred: * Bachelor's degree in Computer Science, Computer Information Systems, Management ...
California, MO · On-site
$150K - $200K/yr
Lead Data Engineer (Remote Work Eligible) page is loaded## Lead Data Engineer (Remote Work Eligible ... Provide mentorship to junior team members and provide strategic input to senior management across ...
California, MO · On-site
$150K - $200K/yr
Lead Data Engineer (Remote Work Eligible) page is loaded## Lead Data Engineer (Remote Work Eligible ... Provide mentorship to junior team members and provide strategic input to senior management across ...
Saint Louis, MO · On-site
$85 - $95/hr
Partner with engineering leadership and client stakeholders to ensure scalability, performance ... Strong understanding of data contracts, schema management, and data governance principles
Saint Louis, MO · On-site
$85 - $95/hr
Partner with engineering leadership and client stakeholders to ensure scalability, performance ... Strong understanding of data contracts, schema management, and data governance principles
O Fallon, MO · On-site
$140K - $231K/yr
Senior Data Engineer (Apache Spark, NiFi, SQL) Mastercard is a global technology company in the payments industry. Our mission is to connect and power an inclusive, digital economy that benefits ...
O Fallon, MO · On-site
$140K - $231K/yr
Senior Data Engineer (Apache Spark, NiFi, SQL) Mastercard is a global technology company in the payments industry. Our mission is to connect and power an inclusive, digital economy that benefits ...
Saint Louis, MO · On-site
$111.30K - $133.70K/yr
Senior Data Engineer The primary responsibilities of this role are to: lead and participate in ... Data Scientists, Product Managers, and business and IT stakeholders, that result in design ...
Saint Louis, MO · On-site
$111.30K - $133.70K/yr
Senior Data Engineer The primary responsibilities of this role are to: lead and participate in ... Data Scientists, Product Managers, and business and IT stakeholders, that result in design ...
Saint Louis, MO · On-site
$124K - $280K/yr
Industry/Sector Not Applicable Specialism Data, Analytics & AI Management Level Senior Manager & Summary At PwC, our people in data and analytics engineering focus on leveraging advanced technologies ...
Saint Louis, MO · On-site
$124K - $280K/yr
Industry/Sector Not Applicable Specialism Data, Analytics & AI Management Level Senior Manager & Summary At PwC, our people in data and analytics engineering focus on leveraging advanced technologies ...
$124K - $280K/yr
Industry/Sector Not Applicable Specialism Data, Analytics & AI Management Level Senior Manager & Summary At PwC, our people in data and analytics engineering focus on leveraging advanced technologies ...
$124K - $280K/yr
Industry/Sector Not Applicable Specialism Data, Analytics & AI Management Level Senior Manager & Summary At PwC, our people in data and analytics engineering focus on leveraging advanced technologies ...
$99.20K - $134.70K/yr
Title and Summary Senior Data Engineer (Apache Spark, NiFi, SQL) Who is Mastercard? Mastercard is a global technology company in the payments industry. Our mission is to connect and power an ...
$99.20K - $134.70K/yr
Title and Summary Senior Data Engineer (Apache Spark, NiFi, SQL) Who is Mastercard? Mastercard is a global technology company in the payments industry. Our mission is to connect and power an ...
| Aspect | Senior Data Engineer Manager | Data Engineer |
|---|---|---|
| Credentials | Bachelor's/Master's in CS, Data Science, or related; often leadership experience | Bachelor's in CS, Data Science, or related |
| Work Environment | Leads teams, manages projects, strategic planning | Develops data pipelines, coding, data modeling |
| Industry Usage | Common in organizations with large data teams | Entry to mid-level roles in data teams |
The Senior Data Engineer Manager oversees data engineering teams and projects, focusing on leadership and strategy, while Data Engineers primarily build and maintain data pipelines and infrastructure. The managerial role requires leadership skills and experience, whereas Data Engineers focus on technical execution.
$45 - $49/hr
Other
This job post has expired today. Applications are no longer accepted.
Job ID : 83614-1
Job Title: Senior Data Engineer with Spark Scala
Location : O'Fallon, MO - onsite (LOCAL ONLY)
Duration : 6 months + possible extension
Rate Range: $45 - $49/hour on W2 (All inclusive)
Note: Applicants must be willing to work on W2 only.
• Languages: Scala, Python (PySpark), SQL
• Big Data: Apache Spark (Core, SQL, Structured Streaming)
• Streaming: Kafka
• Ingestion / Orchestration: Apache NiFi
• Storage: Apache Ozone, Ceph, object storage concepts
• OS & Tooling: Linux, Git, CI/CD, monitoring and logging tools"
We are looking for a highly skilled Senior Data Engineer with deep expertise in Apache Spark, Scala, and PySpark to build and operate large scale batch and streaming data processing systems. The role has a strong emphasis on real time streaming architectures using Kafka and Spark Structured Streaming, alongside ingestion and orchestration with Apache NiFi and scalable storage using Apache Ozone and Ceph. This position is ideal for engineers who enjoy solving complex performance, scalability, latency, and reliability challenges in production data platforms.
Key Responsibilities
• Design, develop, and maintain large scale Spark applications using Scala and PySpark
• Build and operate streaming heavy data pipelines using Kafka and Spark Structured Streaming
• Implement stateful streaming patterns including windowing, watermarking, late data handling, and checkpointing
• Develop robust event replay and reprocessing workflows using Kafka offsets and partitions
• Build ingestion and routing flows using Apache NiFi, including Kafka based ingestion patterns
• Implement end to end ETL/ELT pipelines with strong emphasis on low latency, fault tolerance, and scalability
• Optimize Spark jobs through partitioning strategies, memory tuning, shuffle optimization, and efficient data formats
• Integrate Spark workloads with distributed object storage systems such as Apache Ozone and Ceph
• Ensure data quality, consistency, and auditability through validation, reconciliation, and metadata capture
• Collaborate with platform, infrastructure, and operations teams on production readiness and capacity planning
• Support production systems, including monitoring, incident analysis, and root cause resolution
• Contribute to reusable frameworks, coding standards, and engineering best practices
• Participate in architecture reviews, code reviews, and technical documentation
Required Qualifications
• Bachelor's degree in Computer Science, Engineering, or equivalent practical experience
• Strong hands on experience with Apache Spark in production environments
• Advanced proficiency in Scala and PySpark
• Solid understanding of distributed systems and data processing at scale
• Strong experience with Kafka based streaming architectures
• Hands on experience with Spark Structured Streaming
• Experience building batch and real time pipelines
• Hands on experience with Apache NiFi for data ingestion and flow management
• Strong SQL skills and experience working with structured and semi structured data
• Experience working with object storage or distributed storage platforms
• Proficiency with Linux, shell scripting, and Git based version control
Preferred Qualifications
• Experience with Apache Ozone and/or Ceph as storage backends for analytics workloads
• Experience implementing exactly once / at least once streaming semantics
• Strong background in Spark performance tuning (CPU, memory, I/O, shuffle)
• Experience supporting mission critical production systems with strict SLAs
• Familiarity with CI/CD pipelines and automated testing for data applications
• Experience designing observability for streaming systems (lag, throughput, backpressure)"
Company Benefits & Culture
Inclusive and diverse work environment
Opportunities for professional growth and development
Comprehensive health and wellness benefits
Sourced by ZipRecruiter
Architectural services
11 - 50 Employees
Chattanooga, TN, US
1985
