Senior Big Data Engineer
$65.75 - $87/hr
... manage production services. * Experience with modern streaming and messaging technologies (for example, Redpanda, Kafka, or similar) and real-time data processing patterns. * Solid understanding of ...
$65.75 - $87/hr
... manage production services. * Experience with modern streaming and messaging technologies (for example, Redpanda, Kafka, or similar) and real-time data processing patterns. * Solid understanding of ...
$65.75 - $87/hr
... manage production services. * Experience with modern streaming and messaging technologies (for example, Redpanda, Kafka, or similar) and real-time data processing patterns. * Solid understanding of ...
$63.50 - $84/hr
... manage production services. * Experience with modern streaming and messaging technologies (for example, Redpanda, Kafka, or similar) and real-time data processing patterns. * Solid understanding of ...
$63.50 - $84/hr
... manage production services. * Experience with modern streaming and messaging technologies (for example, Redpanda, Kafka, or similar) and real-time data processing patterns. * Solid understanding of ...
Redpanda (Managed Kafka) as the primary event bus for streaming and async communication. * Language: TypeScript / Node.js throughout backend services. * Infrastructure: Terraform for resource ...
Redpanda (Managed Kafka) as the primary event bus for streaming and async communication. * Language: TypeScript / Node.js throughout backend services. * Infrastructure: Terraform for resource ...
Fort George G Meade, MD · On-site
$108K - $148K/yr
Experience with stream processing systems (Kafka, Redis Streams, AMQP, Redpanda Connect/Benthos) * Excellent customer relations skills, including managing requirements for feature requests
Fort George G Meade, MD · On-site
$108K - $148K/yr
Experience with stream processing systems (Kafka, Redis Streams, AMQP, Redpanda Connect/Benthos) * Excellent customer relations skills, including managing requirements for feature requests
$108K - $148K/yr
Experience with stream processing systems (Kafka, Redis Streams, AMQP, Redpanda Connect/Benthos) * Excellent customer relations skills, including managing requirements for feature requests
$108K - $148K/yr
Experience with stream processing systems (Kafka, Redis Streams, AMQP, Redpanda Connect/Benthos) * Excellent customer relations skills, including managing requirements for feature requests
Redpanda (Managed Kafka) as the primary event bus for streaming and async communication. * Language: TypeScript / Node.js throughout backend services. * Infrastructure: Terraform for resource ...
Redpanda (Managed Kafka) as the primary event bus for streaming and async communication. * Language: TypeScript / Node.js throughout backend services. * Infrastructure: Terraform for resource ...
Experience managing Kafka (or Redpanda) in production environments * Proficiency with Terraform for infrastructure as code * Experience building and maintaining CI/CD pipelines (GitLab, GitHub ...
Experience managing Kafka (or Redpanda) in production environments * Proficiency with Terraform for infrastructure as code * Experience building and maintaining CI/CD pipelines (GitLab, GitHub ...
Customer Success Manager Letter AI is the world's first unified revenue acceleration platform ... Our platform empowers sales teams at customers like Lenovo and Redpanda to ramp up faster with AI ...
Customer Success Manager Letter AI is the world's first unified revenue acceleration platform ... Our platform empowers sales teams at customers like Lenovo and Redpanda to ramp up faster with AI ...
$124K - $166K/yr
... Redpanda data streaming solutions on Kubernetes * Experience in creating data pipelines from scratch with the ability to process data in large volumes and manage data stores * Experience with ...
$124K - $166K/yr
... Redpanda data streaming solutions on Kubernetes * Experience in creating data pipelines from scratch with the ability to process data in large volumes and manage data stores * Experience with ...
Redpanda (Managed Kafka) as the primary event bus for streaming and async communication. * Language: TypeScript / Node.js throughout backend services. * Infrastructure: Terraform for resource ...
Redpanda (Managed Kafka) as the primary event bus for streaming and async communication. * Language: TypeScript / Node.js throughout backend services. * Infrastructure: Terraform for resource ...
Austin, TX · Hybrid
$105K/yr
We're looking for a Renewals Manager to join our growing Customer Success organization. In this role, you'll own a high-velocity book of renewal business across our SMB and mid-market customer ...
Austin, TX · Hybrid
$105K/yr
We're looking for a Renewals Manager to join our growing Customer Success organization. In this role, you'll own a high-velocity book of renewal business across our SMB and mid-market customer ...
$310K - $385K/yr
Experience with stream processing systems such as Kafka and RedPanda * Proficiency with ... Preference for managing healthcare data standards (e.g. HIPAA, EDI) * A desire to be a part of our ...
$310K - $385K/yr
Experience with stream processing systems such as Kafka and RedPanda * Proficiency with ... Preference for managing healthcare data standards (e.g. HIPAA, EDI) * A desire to be a part of our ...
Kubernetes, Redpanda, SingleStore, Terraform. * Cloud Ecosystem: Deep usage of Google Cloud ... volumes, and managing complex, high-reliability distributed systems (2+ years of relevant ...
Kubernetes, Redpanda, SingleStore, Terraform. * Cloud Ecosystem: Deep usage of Google Cloud ... volumes, and managing complex, high-reliability distributed systems (2+ years of relevant ...
... Redpanda Connect, or Benthos • Experience with web accessibility standards such as WCAG 2.1 • ... management such as Jira or Confluence • Possession of excellent customer relations skills ...
... Redpanda Connect, or Benthos • Experience with web accessibility standards such as WCAG 2.1 • ... management such as Jira or Confluence • Possession of excellent customer relations skills ...
New York, NY · On-site
$130K/yr
Kubernetes, Redpanda, SingleStore, Terraform. * Cloud Ecosystem: Deep usage of Google Cloud ... volumes, and managing complex, high-reliability distributed systems (2+ years of relevant ...
New York, NY · On-site
$130K/yr
Kubernetes, Redpanda, SingleStore, Terraform. * Cloud Ecosystem: Deep usage of Google Cloud ... volumes, and managing complex, high-reliability distributed systems (2+ years of relevant ...
Groton, CT · On-site
$105K - $143K/yr
We're seeking a skilled Forward Deployed Data Engineer to build the next-generation data management ... Implement real-time data pipelines with MQTT and Redpanda for stream processing. * Implement ...
Quick apply
Groton, CT · On-site
$105K - $143K/yr
We're seeking a skilled Forward Deployed Data Engineer to build the next-generation data management ... Implement real-time data pipelines with MQTT and Redpanda for stream processing. * Implement ...
New York, NY · On-site
$62.25 - $82.75/hr
Advanced skills in managing containerized environments using Kubernetes and OpenShift. * Practical experience with Confluent Cloud, RedPanda for event streaming architectures. * Experience with API ...
Quick apply
New York, NY · On-site
$62.25 - $82.75/hr
Advanced skills in managing containerized environments using Kubernetes and OpenShift. * Practical experience with Confluent Cloud, RedPanda for event streaming architectures. * Experience with API ...
Manassas, VA · On-site
Familiarity with modern message bus architectures (Redpanda, Kafka) in a defense context ... Clear growth path toward Technical Program Manager or Principal Integrator as the program scales.
Manassas, VA · On-site
Familiarity with modern message bus architectures (Redpanda, Kafka) in a defense context ... Clear growth path toward Technical Program Manager or Principal Integrator as the program scales.
State management (React Context, Zustand, Redux, etc.) Back-end (Essential) * Node.js * Python ... Experience with Kafka, RedPanda, or other streaming platforms is a plus DevOps * CI/CD (GitHub ...
State management (React Context, Zustand, Redux, etc.) Back-end (Essential) * Node.js * Python ... Experience with Kafka, RedPanda, or other streaming platforms is a plus DevOps * CI/CD (GitHub ...
State management (React Context, Zustand, Redux, etc.) Back-end (Essential) * Node.js * Python ... Experience with Kafka, RedPanda, or other streaming platforms is a plus DevOps * CI/CD (GitHub ...
State management (React Context, Zustand, Redux, etc.) Back-end (Essential) * Node.js * Python ... Experience with Kafka, RedPanda, or other streaming platforms is a plus DevOps * CI/CD (GitHub ...
| Aspect | Manager Redpanda | Data Engineer |
|---|---|---|
| Primary Focus | Oversees Redpanda platform operations and team management | Designs, builds, and maintains data pipelines and infrastructure |
| Required Skills | Leadership, Redpanda platform knowledge, project management | Programming, data modeling, ETL processes |
| Certifications | Redpanda platform certifications, leadership certifications | Data engineering certifications, cloud platform certifications |
| Work Environment | Management, team coordination, platform oversight | Technical development, data infrastructure setup |
The Manager Redpanda focuses on leading teams and managing Redpanda platform operations, while the Data Engineer concentrates on building and maintaining data systems. Both roles require technical knowledge, but the Manager Redpanda emphasizes leadership and platform management, whereas the Data Engineer is more hands-on with data infrastructure development.
$65.75 - $87/hr
Full-time
Medical, Dental, Vision, Life, Retirement, PTO
Posted 4 days ago
LiveRamp is the data collaboration platform of choice for the world's most innovative companies. A groundbreaking leader in consumer privacy, data ethics, and foundational identity, LiveRamp is setting the new standard for building a connected customer view with unmatched clarity and context while protecting precious brand and consumer trust. LiveRamp offers complete flexibility to collaborate wherever data lives to support the widest range of data collaboration use cases-within organizations, between brands, and across its premier global network of top-quality partners.
Hundreds of global innovators, from iconic consumer brands and tech giants to banks, retailers, and healthcare leaders turn to LiveRamp to build enduring brand and business value by deepening customer engagement and loyalty, activating new partnerships, and maximizing the value of their first-party data while staying on the forefront of rapidly evolving compliance and privacy requirements.
You will:
Design, build, and optimize scalable data processing pipelines using Spark, Airflow, and related big data technologies to support both batch and real-time use cases.
Lead and collaborate with technical, application, and security stakeholders to deliver reliable, secure Big Data infrastructure leveraging tools and platforms such as Spark, Dataproc, Redpanda, and Temporal.
Own and participate in on-call responsibilities for Big Data platforms, triaging and resolving incidents, responding to tickets, and ensuring systems consistently meet defined SLAs for availability, performance, and data quality.
Design, develop, and automate large-scale infrastructure on Kubernetes using Terraform and other infrastructure-as-code patterns to support highperformance data processing, analytics, and AI/ML workloads.
Define and implement monitoring, alerting, and runbooks to provide endtoend observability and drive continuous improvement in reliability and operational excellence using tools like Grafana.
Onboard, train, and mentor vendor teams and external partners so they can effectively support, operate, and extend the solutions owned by the Big Data Infrastructure (BDI) team.
Drive the technical roadmap for emerging data and infrastructure technologies by evaluating options, building proofs of concept (POCs), and authoring solution selection and design documents.
Champion engineering best practices (code reviews, testing strategies, CI/CD, security and compliance standards) across the Big Data ecosystem to ensure maintainable, resilient, and costefficient systems.
Your team will:
Build and operate the core Big Data Infrastructure (BDI) platform that powers largescale data processing, analytics, and AI/ML workloads across LiveRamp using tools like Dataproc, Redpanda, Airflow and Temporal.
Provide secure, self-service, and costefficient data environments that enable product and data teams to experiment, ship, and scale data collaboration applications quickly.
Partner closely with application, security, SRE, and platform teams to ensure data systems meet LiveRamp's standards for privacy, compliance, reliability, and performance.
Evolve common patterns, tooling, and reference architectures that reduce friction for teams adopting new big data, streaming, and AI/ML capabilities.
Continuously improve platform reliability and developer experience by automating operations, simplifying workflows, and incorporating feedback from internal customers.
About you:
7+ years of experience in software engineering or data engineering, with a focus on large-scale data processing and distributed systems.
Hands-on experience building and operating pipelines with Apache Spark (batch and/or streaming) and at least one orchestration framework such as Airflow or Temporal.
Practical experience running Big Data workloads on cloud platforms (for example, using Dataproc or similar managed compute services).
Proficiency with Kubernetes and infrastructure-as-code tools such as Terraform to provision, configure, and manage production services.
Experience with modern streaming and messaging technologies (for example, Redpanda, Kafka, or similar) and real-time data processing patterns.
Solid understanding of systems design, scalability, reliability, and observability for data-intensive workloads.
Experience participating in or leading on-call rotations and incident management processes for production systems.
Strong collaboration and communication skills, including working with cross-functional partners (security, application owners, vendors) and documenting designs and decisions clearly.
Ability to evaluate new technologies, build POCs, and make data-informed recommendations that influence team and platform roadmaps.
Bachelor's degree in Computer Science, Engineering, Mathematics, or a related technical field, or equivalent practical experience.
Preferred Skills:
Experience working on Big Data Infrastructure, data platform, or core infrastructure teams in a cloud-native environment.
3 years of Software Development experience in one of the languages - Python, Go, Java or Scala.
3 years of experience with Linux Operating Systems
3 years of experience in setting up, managing and optimizing Big Data ecosystems such as Kafka , airflow, Redpanda and temporal.
3 years of experience using cloud-based platforms such as AWS, GCP, Azure or similar.
3 years of experience with a Version Control System (GIT, Subversion, or similar)
Experience in using AI tools, NLP and agent creation
Exposure to security and compliance considerations for data platforms (for example, access control, encryption, data governance, and auditing).
Prior experience mentoring engineers, leading technical initiatives, or coordinating work with vendors and external partners.
The approximate annual base compensation range is $130,00 to $196,500. The actual offer, reflecting the total compensation package and benefits, will be determined by a number of factors including the applicant's experience, knowledge, skills, and abilities, geography, as well as internal equity among our team.
Benefits:LiveRamp is an affirmative action and equal opportunity employer (AA/EOE/W/M/Vet/Disabled) and does not discriminate in recruiting, hiring, training, promotion or other employment of associates or the awarding of subcontracts because of a person's race, color, sex, age, religion, national origin, protected veteran, disability, sexual orientation, gender identity, genetics or other protected status. Qualified applicants with arrest and conviction records will be considered for the position in accordance with the San Francisco Fair Chance Ordinance.
We use automated decision systems (ADS) as part of our recruitment and hiring process. If you require an accommodation or believe that the use of an ADS may create a barrier to your application or participation in the hiring process due to a disability or other protected characteristic, please let us know. We are committed to providing reasonable accommodations and ensuring an equitable hiring experience for all candidates.
California residents: Please see our California Personnel Privacy Policy for more information regarding how we collect, use, and disclose the personal information you provide during the job application process.
To all recruitment agencies: LiveRamp does not accept agency resumes. Please do not forward resumes to our jobs alias, LiveRamp employees or any other company location. LiveRamp is not responsible for any fees related to unsolicited resumes.
Sourced by ZipRecruiter
Marketing
1,001 - 5,000 Employees
San Francisco, CA, US
2011