ZipRecruiter

Log In

1

Software Engineer Data Infrastructure Kafka Jobs

Instacart

Staff Software Engineer, Data Infrastructure

OR · Remote

$114K - $137K/yr

We're looking for a Staff Software Engineer to join our Data Governance and Foundations Team. In ... Experience with event-driven and streaming infrastructure (e.g., Kafka, Flink) for real-time ...

next page

Showing results 1-20

All JobsSoftware Engineer Data Infrastructure Kafka Jobs

Software Engineer Data Infrastructure Kafka information

See salary details

$44.5K

$129.7K

$177.5K

How much do software engineer data infrastructure kafka jobs pay per year?

As of Jun 29, 2026, the average yearly pay for software engineer data infrastructure kafka in the United States is $129,716.00, according to ZipRecruiter salary data. Most workers in this role earn between $114,500.00 and $137,500.00 per year, depending on experience, location, and employer.

What is the difference between Software Engineer Data Infrastructure Kafka vs Data Engineer?

AspectSoftware Engineer Data Infrastructure KafkaData Engineer
Primary FocusDeveloping and maintaining Kafka-based data pipelines and infrastructureDesigning, building, and managing data systems and pipelines across various platforms
Required SkillsKafka, distributed systems, programming (Java, Python), data streamingSQL, ETL, data modeling, cloud platforms, scripting
Work EnvironmentCollaborates with data teams, DevOps, and software developers in tech environmentsWorks with data analysts, data scientists, and business teams in data-driven companies

Both roles involve data infrastructure but differ in scope. Software Engineer Data Infrastructure Kafka specializes in Kafka and streaming data pipelines, while Data Engineers focus on broader data systems and pipelines across multiple platforms. The choice depends on whether the focus is on Kafka-specific infrastructure or comprehensive data system management.

What are the key skills and qualifications needed to thrive as a Software Engineer Data Infrastructure Kafka, and why are they important?

To thrive as a Software Engineer Data Infrastructure Kafka, you need strong programming skills (Java, Scala, or Python), experience with distributed systems, and a solid understanding of data architecture, typically supported by a degree in computer science or a related field. Proficiency in Apache Kafka, stream processing frameworks (e.g., Kafka Streams, Flink), and familiarity with cloud platforms (AWS, GCP, or Azure) are essential, with certifications in cloud technologies or Kafka being advantageous. Excellent problem-solving, collaboration, and communication skills help you work effectively in cross-functional teams and address complex data challenges. These skills and qualities are crucial for building reliable, scalable data pipelines that support business-critical applications.

What does a Software Engineer Data Infrastructure Kafka do?

A Software Engineer Data Infrastructure Kafka specializes in designing, building, and maintaining large-scale data systems that use Apache Kafka for real-time data streaming and processing. This role involves developing robust pipelines, ensuring data reliability and scalability, and supporting the integration of Kafka with other data storage and analytics systems. Engineers in this position also monitor system performance, troubleshoot issues, and implement best practices for security and data management. They work closely with data engineers, application developers, and operations teams to deliver high-quality data solutions.

What are some common challenges faced by Software Engineers working on Data Infrastructure with Kafka, and how can they be addressed?

Software Engineers focusing on Data Infrastructure with Kafka often encounter challenges such as ensuring high availability, managing large-scale data throughput, and maintaining data consistency across distributed systems. Another common hurdle is tuning Kafka for optimal performance under varying workloads. These challenges can be addressed by implementing robust monitoring, practicing careful partitioning and replication strategies, and collaborating closely with DevOps and data engineering teams. Staying updated with Kafka's latest features and best practices also helps in proactively mitigating issues.
More about Software Engineer Data Infrastructure Kafka jobs
What states have the most Software Engineer Data Infrastructure Kafka jobs? States with the most job openings for Software Engineer Data Infrastructure Kafka jobs include:
What job categories do people searching Software Engineer Data Infrastructure Kafka jobs look for? The top searched job categories for Software Engineer Data Infrastructure Kafka jobs are:
Software Engineer Data Infrastructure Kafka jobs near you
Infographic showing various Software Engineer Data Infrastructure Kafka job openings in the United States as of June 2026, with employment types broken down into 76% Full Time, 14% Part Time, and 10% Contract. Highlights an 87% Physical, 3% Hybrid, and 10% Remote job distribution, with an average salary of $129,716 per year, or $62.4 per hour.
Software Engineer, Data Infrastructure

Software Engineer, Data Infrastructure

Peregrine Technologies

San Francisco, CA • On-site

$160K - $220K/yr

Full-time

Posted 17 days ago

Key responsibilities

  • Design and operate a high-throughput, real-time data integration platform across diverse customer environments.

  • Architect a scalable open table format layer for reliable data storage at petabyte scale.

  • Build and optimize distributed data processing pipelines with Apache Spark and adjacent streaming technologies.

Job description

Backed by leading Silicon Valley investors, Peregrine helps public safety organizations, state and local and governments, federal agencies, and private-sector institutions address society's challenges with unprecedented speed and accuracy. Our AI-enabled platform turns siloed and disconnected data into operational intelligence - instantly surfacing mission-critical information to empower better, faster decisions that improve outcomes at every touchpoint. Today Peregrine supports hundreds of customers across 30+ states and two countries, serving more than 125 million people - and we're amplifying our impact as we expand into the enterprise and internationally.
Team
As an engineering team, we believe strongly that empathy improves our solutions. Seeing how people use the product is a priority and the way we get to the right answer. Engineers will have the opportunity to work closely with our team onsite to understand the variety of use cases that Peregrine serves.
We value both ownership and collaboration-you will take full responsibility for major features and work closely with other engineers to drive them to completion. We believe that humility and empathy are essential for building the right solutions-you will collaborate directly with our deployment team and users as we iterate to solve their problems. Perseverance and creativity are crucial to executing our vision.
Role
We are looking for a Data Infrastructure Engineer to join our growing team, where you will have deep ownership over the data layer that underpins everything Peregrine does. You will join the team who will architect and build the systems that ingest, store, and serve massive volumes of real-time operational data - enabling our customers to make critical decisions with speed and confidence.
This is an individual contributor role for someone who thrives on hard technical problems and brings the experience and judgment to shape foundational infrastructure decisions. You will tackle a wide range of complex challenges, including:
  • Designing and operating a high-throughput, real-time data integration platform across diverse customer environments
  • Architecting a scalable open table format layer for reliable data storage at petabyte scale
  • Building and optimizing distributed data processing pipelines with Apache Spark and adjacent streaming technologies
  • Driving performance, reliability, and cost efficiency across the full data infrastructure stack
  • Collaborating with platform and product engineering teams to define data contracts, schemas, and integration patterns
  • Establishing best practices, tooling, and patterns that raise the quality bar for data infrastructure across the organization

Our stack is constantly evolving but is built on AWS GovCloud, Apache Iceberg, Apache Spark, Apache Kafka, Airflow, Kubernetes, and more.
About You
  • Deep passion for data infrastructure - you care about building systems that are correct, fast, and resilient at scale
  • Thrive on ambiguity and are energized by defining the right solution to hard, open-ended problems
  • Strong technical vision with the ability to translate complex data requirements into clean, durable infrastructure designs
  • Desire to own significant portions of the data stack end-to-end, from ingestion to serving
  • Committed to operational excellence - you build things you're proud to operate
What We Look For
  • 2-5 years of experience operating large-scale data infrastructure systems in production environments
  • Experience with open table formats, particularly Apache Iceberg - including schema evolution, partitioning strategies, compaction, and time travel
  • Extensive hands-on experience with Apache Spark for batch and streaming data processing at scale
  • Background in real-time data integration and stream processing, leveraging technologies such as Apache Kafka, Apache Flink, or equivalents
  • Experience with data pipeline orchestration using Airflow or similar tools
  • Strong software engineering fundamentals in Python and/or Scala, with a track record of writing production-quality code
  • Experience with AWS or comparable cloud platforms, including S3-based data lake architectures
  • Experience with Kubernetes and containerized deployment of data workloads
  • Degree in Computer Science, Engineering, or a related field, or equivalent practical experience
  • Located in San Francisco and open to working in office

Salary Range: $160,000 - $220,000 Annually + Benefits + Equity (if applicable) + Bonus (if applicable)
Actual compensation is influenced by a wide array of factors including but not limited to skill set, level of experience, certifications or licenses, and specific work location. Information on the benefits offered is here.
Pursuant to the San Francisco Fair Chance Ordinance, we will consider for employment qualified applicants with arrest and conviction records.
Peregrine Technologies is committed to creating an inclusive environment for all employees. We celebrate diversity and are a proud equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, gender, gender identity or expression, sexual orientation, national origin, genetics, disability, age, or veteran status.

Apply