Live In Kafka Jobs in Illinois (NOW HIRING)

Overview

Being good neighbors – helping people, investing in our communities, and making the world a better place – is who we are at State Farm. It is at the core of how we operate and the reason for our success. Come join a #1 team and do some good!

HYBRID: Qualified candidates must live or relocate within a 180‑mile radius of a hub location listed below and should plan to spend time working from home and some time working in the office as part of our hybrid work environment.

HUB LOCATIONS: Bloomington, IL; Dunwoody, GA; Richardson, TX; or Tempe, AZ

SPONSORSHIP: Applicants for this position are required to be eligible to lawfully work in the U.S. immediately; employer will not sponsor applicants for U.S. work authorization (e.g., H-1B visa) for this opportunity.

Grow Your Skills, Grow Your Potential Responsibilities

In State Farm's contact center environments, the quality of our data determines the quality of every decision we make across tens of millions of contacts annually. Our data engineering team builds the canonical models, event streams, and pipelines that turn raw contact‑center activity into trusted, governed, and analytics‑ready data products used across the enterprise. We’re building an event‑driven backbone to connect streaming data and intelligence to contact center leaders to enable rapid decision making.

• Define Canonical Data and Event Schemas – Contribute to our shared canonical model, the versioned types that govern every domain event crossing an API or system boundary.
• Design and Deliver Event‑Driven Architectures – Architect and implement scalable, resilient event‑based systems using AWS‑native messaging and streaming services including Kinesis, SQS, SNS, and EventBridge. Help shape the platform’s evolution toward Confluent Kafka as it becomes available to our workloads.
• Design and Build Data Pipelines – Architect and implement scalable event‑driven and batch pipelines using Kinesis and Kafka, Lambda, AWS Glue, Step Functions, and open‑source tooling to ingest, transform, and deliver enterprise data for use in AI and analytics applications.
• Build Change‑Data‑Capture Pipelines – Implement CDC pipelines from operational stores (DynamoDB Streams, Neptune Streams, RDS logical replication) into canonical domain events on the event backbone.
• Build Materialized Read Models – Design and build operational read projections in DynamoDB, OpenSearch, and Aurora that turn streaming state into sub‑second queryable derived state for leader‑facing applications, APIs, and near real‑time dashboards.
• Model Enterprise Data – Design and maintain dimensional models, entity‑relationship models, and semantic layer definitions using dbt to power analytics, reporting, and AI/ML workloads.
• Engineer the Lakehouse Transition – Help design and execute the path from our current AWS‑native analytics stack toward an enterprise lakehouse on Databricks, Delta Lake, and Iceberg.
• Develop Data Access APIs – Build data service APIs that expose governed, well‑documented data products to downstream consumers including analytics tools and AI systems.
• Engineer Derived‑Fact Write‑Back – Build the surface that lets ML models and AI agents write inferred facts back to the event backbone with confidence and provenance fields, consumable like any other state.
• Enable AI/ML Data Readiness – Prepare and deliver feature‑engineered datasets, vector embeddings, and data pipelines for AI and GenAI workloads.
• Champion DevSecOps for Data – Integrate data pipelines into CI/CD workflows and apply observability for pipeline health and data freshness.
• Document and Share Knowledge – Author data dictionaries, pipeline runbooks, and data model documentation; contribute to internal data communities of practice.
• Mentor and Collaborate – Provide technical leadership, code reviews, and guidance to engineers; advocate for data engineering best practices across teams.

Qualifications

Preferred Skills

• Hands‑on experience designing and building event‑driven data pipelines on Kafka or Kinesis with idempotent consumers, schema evolution, and dead‑letter handling.
• Experience designing canonical or shared data models that survive across multiple consumers and versions over time.
• Experience with change‑data‑capture (CDC) patterns from operational stores (DynamoDB, RDS, graph) into event streams.
• Experience designing materialized read models that power sub‑second operational queries from streaming state – or comparable experience translating streaming state into queryable form.
• Experience with at‑least‑once vs. exactly‑once delivery semantics and strategies for idempotent event consumers.
• Hands‑on experience with the Databricks Lakehouse Platform (Lakehouse, Delta Lake, Unity Catalog, Workflows, Databricks SQL).
• Proficiency in data modeling: dimensional modeling, star/snowflake schemas, entity‑relationship design, and dbt‑based semantic layer definition.
• Strong PySpark and/or SQL skills for large‑scale data transformation, validation, and pipeline development.
• Experience with Delta Lake and modern lakehouse architectures (Iceberg) and cloud data warehouses (Redshift, Databricks SQL Warehouse).
• Familiarity with data quality frameworks, data lineage, and cataloging using Unity Catalog or similar governance platforms.
• Experience integrating data pipelines into CI/CD workflows using GitLab and Terraform (Databricks Asset Bundles a plus).
• Understanding of data governance, access control, and compliance requirements for sensitive enterprise data.
• Proficiency with AI‑assisted development tools (GitHub Copilot) to accelerate delivery.
• Strong analytical and communication skills with ability to translate business requirements into scalable data models.

Ideal Candidates Also Have

• Experience in insurance, financial services, or other regulated industries with strict data governance requirements.
• Familiarity with cloud contact center applications.
• Experience building data products consumed by multiple teams including analytics, AI/ML, and executive reporting.
• Background with real‑time or near real‑time streaming pipelines using Amazon OpenSearch, Kinesis, EventBridge, and event‑driven architectures.
• Familiarity with graph data models and graph databases (Neptune, Neo4j) for relationship‑rich data domains.
• Prior experience leading or contributing to a migration from a traditional warehouse stack onto a lakehouse platform (Redshift/Snowflake → Databricks/Iceberg).

Our Benefits

Because work‑life balance is a priority at State Farm, compensation is based on our standard 38:45‑hour work week!

• Potential starting salary range: $81,000 – $142,000 (Starting salary will be based on skills, background, and experience. High end of the range limited to applicants with significant relevant experience.)
• Potential yearly incentive pay up to 15% of base salary.

At State Farm, we offer more than just a paycheck. Check out our suite of benefits designed to give you the flexibility you need to take care of you and your family!

• Get Paid! On top of our competitive pay, you are eligible for an annual raise and bonus.
• Stay Well! Focus on you and your family’s health with our robust health and wellbeing programs. State Farm pays most of your healthcare premium, and we offer multiple healthcare plan options, including a high deductible plan. All medical plans provide 100% coverage for in‑network preventative care and you and your family have access to vision, dental, telemedicine, 24/7 mental health professionals, and much more.
• Develop and Grow! Take advantage of educational benefits like industry‑leading training programs, top‑notch tuition assistance programs, employee resource groups, and mentoring.
• Plan Ahead! Plan for those big moments in life with benefits like fertility/IVF/adoption assistance, college coaching, national discount programs, interactive monthly financial workshops, free financial coaching, and more. You can also start a savings account or consider financing through our State Farm Federal Credit Union.
• Take a Little “You” Time! You will have access to our generous time off policies designed so you can plan around holidays, family events, volunteering, or just to take a relaxing day off. With the opportunity to initially earn up to 20 days annually plus parental leave, paid holidays, celebration day, life leave (40 hours/year), bereavement leave, and community service/education support days, there will be plenty of time for you.
• Give Back! We offer several ways to give back through our Matching Gift Program, Good Neighbor Grant Program, and the Employee Assistance Fund.
• Finish Strong! Plan for retirement using free financial advisors and a 401(k) plan with company contributions of up to 7% of your salary.