ZipRecruiter

Log In

1

Llm Backend Engineer Jobs (NOW HIRING)

Arena Club

Staff Backend Engineer

Los Angeles, CA · On-site +1

$200K - $230K/yr

About the Role Join our fast-growing startup as a Staff Backend Engineer and help scale the ... AI Development Tools: AI-powered coding assistants (Claude Code, Cursor, etc.), LLM-based ...

Arena Club

Staff Backend Engineer

Los Angeles, CA · On-site

$200K - $230K/yr

About the Role Join our fast-growing startup as a Staff Backend Engineer and help scale the ... AI Development Tools: AI-powered coding assistants (Claude Code, Cursor, etc.), LLM-based ...

Arena Club

Staff Backend Engineer

Los Angeles, CA · On-site +1

About the Role Join our fast-growing startup as a Staff Backend Engineer and help scale the ... AI Development Tools: AI-powered coding assistants (Claude Code, Cursor, etc.), LLM-based ...

Apple

Sr Python Backend Engineer, AIML

Cupertino, CA · On-site

... LLM-powered workflows, AI agents, and modern AI-assisted development practices. Minimum ... backend development experience, with solid foundational programming skills (algorithms, data ...

You.com

Senior Backend Engineer

San Francisco, CA · Remote

About the Role We're looking for a strong, generalist backend engineer to help build the platform ... LLM/AI model integrations and products * Strong product instincts and a builder's mindset ...

Driver AI Inc.

Software Engineer - Backend

Austin, TX · Remote

... LLM expertise to tackle context engineering at scale. Driver builds the context layer for employees ... Software Engineer - Backend Location : Remote or Austin, TX About the Role Our core innovation, the ...

Driver AI Inc.

Software Engineer - Backend

Austin, TX · On-site +1

$175K - $275K/yr

... LLM expertise to tackle context engineering at scale. Driver builds the context layer for employees ... Software Engineer - Backend Location : Remote or Austin, TX About the Role Our core innovation, the ...

IntraEdge

Sr. Engineer - Backend

Woonsocket, RI · On-site

We are seeking a Senior Backend Engineer with strong experience in Functional Java , SpringBoot ... Familiarity with LLM concepts (prompt engineering, APIs for model access, basic NLP tasks)

Hebbia

Backend Engineer, Agents

Manhattan, NY · On-site

The Backend Engineer will leverage expertise in systems and software to build efficient solutions ... Preferred : • Experience building agentic systems or LLM enabled products. • Frequent user of ...

ClinDCast LLC

Python Backend Engineer

Sunnyvale, CA · On-site

$60 - $65/hr

We are seeking an experienced Python Backend Engineer to design, develop, and maintain scalable ... Knowledge of AI/LLM integration frameworks such as LangChain or LlamaIndex. Strong experience with ...

Hebbia

Backend Engineer, Agents

San Francisco, CA · On-site

The Backend Engineer will leverage expertise in systems and software to build efficient solutions ... Preferred : • Experience building agentic systems or LLM enabled products. • Frequent user of ...

Soulside AI

Backend Engineer (Remote)

You'll be responsible for designing and implementing scalable backend systems that process ... Design and optimize LLM integration pipelines, including prompt engineering, usage monitoring, and ...

previous page

Showing results 41-60

All JobsLlm Backend Engineer Jobs

Llm Backend Engineer information

See salary details

$60.5K

$147.7K

$199K

How much do llm backend engineer jobs pay per year?

As of Aug 12, 2026, the average yearly pay for llm backend engineer in the United States is $147,662.00, according to ZipRecruiter salary data. Most workers in this role earn between $124,000.00 and $172,000.00 per year, depending on experience, location, and employer.

What are some common challenges faced by LLM backend engineers when deploying large language models in production?

LLM Backend Engineers often encounter challenges such as optimizing inference latency, managing high resource consumption, and ensuring scalability for production workloads. Balancing model performance with cost efficiency requires careful selection of hardware, batching strategies, and model quantization techniques. Additionally, they must address security and privacy concerns associated with handling sensitive data processed by the models. Collaboration with data scientists and DevOps teams is essential to streamline model updates and monitor system health.

What is an LLM backend engineer?

LLM Backend Engineers are software engineers who specialize in designing, building, and optimizing the backend infrastructure that supports large language models (LLMs) like GPT-4. They focus on integrating LLMs into products and services, ensuring scalable APIs, managing data pipelines, and optimizing inference performance. Their work often involves deploying models in cloud environments, monitoring system reliability, and collaborating with AI researchers to bring advancements into production. LLM Backend Engineers play a critical role in making AI-powered applications robust, efficient, and accessible to end users.

What are the key skills and qualifications needed to thrive as an LLM backend engineer?

To thrive as an LLM Backend Engineer, you need a solid foundation in software engineering, backend architecture, and experience working with large language models, typically supported by a degree in computer science or a related field. Proficiency with programming languages like Python or Java, cloud platforms (AWS, GCP, Azure), and machine learning frameworks such as TensorFlow or PyTorch is essential, along with familiarity with APIs and containerization tools like Docker or Kubernetes. Strong problem-solving, collaboration, and communication skills distinguish top performers in this role. These skills ensure robust, scalable, and efficient deployment of LLM-powered applications while enabling effective teamwork and innovation.
More about Llm Backend Engineer jobs
What cities are hiring for Llm Backend Engineer jobs? Cities with the most Llm Backend Engineer job openings:
What states have the most Llm Backend Engineer jobs? States with the most job openings for Llm Backend Engineer jobs include:
What job categories do people searching Llm Backend Engineer jobs look for? The top searched job categories for Llm Backend Engineer jobs are:
Llm Backend Engineer jobs near you
Infographic showing various Llm Backend Engineer job openings in the United States as of August 2026, with employment types broken down into 92% Full Time, 2% Part Time, and 6% Contract. Highlights an 86% Physical, 5% Hybrid, and 9% Remote job distribution, with an average salary of $147,662 per year, or $71 per hour.

Founding Backend Engineer (AI Focus) - Atrix

Pear

New York, NY • On-site

Full-time

Medical, Dental, Vision, PTO

Re-posted 22 days ago

Job description

Breakthrough medicines all start the same way: years of research, mountains of data, and countless decisions made by medical teams within the world's premier biopharma labs.
These decisions then determine which treatments make it to market, how doctors learn about them, and ultimately which patients get access to life-saving care and when.
We're at an inflection point. AI-powered drug discovery, precision medicine, and an explosion of real-world evidence have accelerated what's possible, but the teams translating science into patient access are drowning in data they can't use.
The tools haven't kept up with the stakes.
Atrix builds purpose-built AI for life sciences, designed for the accuracy and rigor biopharma requires.
We're live with top-10 pharmaceutical companies, scaling fast, and focused on one thing: helping the right treatments reach the right patients, sooner
The Role Description
You'll be a core builder of Atrix's backend infrastructure, powering AI-native workflows that ingest unstructured data, structure it with AI, and deliver insights that matter-instantly and reliably.
You'll work on scalable data pipelines, robust APIs, and real-time inference systems, supporting everything from complex agent workflows to customer-facing dashboards. Whether it's building retrieval pipelines, optimizing async background workers, or designing clean, secure APIs, your work will be foundational to how our platform works and scales.
We're building a modular, agentic platform that connects to customer-approved data sources, transforms unstructured data using LLM-based agents, and delivers decision-ready intelligence to regulated teams.
You'll tackle rich backend engineering challenges like:
  • System Design & Reliability - Scale to handle massive amounts of records with fast response times
  • LLM Ops - Make AI outputs explainable, auditable, and efficient
  • Multi-Tenant Infra - Isolate customer data across parallelized pipelines
  • AI Performance Tuning - Optimize inference cost, caching, and latency
  • RAG Pipelines & Contextual Workflows - Connect structured + unstructured data for real-world usage

You'll work closely with the Engineering Lead and CEO, and collaborate across product, AI, and GTM. As one of our earliest backend engineers, you'll influence everything from our architecture to hiring bar.
This is an in-person role based in NYC, with flexibility for remote work on some days as needed.
Responsibilities
This role is perfect for someone who thrives on solving deep technical problems, loves optimizing systems for speed and scale, and gets energized by seeing their work used in real-world, high-stakes environments. If you're excited about powering AI workflows behind the scenes-and building infrastructure that's as elegant as it is reliable-we'd love to talk.
In this role, you will:
  • Design and own major modules of our backend architecture (APIs, workers, orchestration)
  • Optimize performance and reliability of data processing and AI inference systems
  • Build and maintain LLM-powered backend features and multi-step workflows
  • Collaborate with product and AI teams to launch performant, production-grade AI features
  • Improve observability, traceability, and modularity of backend systems
  • Lead customer-focused feature delivery with high attention to security and compliance

Must haves
  • 5+ years backend engineering experience in fast-paced product teams
  • 2+ years of Python experience, including async programming
  • Expertise with one or more frameworks: FastAPI, Flask, Django
  • Strong experience with SQL (PostgreSQL) and ORMs (SQLAlchemy or equivalent)
  • Experience building APIs, background workers, and distributed systems
  • Understanding of how LLMs, embeddings, or RAG workflows function
  • Startup experience or demonstrated ownership in complex backend projects

Nice to haves
  • Familiarity with Celery, LangChain, LlamaIndex, or other LLM orchestration tools
  • Exposure to RAG or context injection workflows
  • Experience with AWS, Terraform, Kafka, or PyTorch
  • Background in healthcare, data security, or regulated environments

Why Join Atrix as a Backend Engineer?
Mission-Driven Impact
Your code will help life sciences teams deliver better treatments-faster. Every system you build supports real-world clinical outcomes.
Real AI, Real Stakes
This isn't research tooling. You'll make AI work for real customers-ensuring reliability, explainability, and speed in high-stakes environments.
Deep Backend Ownership
Shape architecture, modular services, and engineering culture from day one.
Scale + Complexity
You'll design systems that operate on thousands of records, across multi-client, multi-agent environments-with LLMs at the core.
Early Team, High Growth
Be part of a tight-knit, mission-oriented team, and grow into our Backend Lead Engineer as we scale.
What We Offer
  • Compensation: Competitive salary + equity package
  • Health & wellness support - Stipend + medical (vision, dental, health) insurance coverage
  • Unlimited PTO - Recharge when you need to
  • High learning velocity - Get exposed to AI, infra, GTM, and real customers daily

Apply