ZipRecruiter

Log In

1

Llm Backend Engineer Jobs (NOW HIRING)

Braintrust

Software Engineer, Backend

New York, NY · On-site

What you'll do As a backend engineer at Braintrust, you'll help build the core platform powering LLM-native development: * Design and ship features that give users deep insight into their LLM usage ...

HelloGov AI

Back-End Engineer

Miami, FL · On-site

We are looking for a talented Back-End Software Engineer to join our on-site engineering team in ... Familiarity with AI-powered applications, LLM integrations, and modern AI tools. * Ability to ...

HelloGov AI

Back-End Engineer

Miami, FL

We are looking for a talented Back-End Software Engineer to join our on-site engineering team in ... Familiarity with AI-powered applications, LLM integrations, and modern AI tools. * Ability to ...

HelloGov AI

Back-End Engineer

Miami, FL · On-site

We are looking for a talented Back-End Software Engineer to join our on-site engineering team in ... Familiarity with AI-powered applications, LLM integrations, and modern AI tools. * Ability to ...

Pump.co

Backend Engineer

San Francisco, CA · On-site

$120K - $200K/yr

Python, FastAPI, PostgreSQL, AWS, Docker, Kubernetes (nice to have), LLM/AI (bonus) As a Backend Engineer , you'll contribute directly to the core of our product and infrastructure. This role is ...

Pump.co

Backend Engineer

San Francisco, CA

Python, FastAPI, PostgreSQL, AWS, Docker, Kubernetes (nice to have), LLM/AI (bonus) As a Backend Engineer , you'll contribute directly to the core of our product and infrastructure. This role is ...

Braintrust

Software Engineer, Backend

They are seeking a backend engineer to build the infrastructure behind cutting-edge AI development ... Responsibilities : • Design and ship features that give users deep insight into their LLM usage ...

Veris AI

Senior Backend Engineer

Manhattan, NY · On-site

They are seeking a Senior Backend Engineer to strengthen their engineering team by building the ... LLM tooling. • Solid grasp of engineering best practices: code reviews, automated testing ...

Town.com, Inc

Staff Backend Engineer

New York, NY · On-site

$250K - $300K/yr

You'll be a foundational hire, working across the full backend -- from LLM orchestration and ... The environment is greenfield, and the problems are unlike traditional backend engineering: non ...

Town.com, Inc

Staff Backend Engineer

New York, NY · On-site

$250K - $300K/yr

You'll be a foundational hire, working across the full backend -- from LLM orchestration and ... The environment is greenfield, and the problems are unlike traditional backend engineering: non ...

Arena Club

Staff Backend Engineer

San Francisco, CA · Remote

$220K - $270K/yr

About the Role We're seeking a Staff Backend Engineer to help scale the backbone of our platform ... AI Development Tools: AI-powered coding assistants (Claude Code, Cursor, etc.), LLM-based ...

Apple

Sr Python Backend Engineer, AIML

Cupertino, CA

$184K - $324K/yr

Build and integrate LLM-powered workflows, AI agents, and agentic pipelines into production backend services. Collaborate with front-end developers, product managers, and cross-functional ...

Arena Club

Staff Backend Engineer

San Francisco, CA · Remote

About the Role We're seeking a Staff Backend Engineer to help scale the backbone of our platform ... AI Development Tools: AI-powered coding assistants (Claude Code, Cursor, etc.), LLM-based ...

Arena Club

Staff Backend Engineer

San Francisco, CA · On-site

$220K - $270K/yr

About the Role We're seeking a Staff Backend Engineer to help scale the backbone of our platform ... AI Development Tools: AI-powered coding assistants (Claude Code, Cursor, etc.), LLM-based ...

previous pagenext page

Showing results 21-40

All JobsLlm Backend Engineer Jobs

Llm Backend Engineer information

See salary details

$60.5K

$147.7K

$199K

How much do llm backend engineer jobs pay per year?

As of Aug 11, 2026, the average yearly pay for llm backend engineer in the United States is $147,662.00, according to ZipRecruiter salary data. Most workers in this role earn between $124,000.00 and $172,000.00 per year, depending on experience, location, and employer.

What are some common challenges faced by LLM backend engineers when deploying large language models in production?

LLM Backend Engineers often encounter challenges such as optimizing inference latency, managing high resource consumption, and ensuring scalability for production workloads. Balancing model performance with cost efficiency requires careful selection of hardware, batching strategies, and model quantization techniques. Additionally, they must address security and privacy concerns associated with handling sensitive data processed by the models. Collaboration with data scientists and DevOps teams is essential to streamline model updates and monitor system health.

What is an LLM backend engineer?

LLM Backend Engineers are software engineers who specialize in designing, building, and optimizing the backend infrastructure that supports large language models (LLMs) like GPT-4. They focus on integrating LLMs into products and services, ensuring scalable APIs, managing data pipelines, and optimizing inference performance. Their work often involves deploying models in cloud environments, monitoring system reliability, and collaborating with AI researchers to bring advancements into production. LLM Backend Engineers play a critical role in making AI-powered applications robust, efficient, and accessible to end users.

What are the key skills and qualifications needed to thrive as an LLM backend engineer?

To thrive as an LLM Backend Engineer, you need a solid foundation in software engineering, backend architecture, and experience working with large language models, typically supported by a degree in computer science or a related field. Proficiency with programming languages like Python or Java, cloud platforms (AWS, GCP, Azure), and machine learning frameworks such as TensorFlow or PyTorch is essential, along with familiarity with APIs and containerization tools like Docker or Kubernetes. Strong problem-solving, collaboration, and communication skills distinguish top performers in this role. These skills ensure robust, scalable, and efficient deployment of LLM-powered applications while enabling effective teamwork and innovation.
More about Llm Backend Engineer jobs
What cities are hiring for Llm Backend Engineer jobs? Cities with the most Llm Backend Engineer job openings:
What states have the most Llm Backend Engineer jobs? States with the most job openings for Llm Backend Engineer jobs include:
What job categories do people searching Llm Backend Engineer jobs look for? The top searched job categories for Llm Backend Engineer jobs are:
Llm Backend Engineer jobs near you
Infographic showing various Llm Backend Engineer job openings in the United States as of August 2026, with employment types broken down into 92% Full Time, 2% Part Time, and 6% Contract. Highlights an 86% Physical, 5% Hybrid, and 9% Remote job distribution, with an average salary of $147,662 per year, or $71 per hour.

Software Engineer, Backend

Braintrust

New York, NY • On-site

Full-time

Medical, Dental, Vision

Re-posted 19 days ago

Job description

About the company
Braintrust is the agent observability platform. By actively applying intelligence to agent traces and automatically surfacing the most critical patterns, Braintrust gives teams the visibility to understand how agents behave in production and the tools to improve them.
Teams at Notion, Stripe, Box, OpenAI, and Cloudflare use Braintrust to trace their agents, find the issues in their observability data, and run evals that tell them how to improve.
About the role
We're looking for a backend engineer who's excited to build the infrastructure behind cutting-edge AI development tools.
Under the hood, Braintrust is a real-time, high-availability data platform that operates in both SaaS and self-hosted environments. You'll help scale our ingestion pipelines, evolve our open source libraries, and ensure everything runs reliably, observably, and fast. If you thrive in a fast-paced environment and enjoy solving hard systems problems with a product mindset, we'd love to work with you.
What you'll do
As a backend engineer at Braintrust, you'll help build the core platform powering LLM-native development:
  • Design and ship features that give users deep insight into their LLM usage and performance.
  • Build from scratch the unifying layer that proxies, caches, and normalizes calls across OpenAI, Anthropic, Gemini, and every other provider our customers depend on.
  • Develop and evolve robust, efficient open source libraries for tracing and evaluating LLM calls inside customer applications. Check out our SDKs
  • Build highly available real-time data pipelines to ingest, store, and query large volumes of structured and semi-structured AI usage data.

You will also collaborate closely with our product engineers to ship polished, end-to-end features that solve real problems for our customers, such as:
  • Bundling and uploading javascript and python code snippets to be used as online, on-demand function calls for LLM workflows.
  • Support for parsing and uploading attachments from LLM multimodal outputs into long term storage.
  • Role-based access controls and data retention policies for enterprise use cases.

Ideal candidate credentials
  • 5+ years of backend engineering experience across multiple parts of the stack
  • Strong systems thinking: you've owned or contributed to infrastructure projects and know how to design for scale, uptime, and data integrity
  • Proficient in modern backend technologies (e.g., Node.js, Go, Python, Rust, Postgres, Redis, Terraform, Docker, AWS)
  • Experience with observability tooling (e.g., Grafana, Prometheus, Datadog, or similar) and a passion for making systems operable, available and rock solid.
  • Comfortable with ambiguous problems and excited to partner across the company to ship impactful work
  • Clear communicator who documents decisions, shares context, and helps lift the team
Benefits include
  • Medical, dental, and vision insurance
  • Daily lunch, snacks, and beverages
  • Flexible time off
  • Competitive salary and equity
  • Wifi & cellphone stipend
Equal opportunity
Braintrust is an equal opportunity employer. All applicants will be considered for employment without attention to race, color, religion, sex, sexual orientation, gender identity, national origin, veteran or disability status.

Apply