ZipRecruiter

Log In

1

Ml Inference Jobs in Hanover, MD (NOW HIRING)

Synergy ECP

AI/ML Engineer

Annapolis Junction, MD · On-site

They are seeking AI/ML Engineers to build, deploy, and maintain machine learning models and data ... inference • Collaborate with software and DevOps teams for integration • Monitor model ...

Commence

AI/ML Engineer

Washington, DC · On-site

$125K - $155K/yr

Experience building APIs or services (e.g., FastAPI, Flask) for model inference. * Solid ... ML workflows. * Experience working with healthcare datasets such as EHRs, claims, FHIR, or HL7.

Ignite Digital

Mid AI/ML Engineer

Washington, DC · Hybrid

Build RAG architectures and high-performing inference systems. * Create scalable infrastructure to support training, experimentation, and deployment. * Lead integration of AI/ML models into ...

Expression

AI/ML Engineer

Washington, DC · On-site +1

$130K - $170K/yr

Develop efficient inference pipelines supporting heterogeneous compute environments ranging from ... AI/ML, software engineering, information technology, and electromagnetic spectrum management ...

Expression Networks

AI/ML Engineer

Washington, DC · On-site

$140 - $190/hr

Develop efficient inference pipelines supporting heterogeneous compute environments ranging from ... AI/ML, software engineering, information technology, and electromagnetic spectrum management ...

Oddball

Applied AI/ML Engineer

Washington, DC · On-site +1

Implement data pipelines and feature engineering workflows to support model training and inference ... Proficiency in Python and common ML libraries (e.g., PyTorch, TensorFlow, scikit-learn)

Oddball

Applied AI/ML Engineer

Washington, DC · On-site

Implement data pipelines and feature engineering workflows to support model training and inference ... Proficiency in Python and common ML libraries (e.g., PyTorch, TensorFlow, scikit-learn)

Xometry

Senior Principal, ML/AI

North Bethesda, MD

$125K - $173K/yr

... inference latency, model retraining intervals, feature pipelines, cost of compute, or automation of ... Demonstrated experience architecting and delivering large scale ML/AI solutions - end-to-end from ...

next page

Showing results 1-20

All JobsMl Inference JobsMl Inference Jobs in Hanover, MD

Ml Inference information

See Hanover, MD salary details

$37.3K

$122.1K

$195.5K

How much do ml inference jobs pay per year?

As of Aug 27, 2026, the average yearly pay for ml inference in Hanover, MD is $122,124.00, according to ZipRecruiter salary data. Most workers in this role earn between $98,000.00 and $135,300.00 per year, depending on experience, location, and employer.

What is ML inference?

ML inference refers to the process of using a trained machine learning model to make predictions or decisions based on new data. After a model has been trained on historical data, inference is the phase where that model is deployed and used in real-world applications, such as recognizing speech, detecting objects in images, or recommending products. The focus in ML inference is on speed, efficiency, and scalability to ensure quick predictions, often in real time. This process is critical for practical applications like mobile apps, web services, and embedded systems. Optimizing inference involves reducing latency, memory usage, and computational requirements.

What are the key skills and qualifications needed to thrive in ML inference?

To thrive in ML Inference, you need a solid background in machine learning principles, programming (Python or C++), and experience with deploying models at scale, often supported by a degree in computer science or a related field. Familiarity with frameworks and tools such as TensorFlow, PyTorch, ONNX, and cloud platforms like AWS SageMaker or Google AI Platform is typically required. Strong problem-solving skills, attention to detail, and effective communication are crucial soft skills for collaborating with multidisciplinary teams and optimizing model performance. These skills ensure efficient, scalable, and reliable deployment of machine learning solutions in real-world applications.

What are some common challenges faced by ML inference engineers when deploying models to production?

ML Inference Engineers often encounter challenges such as optimizing model latency and throughput to meet production requirements, ensuring compatibility with diverse hardware environments, and managing model versioning and updates without disrupting service. Additionally, balancing resource utilization and inference accuracy while monitoring real-time performance metrics is crucial. Collaboration with data scientists, DevOps, and software engineers is typically essential to streamline deployment and maintain robust, scalable inference pipelines.

What is the difference between Ml Inference vs Data Scientist?

AspectML InferenceData Scientist
Required CredentialsKnowledge of machine learning models, programming skillsDegree in data science, statistics, or related fields
Work EnvironmentDeploying models in production, real-time data processingData analysis, model development, research
Industry UsageAI product deployment, software companiesResearch institutions, tech firms, consulting

ML Inference focuses on deploying trained models to make predictions on new data, often in real-time. Data Scientists develop and analyze models, working primarily in research and development. While both roles require understanding of machine learning, ML Inference emphasizes deployment and operationalization, whereas Data Scientists focus on model creation and analysis.

What are popular job titles related to Ml Inference jobs in Hanover, MD?

For Ml Inference jobs in Hanover, MD, the most frequently searched job titles are:

What job categories do people searching Ml Inference jobs in Hanover, MD look for?

The top searched job categories for Ml Inference jobs in Hanover, MD are:

What cities near Hanover, MD are hiring for Ml Inference jobs?

Cities near Hanover, MD with the most Ml Inference job openings:

Ml Inference jobs near you

Machine Learning Engineer (GoLang)

Washington, DC

Comcast
See more jobs at Comcastexternal

$142K - $213K/yr

Full-time

Re-posted 2 days ago

Job description

Make your mark at Comcast -- a Fortune 30 global media and technology company. From the connectivity and platforms we provide, to the content and experiences we create, we reach hundreds of millions of customers, viewers, and guests worldwide. Become part of our award-winning technology team that turns big ideas into cutting-edge products, platforms, and solutions that our customers love. We create space to innovate, and we recognize, reward, and invest in your ideas, while ensuring you can proudly bring your authentic self to the workplace. Join us. You’ll do the best work of your career right here at Comcast. (In most cases, Comcast prefers to have employees on-site collaborating unless the team has been designated as virtual due to the nature of their work. If a position is listed with both office locations and virtual offerings, Comcast may be willing to consider candidates who live greater than 100 miles from the office for the remote option.)

Job Summary

Multimodal Analysis Framework (MAF)** is an end‑to‑end platform designed to process diverse content sources—including **video, images, audio, and documents**—to generate rich, structured metadata. The platform unifies multiple ML/AI models to extract curated insights at scale, tailored to specific business needs. MAF supports both **on‑demand** workloads (batch uploads, ad‑hoc analysis) and **real‑time streaming** workflows, enabling continuous metadata generation for live content streams. Customers can define their metadata requirements—such as entity extraction, scene segmentation, object detection, transcription, summarization, or multimodal correlation—and the framework orchestrates the appropriate models and toolchains to deliver high‑quality outputs. Through flexible APIs and UI‑based workflows, customers and internal teams can visualize metadata, trigger enrichment, monitor processing, and integrate results into downstream applications. The platform emphasizes modularity, scalability, and extensibility to support new ML models, LLM‑based agents, and cross‑modal inference as use cases evolve. We are looking for a **mid-level Backend Engineer** to join our **Machine Learning Platform team**. This role focuses on building **scalable backend systems** that power ML workloads, including **video, image, and document processing**, and enable **LLM-driven applications** through **agents and MCP servers**. You will work primarily in **Golang**, deploy and operate services on **Kubernetes**, manage infrastructure with **Terraform**, and build on **AWS**. A core part of the role is designing platform capabilities that allow **LLMs to safely and reliably interact with tools, data, and services** via **agent frameworks and MCP servers**.

Job Description

Backend Engineering (Golang)

  • Design, build, and maintain **high-performance backend services** in **Golang** for ML and AI platform use cases.
  • Develop **REST and gRPC APIs** for inference, processing pipelines, orchestration, and platform services.
  • Implement asynchronous and distributed processing patterns (workers, queues, event-driven systems).
  • Ensure backend services meet production standards for **scalability, reliability, and security**.

ML Platform & Processing Pipelines 

  • Build and operate backend systems supporting:
    • Video processing** (frame extraction, metadata generation, embeddings, indexing).
    • Image processing** (OCR, classification, detection, embedding generation).
    • Document processing** (parsing, layout analysis, chunking, OCR, retrieval pipelines).
  • Integrate ML inference services into backend workflows with attention to **latency, throughput, and cost**.
  • Work closely with ML engineers and data scientists to productionize models and pipelines.

LLMs, Agents, and MCP Servers

  • Build **LLM-enabled backend services** using structured prompting, tool/function calling, and retrieval-augmented generation (RAG).
  • Design and implement **agentic workflows** (multi-step reasoning, tool orchestration, retries, guardrails).
  • Develop and operate **MCP servers** that expose internal platform capabilities (search, retrieval, processing, data access) to LLM-based applications.
  • Enforce **security, access control, and observability** for agent and MCP interactions.

Vector Search & Retrieval 

  • Design and maintain vector-based retrieval systems using **Milvus**.
  • Implement embedding ingestion, indexing, and query pipelines at scale.
  • Optimize retrieval quality, latency, and relevance for downstream LLM applications.

Cloud, Kubernetes & Infrastructure 

  • Deploy and operate backend and ML services on **Kubernetes** (scaling, rollouts, resource management).
  • Use **Terraform** for infrastructure provisioning and continuous delivery of cloud resources.
  • Build and operate primarily on **AWS**, leveraging services such as:
    • Compute, networking, and IAM
    • Object storage
    • Managed Kubernetes
    • Logging and monitoring services

Reliability, Quality & Operations

  • Implement observability using logs, metrics, and traces; define SLOs and alerts.
  • Write automated tests (unit, integration) and contribute to CI/CD pipelines.
  • Participate in on-call rotations and incident response; drive post-incident improvements.

Required Qualifications 

  • **3–6 years** of professional software engineering experience.
  • Strong backend engineering experience with **Golang**.
  • Experience building and operating **APIs** (REST and/or gRPC) in production.
  • Hands-on experience with **Kubernetes** in production environments.
  • Experience using **Terraform** for infrastructure provisioning and deployment.
  • Solid working knowledge of **AWS** cloud services and core architectural concepts.
  • Experience building or supporting **ML processing pipelines** (video, image, or document).
  • Practical experience using **LLMs** in production systems.
  • Experience developing **agents** and/or **MCP servers**, or equivalent tool-integration platforms.

Preferred / Nice-to-Have Qualifications 

  • Experience with **Milvus** or other vector databases in production.
  • Familiarity with GPU-backed workloads and ML inference optimization.
  • Experience with messaging/streaming systems (Kafka, SQS, SNS, etc.).
  • Knowledge of secure system design for AI platforms (IAM, secrets management, least-privilege access).
  • Experience working on internal developer platforms or ML infrastructure teams.

Disclaimer:

  • This information has been designed to indicate the general nature and level of work performed by employees in this role. It is not designed to contain or be interpreted as a comprehensive inventory of all duties, responsibilities and qualifications.

Comcast is an equal opportunity workplace. We will consider all qualified applicants for employment without regard to race, color, religion, age, sex, sexual orientation, gender identity, national origin, disability, veteran status, genetic information, or any other basis protected by applicable law.


Skills:

Kubernetes; Cloud Platform; Collaboration; Large Language Models (LLMs); Model Context Protocol; Go Programming Language


Salary:

District of Columbia Pay Range: $142,651.46 - $213,977.19

Comcast intends to offer the selected candidate base pay within this range, dependent on job-related, non-discriminatory factors such as experience. The application window is 30 days from the date job is posted, unless the number of applicants requires it to close sooner or later.


Base pay is one part of the Total Rewards that Comcast provides to compensate and recognize employees for their work. Most sales positions are eligible for a Commission under the terms of an applicable plan, while most non-sales positions are eligible for a Bonus. Additionally, Comcast provides best-in-class Benefits to eligible employees. We believe that benefits should connect you to the support you need when it matters most, and should help you care for those who matter most. That’s why we provide an array of options, expert guidance and always-on tools, that are personalized to meet the needs of your reality - to help support you physically, financially and emotionally through the big milestones and in your everyday life. Please visit the compensation and benefits summary on our careers site for more details.


Education

Bachelor's Degree

While possessing the stated degree is preferred, Comcast also may consider applicants who hold some combination of coursework and experience, or who have extensive related professional experience.

Relevant Work Experience

5-7 Years

Apply