ZipRecruiter

Log In

1

Ml Inference Jobs in Baltimore, MD (NOW HIRING)

Synergy ECP

AI/ML Engineer

Annapolis Junction, MD · On-site

They are seeking AI/ML Engineers to build, deploy, and maintain machine learning models and data ... inference • Collaborate with software and DevOps teams for integration • Monitor model ...

Synergy ECP

AI/ML Engineer

Annapolis, MD · On-site

$270K/yr

Build data pipelines and workflows for model training and inference * Collaborate with software and ... Python and ML frameworks (TensorFlow, PyTorch, Scikit-learn) * Experience with data processing and ...

Synergy ECP

AI/ML Engineer

Annapolis Junction, MD

$270K/yr

Build data pipelines and workflows for model training and inference * Collaborate with software and ... Python and ML frameworks (TensorFlow, PyTorch, Scikit-learn) * Experience with data processing and ...

Synergy ECP

AI/ML Engineer

Annapolis Junction, MD · On-site

$270K/yr

Build data pipelines and workflows for model training and inference * Collaborate with software and ... Python and ML frameworks (TensorFlow, PyTorch, Scikit-learn) * Experience with data processing and ...

Deloitte

AI Engineer Consultant

Baltimore, MD · Hybrid

Deliver governed datasets and feature engineering/serving for ML training and real-time inference (online/offline consistency, caching, latency SLOs, backfills). A successful candidate would possess ...

DCCA

CAP AI SE4

Annapolis Junction, MD · On-site

... inference, monitoring) • Experience with ML pipeline development, model deployment, and DevOps/MLOps • Experience with Amazon SageMaker is a plus • Excellent communication and collaboration ...

next page

Showing results 1-20

All JobsMl Inference JobsMl Inference Jobs in Baltimore, MD

Ml Inference information

See Baltimore, MD salary details

$37.3K

$122K

$195.3K

How much do ml inference jobs pay per year?

As of Aug 15, 2026, the average yearly pay for ml inference in Baltimore, MD is $121,958.00, according to ZipRecruiter salary data. Most workers in this role earn between $97,900.00 and $135,100.00 per year, depending on experience, location, and employer.

What is ML inference?

ML inference refers to the process of using a trained machine learning model to make predictions or decisions based on new data. After a model has been trained on historical data, inference is the phase where that model is deployed and used in real-world applications, such as recognizing speech, detecting objects in images, or recommending products. The focus in ML inference is on speed, efficiency, and scalability to ensure quick predictions, often in real time. This process is critical for practical applications like mobile apps, web services, and embedded systems. Optimizing inference involves reducing latency, memory usage, and computational requirements.

What is the difference between Ml Inference vs Data Scientist?

AspectML InferenceData Scientist
Required CredentialsKnowledge of machine learning models, programming skillsDegree in data science, statistics, or related fields
Work EnvironmentDeploying models in production, real-time data processingData analysis, model development, research
Industry UsageAI product deployment, software companiesResearch institutions, tech firms, consulting

ML Inference focuses on deploying trained models to make predictions on new data, often in real-time. Data Scientists develop and analyze models, working primarily in research and development. While both roles require understanding of machine learning, ML Inference emphasizes deployment and operationalization, whereas Data Scientists focus on model creation and analysis.

What are some common challenges faced by ML inference engineers when deploying models to production?

ML Inference Engineers often encounter challenges such as optimizing model latency and throughput to meet production requirements, ensuring compatibility with diverse hardware environments, and managing model versioning and updates without disrupting service. Additionally, balancing resource utilization and inference accuracy while monitoring real-time performance metrics is crucial. Collaboration with data scientists, DevOps, and software engineers is typically essential to streamline deployment and maintain robust, scalable inference pipelines.

What are the key skills and qualifications needed to thrive in ML inference?

To thrive in ML Inference, you need a solid background in machine learning principles, programming (Python or C++), and experience with deploying models at scale, often supported by a degree in computer science or a related field. Familiarity with frameworks and tools such as TensorFlow, PyTorch, ONNX, and cloud platforms like AWS SageMaker or Google AI Platform is typically required. Strong problem-solving skills, attention to detail, and effective communication are crucial soft skills for collaborating with multidisciplinary teams and optimizing model performance. These skills ensure efficient, scalable, and reliable deployment of machine learning solutions in real-world applications.

Is ML inference a high paying job?

ML inference roles are generally well-paying, especially for those with skills in machine learning frameworks, programming, and cloud platforms. Salaries vary based on experience, location, and industry, but they tend to be higher than average for tech-related positions.

What are popular job titles related to Ml Inference jobs in Baltimore, MD?

For Ml Inference jobs in Baltimore, MD, the most frequently searched job titles are:

What job categories do people searching Ml Inference jobs in Baltimore, MD look for?

The top searched job categories for Ml Inference jobs in Baltimore, MD are:

What cities near Baltimore, MD are hiring for Ml Inference jobs?

Cities near Baltimore, MD with the most Ml Inference job openings:

Ml Inference jobs near you

Software Engineer -- AI Inference

Intezra, Inc.

Columbia, MD • On-site

$130K - $160K/yr

Full-time

Dental, Vision, Retirement, PTO

Posted 11 days ago

Job description

Job Description

Software Engineer — AI Inference

Columbia, MD | Full Time | TS/SCI with Polygraph Required

Position: Software Engineer — AI Inference (Software Engineer, Level 1) — 2 positions

Location: Columbia, MD (on-site)

Category: Software Engineering / AI Infrastructure / LLM Inference

Clearance Requirement: Active TS/SCI with Polygraph

Compensation: $130,000 – $160,000 (annualized USD)

Experience Requirement:

  • 3+ years with a Bachelor's degree, OR

  • 4 additional years of relevant experience in lieu of a degree

Description

Join us in building the next generation of AI infrastructure that will power innovation across the customer organization. We're seeking software engineers to support our AI infrastructure team — helping build and maintain the foundation for customer AI capabilities while supporting a broader ecosystem of AI-enabled applications.

Your focus will be ensuring access to the highest-quality LLMs for users throughout the inference software stack. Requirements shift quickly as needs evolve and new technologies emerge, so you'll keep sharpening your skills and learning to turn loosely defined problems into working solutions.

Responsibilities

  • Procure, configure, and test new inference models, preparing them for release to the user base.

  • Develop in-house services and techniques to guarantee continual high-quality inference service for customers.

  • Work with model vendor teams and representatives to create reliable pipelines for closed-source model usage.

  • Collaborate with teammates on surge efforts to support short-term, high-priority inference needs.

  • Engage with other teams in the organization to establish solid infrastructure for services and integrate LLM-powered tools for user needs.

Skills Requirements

  • Experience with Python and/or other modern programming languages.

  • Familiarity with Argo CD and/or other CI/CD frameworks.

  • Experience with Kubernetes/Helm.

  • Familiarity with AWS or other cloud service providers.

  • Ability to learn new technologies quickly.

  • Strong communication skills and willingness to ask questions.

Nice to Haves

  • Experience with vLLM, LiteLLM, or similar inference-serving frameworks.

  • Experience with other LLM hosting frameworks and practices.

  • Experience supporting production software using Site Reliability Engineering (SRE) best practices.

  • Experience with Elastic, Grafana/Prometheus, or other observability frameworks and practices.

  • Experience with Docker and containerization.

Compensation Employment Policy

Salary is determined by multiple factors, including location, education, experience, skills, and organizational requirements. The projected compensation range for this position is $130,000 – $160,000 (annualized USD).

Benefits Overview

At Intezra, Inc., we offer a comprehensive benefits package designed to support long-term career growth and work-life balance:

  • Three CareFirst medical plans available; Intezra pays up to 100% of healthcare premiums and up to 100% of deductibles (based on plan selection) for employees and dependents

  • Intezra pays 100% for CareFirst Dental and Vision plans for employees and dependents

  • 401(k): 15% company contribution (no match required)

  • PTO: 160 hours, increasing with seniority

  • 12 Floating Holidays

  • 4 Code Red Days

EEO Statement

Intezra, Inc. provides equal employment opportunities to all employees and applicants and prohibits discrimination and harassment of any kind without regard to race, color, religion, age, sex, national origin, disability status, genetics, pregnancy, protected veteran status, sexual orientation, gender identity or expression, or any other characteristic protected by federal, state, or local laws.

About Us

Intezra Inc. is a small business prime contractor for all realms of cyber and AI/ML development, from tactical-level tools and capabilities to enterprise-level infrastructure and operations. Our leadership team and staff have helped solve the most complex challenges for the intelligence community for over 15 years.

, About Intezra, Inc.

Intezra Inc. is a small business prime contractor for all realms of cyber and AI/ML development, from tactical-level tools and capabilities to enterprise-level infrastructure and operations. Our leadership team and staff have helped solve the most complex challenges for the intelligence community for over 15 years.