ZipRecruiter

Log In

1

Ml Inference Jobs in Baltimore, MD (NOW HIRING)

vetegrity

AI System Engineer

Columbia, MD · On-site

AI/ML system design and implementation * High-level understanding of the AI lifecycle (development, training, inference, monitoring) * Experience with ML pipeline development, model deployment, and ...

SimpliGov

Senior AI/ML Engineer

Baltimore, MD · On-site

$103K - $142K/yr

Operateinside our compliance boundary: self-hosted observability, GovCloud inference paths ... ownership, including 2+ years shipping LLM or ML-backed features to real users

New

Parsons

Senior Software Developer (MLOps)

Aberdeen, MD · On-site

$58.50 - $77.50/hr

Collaborate with data scientists and ML engineers to productionize models, including feature pipelines, inference services, and monitoring * Implement and maintain end-to-end MLOps workflows ...

SimpliGov LLC

Senior AI/ML Engineer

Baltimore, MD · On-site

$103K - $142K/yr

Operate inside our compliance boundary: self-hosted observability, GovCloud inference paths ... ML-backed features to real users * Demonstrated evaluation discipline: you can show how you ...

SimpliGov LLC

Senior AI/ML Engineer

Baltimore, MD · Remote

$107K - $146K/yr

Operate inside our compliance boundary: self-hosted observability, GovCloud inference paths ... ML-backed features to real users * Demonstrated evaluation discipline: you can show how you ...

New

Techximius

AI Software Engineer - Java

Aberdeen, MD · On-site

$130K - $160K/yr

... ML frameworks, tools, or libraries such as TensorFlow, PyTorch, OpenAI APIs, LangChain, Hugging Face, or similar technologies is a plus. * Understanding of data pipelines, model deployment, inference ...

NetSage

AI Software Engineer-Principal

Annapolis Junction, MD · On-site

$148K - $198K/yr

The ideal candidate for this position will possess deep expertise in AI/ML technologies, software ... Strong understanding of statistical modeling, probability theory, Bayesian inference, covariance ...

previous pagenext page

Showing results 21-40

All JobsMl Inference JobsMl Inference Jobs in Baltimore, MD

Ml Inference information

See Baltimore, MD salary details

$37.3K

$122K

$195.3K

How much do ml inference jobs pay per year?

As of Aug 15, 2026, the average yearly pay for ml inference in Baltimore, MD is $121,958.00, according to ZipRecruiter salary data. Most workers in this role earn between $97,900.00 and $135,100.00 per year, depending on experience, location, and employer.

What is ML inference?

ML inference refers to the process of using a trained machine learning model to make predictions or decisions based on new data. After a model has been trained on historical data, inference is the phase where that model is deployed and used in real-world applications, such as recognizing speech, detecting objects in images, or recommending products. The focus in ML inference is on speed, efficiency, and scalability to ensure quick predictions, often in real time. This process is critical for practical applications like mobile apps, web services, and embedded systems. Optimizing inference involves reducing latency, memory usage, and computational requirements.

What is the difference between Ml Inference vs Data Scientist?

AspectML InferenceData Scientist
Required CredentialsKnowledge of machine learning models, programming skillsDegree in data science, statistics, or related fields
Work EnvironmentDeploying models in production, real-time data processingData analysis, model development, research
Industry UsageAI product deployment, software companiesResearch institutions, tech firms, consulting

ML Inference focuses on deploying trained models to make predictions on new data, often in real-time. Data Scientists develop and analyze models, working primarily in research and development. While both roles require understanding of machine learning, ML Inference emphasizes deployment and operationalization, whereas Data Scientists focus on model creation and analysis.

What are some common challenges faced by ML inference engineers when deploying models to production?

ML Inference Engineers often encounter challenges such as optimizing model latency and throughput to meet production requirements, ensuring compatibility with diverse hardware environments, and managing model versioning and updates without disrupting service. Additionally, balancing resource utilization and inference accuracy while monitoring real-time performance metrics is crucial. Collaboration with data scientists, DevOps, and software engineers is typically essential to streamline deployment and maintain robust, scalable inference pipelines.

What are the key skills and qualifications needed to thrive in ML inference?

To thrive in ML Inference, you need a solid background in machine learning principles, programming (Python or C++), and experience with deploying models at scale, often supported by a degree in computer science or a related field. Familiarity with frameworks and tools such as TensorFlow, PyTorch, ONNX, and cloud platforms like AWS SageMaker or Google AI Platform is typically required. Strong problem-solving skills, attention to detail, and effective communication are crucial soft skills for collaborating with multidisciplinary teams and optimizing model performance. These skills ensure efficient, scalable, and reliable deployment of machine learning solutions in real-world applications.

Is ML inference a high paying job?

ML inference roles are generally well-paying, especially for those with skills in machine learning frameworks, programming, and cloud platforms. Salaries vary based on experience, location, and industry, but they tend to be higher than average for tech-related positions.

What are popular job titles related to Ml Inference jobs in Baltimore, MD?

For Ml Inference jobs in Baltimore, MD, the most frequently searched job titles are:

What job categories do people searching Ml Inference jobs in Baltimore, MD look for?

The top searched job categories for Ml Inference jobs in Baltimore, MD are:

What cities near Baltimore, MD are hiring for Ml Inference jobs?

Cities near Baltimore, MD with the most Ml Inference job openings:

Ml Inference jobs near you

Senior Principal Data Scientist / AI-ML SME (Analytic Superiority)

Leidos

Fort George G Meade, MD • On-site

$154K - $278K/yr

Full-time

Re-posted 16 hours ago

Leidos rating

8.3

Company rating: 8.3 out of 10

Based on 152 frontline employees who took The Breakroom Quiz

79th of 492 rated business services

Job description

Mission Overview
The Leidos Intel Sector is looking for a premier AI/ML Subject Matter Expert (SME) to serve as a Technical Closer for our COSS 3.0 program supporting USCYBERCOM and the Cyber National Mission Force (CNMF) at Fort Meade, MD. In this elite role, you will architect and engineer the cross-platform AI frameworks required to achieve absolute analytic superiority.
You will compress both defensive cyber operations (identifying network vulnerabilities and gaps) and offensive operations (vulnerability discovery and automated targeting) from days down to minutes. As a Technical Closer, you will mentor senior technologists by example-working "fingers-on-keyboard" to solve the command's most complex technical roadblocks and pushing production-grade AI directly into multi-cloud, hybrid, and air-gapped mission enclaves.
Core Technical Requirements
  • Platform-Agnostic Infrastructure & MLOps: Architect, deploy, and scale distributed AI workloads across any environment required, including AWS SageMaker, Google Vertex AI, Azure Government, and bare-metal, air-gapped server racks.
  • Agentic AI & Cyber Automation: Deploy and optimize tools like LangGraph, CrewAI, or AutoGPT to automate cyber threat identification and offensive target generation at wire speed.
  • Low-Level Model Engineering & Optimization: Fine-tune open-source large language models (e.g., Llama 3, Mistral) inside secure enclaves using PyTorch or TensorFlow. Utilize NVIDIA TensorRT, Triton Inference Server, vLLM, and quantization libraries (bitsandbytes) to compress models for high-throughput execution under strict hardware constraints.
  • Advanced RAG Architectures: Direct the engineering of enterprise Retrieval-Augmented Generation (RAG) stacks using LangChain paired with high-performance vector databases like Milvus, Qdrant, or Pinecone.
  • Autonomous Cyber Integration: Connect intelligent agents directly into security orchestration platforms (e.g., Palo Alto Cortex XSIAM/XSOAR) to trigger automated network defense actions and ingest massive, real-time PCAP and telemetry streams via Apache Kafka/Spark.
  • Polyglot Engineering: Demonstrate engineering mastery in Python, Go, Rust, and C/C++ to build ultra-fast cyber tools, write optimized GPU kernels, and interface with distributed frameworks like Ray.

Mission & Domain Expertise
  • Dual-Spectrum Operations: Proven capability to support both Defensive Cyber Operations (DCO) (log parsing, behavioral threat hunting, anomaly detection) and Offensive Cyber Operations (OCO) (automated vulnerability discovery, exploit generation, payload optimization).
  • Mission Platform Orchestration: Experience integrating custom AI/ML pipelines into unified mission systems and high-value data streams found across Project Maven, Palantir Foundry, and tactical command frameworks.

Required Experience & Background
  • Total Technical Experience: 15+ years of hands-on experience in software engineering, data science, or distributed systems.
  • Core AI/ML Focus: 5+ years of specialized experience in Machine Learning Engineering, deep learning, or LLM optimization.
  • DoD/IC Ecosystem: 3-5 years working within the DoD/IC cyber ecosystem, specifically building tools that map vulnerabilities or accelerate targeting cycles.
  • Clearance: Active TS/SCI with Polygraph..
  • Work Location: On-site at Fort Meade, MD (SCIF environment).

Preferred Certifications & Military Equivalency
  • Industry Certifications: Google Cloud Professional Machine Learning Engineer, AWS Certified Machine Learning - Specialty, or NVIDIA Generative AI/LLM Associate.
  • Cyber Mission Force (CMF) Equivalency: Prior certification as a CMF Exploitation Analyst (EA), Digital Network Analyst (DNA), or specialized technical experience as an Army 17A/170A, Navy 181X, or Air Force 17D/17S.

Educational Background
  • Primary Requirement: Master's Degree or PhD in Data Science, Artificial Intelligence, Computer Science, Mathematics, or a related quantitative field. Additional years of experience may be considered in lieu of degree.

If you're looking for comfort, keep scrolling. At Leidos, we outthink, outbuild, and outpace the status quo - because the mission demands it. We're not hiring followers. We're recruiting the ones who disrupt, provoke, and refuse to fail. Step 10 is ancient history. We're already at step 30 - and moving faster than anyone else dares.
Original Posting:
July 16, 2026
For U.S. Positions: While subject to change based on business needs, Leidos reasonably anticipates that this job requisition will remain open for at least 3 days with an anticipated close date of no earlier than 3 days after the original posting date as listed above.
Pay Range:
Pay Range $154,050.00 - $278,475.00
The Leidos pay range for this job level is a general guideline only and not a guarantee of compensation or salary. Additional factors considered in extending an offer include (but are not limited to) responsibilities of the job, education, experience, knowledge, skills, and abilities, as well as internal equity, alignment with market data, applicable bargaining agreement (if any), or other law.

What Leidos employees say

Pay

Benefits

Hours and flexibility

Workplace

Get the full story on Breakroom

Leidos logo

About Leidos

Sourced by ZipRecruiter

At Leidos, we deliver innovative solutions through the efforts of our diverse and talented people who are dedicated to our customers' success. We empower our teams, contribute to our communities, and operate sustainable practices. Everything we do is built on a commitment to do the right thing for our customers, our people, and our community.

Industry

It services

Company size

10,000+ Employees

Headquarters location

Reston, VA, US

Social media

FacebookTwitter

View All Leidos Jobs

Apply