ZipRecruiter

Log In

1

Ml Inference Jobs in Raleigh, NC (NOW HIRING)

CVS Health

Staff Software Architect

Raleigh, NC · On-site

Lead the design of highly scalable, fault-tolerant, and cost-optimized platforms supporting clinical AI/ML workloads - including LLM inference pipelines, document processing at scale, and real-time ...

New

Redhat

Senior Software Engineer, AI Inference

Raleigh, NC · On-site +1

$133K - $220K/yr

The Red Hat AI Inference team accelerates AI for the enterprise and brings operational simplicity ... D. in an ML-related domain is a significant advantage. The following is considered a plus:

General Motors

Staff ML Compiler Engineer

Raleigh, NC · On-site

We own the compiler that turns high-level models into fast, reliable inference across GPUs powering ... Experience with ML frameworks (e.g.,PyTorch, TensorFlow, JAX) and software stack (e.g.,ONNX,MLIR ...

Truist

ML Platform Engineer

Raleigh, NC · On-site

... inference services, and monitoring frameworks. 5. Proven experience with MLOps and platform engineering practices , including CI/CD for ML, automated deployment, lifecycle management, and ...

Deloitte

AI Engineer Consultant

Raleigh, NC · Hybrid

Deliver governed data and features for ML/GenAI (curated datasets, feature pipelines/serving) supporting training and real-time inference, including consistency, caching, backfills, and latency SLOs.

Deloitte

AI Engineer Senior Consultant

Raleigh, NC · Hybrid

$101K - $139K/yr

Deliver governed datasets and feature engineering/serving for ML training and real-time inference (online/offline consistency, caching, latency SLOs, backfills). * Implement safety, privacy, and ...

Deloitte

AI Data Engineer - Senior Consultant

Raleigh, NC · Hybrid

$101K - $139K/yr

Deliver governed datasets and feature engineering/serving for ML training and real-time inference (online/offline consistency, caching, latency SLOs, backfills). * Implement safety, privacy, and ...

next page

Showing results 1-20

All JobsMl Inference JobsMl Inference Jobs in Raleigh, NC

Ml Inference information

See Raleigh, NC salary details

$36.5K

$119.3K

$191K

How much do ml inference jobs pay per year?

As of Jun 24, 2026, the average yearly pay for ml inference in Raleigh, NC is $119,312.00, according to ZipRecruiter salary data. Most workers in this role earn between $95,800.00 and $132,200.00 per year, depending on experience, location, and employer.

What is ML inference?

ML inference refers to the process of using a trained machine learning model to make predictions or decisions based on new data. After a model has been trained on historical data, inference is the phase where that model is deployed and used in real-world applications, such as recognizing speech, detecting objects in images, or recommending products. The focus in ML inference is on speed, efficiency, and scalability to ensure quick predictions, often in real time. This process is critical for practical applications like mobile apps, web services, and embedded systems. Optimizing inference involves reducing latency, memory usage, and computational requirements.

What is the difference between Ml Inference vs Data Scientist?

AspectML InferenceData Scientist
Required CredentialsKnowledge of machine learning models, programming skillsDegree in data science, statistics, or related fields
Work EnvironmentDeploying models in production, real-time data processingData analysis, model development, research
Industry UsageAI product deployment, software companiesResearch institutions, tech firms, consulting

ML Inference focuses on deploying trained models to make predictions on new data, often in real-time. Data Scientists develop and analyze models, working primarily in research and development. While both roles require understanding of machine learning, ML Inference emphasizes deployment and operationalization, whereas Data Scientists focus on model creation and analysis.

Which 3 jobs will survive AI?

For ML Inference roles, jobs that require complex problem-solving, creativity, and emotional intelligence are more likely to persist, such as data scientists, AI ethics specialists, and machine learning engineers. These roles involve tasks that are difficult to automate and often require specialized skills, domain knowledge, and critical thinking. Continuous learning and expertise in AI tools and programming languages like Python or TensorFlow can also enhance job security in this field.

What engineers make $500,000?

Senior machine learning engineers with extensive experience, specialized skills in deep learning, and strong industry demand can earn $500,000 or more annually, especially in high-cost-of-living areas or within top tech companies. Achieving this level typically requires advanced degrees, certifications, and a proven track record of impactful projects.

What is a $900,000 AI job?

A $900,000 AI job typically refers to high-level roles in artificial intelligence, such as senior machine learning engineers or AI research directors, often requiring advanced skills in deep learning, data science, and experience with tools like TensorFlow or PyTorch. These positions usually involve leadership responsibilities, strategic planning, and may require multiple years of specialized experience or advanced degrees.

Is ML a high paying job?

Machine Learning (ML) inference roles are generally well-paid due to the specialized skills required, such as knowledge of algorithms, programming, and data analysis. Salaries vary based on experience, location, and industry, but they tend to be higher than average for tech positions. Advanced roles often require proficiency with tools like TensorFlow or PyTorch and may include certifications or advanced degrees.

What are some common challenges faced by ML Inference Engineers when deploying models to production?

ML Inference Engineers often encounter challenges such as optimizing model latency and throughput to meet production requirements, ensuring compatibility with diverse hardware environments, and managing model versioning and updates without disrupting service. Additionally, balancing resource utilization and inference accuracy while monitoring real-time performance metrics is crucial. Collaboration with data scientists, DevOps, and software engineers is typically essential to streamline deployment and maintain robust, scalable inference pipelines.

What are the key skills and qualifications needed to thrive in ML Inference, and why are they important?

To thrive in ML Inference, you need a solid background in machine learning principles, programming (Python or C++), and experience with deploying models at scale, often supported by a degree in computer science or a related field. Familiarity with frameworks and tools such as TensorFlow, PyTorch, ONNX, and cloud platforms like AWS SageMaker or Google AI Platform is typically required. Strong problem-solving skills, attention to detail, and effective communication are crucial soft skills for collaborating with multidisciplinary teams and optimizing model performance. These skills ensure efficient, scalable, and reliable deployment of machine learning solutions in real-world applications.
What are popular job titles related to Ml Inference jobs in Raleigh, NC? For Ml Inference jobs in Raleigh, NC, the most frequently searched job titles are:
What job categories do people searching Ml Inference jobs in Raleigh, NC look for? The top searched job categories for Ml Inference jobs in Raleigh, NC are:
What cities near Raleigh, NC are hiring for Ml Inference jobs? Cities near Raleigh, NC with the most Ml Inference job openings:
Ml Inference jobs near you
Infographic showing various Ml Inference job openings in Raleigh, NC as of June 2026, with employment types broken down into 60% Full Time, 20% Part Time, and 20% Contract. Highlights an 60% In-person, and 40% Remote job distribution, with an average salary of $119,312 per year, or $57.4 per hour.
Staff Software Architect

Staff Software Architect

CVS Health

Raleigh, NC • On-site

Other

Medical, Dental, Vision, Retirement, PTO

Posted 2 days ago

CVS Health rating

5.8

Company rating: 5.8 out of 10

Based on 4,252 frontline employees who took The Breakroom Quiz

77th of 99 rated pharmacies

Job description

We're building a world of health around every individual - shaping a more connected, convenient and compassionate health experience. At CVS Health®, you'll be surrounded by passionate colleagues who care deeply, innovate with purpose, hold ourselves accountable and prioritize safety and quality in everything we do. Join us and be part of something bigger - helping to simplify health care one person, one family and one community at a time.

Position Summary

We are looking for a Staff Software Architect to serve as a technical leader within the Clinical Insights Engine team at CVS Health. This is a senior individual contributor role with enterprise-wide influence. You will define the long-term software and systems architecture vision for the Clinical Insights Engine platform, drive cross-functional alignment on engineering strategy, and lead the design of mission-critical systems that power clinical AI models, ML inference pipelines, enterprise reporting, and large-scale healthcare analytics. You will operate as a thought leader and trusted authority - setting direction for architecture decisions across multiple engineering squads, ML teams, and product verticals.

Responsibilities

  • Drive the end-to-end software architecture vision and roadmap for the Clinical Insights Engine platform across GCP and Azure environments

  • Serve as technical authority for system design and architecture decisions, providing guidance and governance across multiple engineering squads and product teams

  • Lead the design of highly scalable, fault-tolerant, and cost-optimized platforms supporting clinical AI/ML workloads - including LLM inference pipelines, document processing at scale, and real-time clinical data services

  • Define and govern ML architecture patterns - including feature stores, model registries, model serving infrastructure, experiment tracking, and MLOps pipelines - ensuring reproducibility, observability, and production-grade reliability

  • Lead architecture for LLM-integrated systems including prompt engineering frameworks, retrieval-augmented generation (RAG) pipelines, embedding stores, and clinical NLP processing

  • Define CIE-wide engineering standards, reference architectures, and platform patterns that promote consistency, reuse, and velocity across squads

  • Define enterprise software standards, architecture patterns, and reference architectures adopted across HCD and the broader CVS Health engineering community

  • Partner with CIE portfolio leads, product managers, and clinical stakeholders to align technical architecture with clinical program outcomes and business strategy

  • Represent CIE in enterprise architecture forums, HCD data governance councils, and cross-functional technical leadership groups within CVS Health

  • Champion engineering best practices across CIE squads: event-driven design, API-first development, data quality at source, and self-service platform patterns

  • Identify and proactively address architectural risk, technical debt, and data quality issues across CIE's clinical data platform

  • Mentor senior engineers and architects across CIE squads; build a culture of technical rigor, curiosity, and innovation

  • Ensure all CIE systems meet HIPAA compliance, enterprise security standards, and clinical data governance requirements

Required Qualifications

  • 8+ years of experience in software or data architecture, platform engineering, or a closely related field

  • 5+ years of experience designing enterprise-scale cloud systems in GCP and/or Azure in a production healthcare or regulated industry environment

  • Deep expertise in cloud-native services relevant to CIE's stack: BigQuery, Dataflow, Vertex AI, Cloud Run, GKE, Pub/Sub, Cloud Storage, or Azure equivalents

  • Demonstrated experience architecting clinical AI/ML platforms - including feature stores, model serving infrastructure, MLOps tooling, and LLM integration patterns applied to healthcare data

  • Strong command of distributed systems design: event streaming, asynchronous processing, API gateway patterns, service mesh, and fault-tolerant pipeline architecture

  • Experience defining and executing multi-year platform roadmaps across large, multi-squad engineering programs

  • Proven ability to influence and align senior technical and business stakeholders across portfolio and product boundaries without direct authority

  • Deep familiarity with clinical and healthcare data domains: clinical notes, EHR data, claims, pharmacy, member, and provider data

  • Strong understanding of HIPAA, PHI handling requirements, clinical data privacy regulations, and enterprise security frameworks

  • Experience working in SAFe Agile, program increment planning, and large-scale Agile delivery

Preferred Qualifications

  • Google Cloud Professional Architect, GCP Data Engineer, or Azure Solutions Architect certification

  • Hands-on experience with LLM-based clinical document processing, RAG architectures, OCR pipelines, or clinical NLP - directly applicable to CIE's patient fact extraction and potential conditions workflows

  • Experience designing and operating production ML platforms at scale using Vertex AI Pipelines, Kubeflow, MLflow, or equivalent

  • Familiarity with data mesh principles, domain-oriented data ownership, and federated governance - directly relevant to CIE's multi-portfolio operating model

  • Background in real-time ML serving, A/B testing infrastructure, and model monitoring and drift detection in clinical AI contexts

  • Prior experience in health plan, healthcare delivery, or clinical operations environments; familiarity with clinical workflows and value-based care programs is a strong plus

  • Published thought leadership, conference presentations, or patents in AI/ML systems, clinical NLP, or healthcare data architecture

Education

  • Bachelor's degree in Computer Science, Software Engineering, Information Systems, or a related field required

  • Master's degree in Computer Science, Machine Learning, or equivalent advanced technical experience strongly preferred

Pay Range

The typical pay range for this role is:

$106,605.00 - $260,590.00

This pay range represents the base hourly rate or base annual full-time salary for all positions in the job grade within which this position falls. The actual base salary offer will depend on a variety of factors including experience, education, geography and other relevant factors. This position is eligible for a CVS Health bonus, commission or short-term incentive program in addition to the base pay range listed above. This position also includes an award target in the company's equity award program.

Our people fuel our future. Our teams reflect the customers, patients, members and communities we serve and we are committed to fostering a workplace where every colleague feels valued and that they belong.

Great benefits for great people

We take pride in offering a comprehensive and competitive mix of pay and benefits that reflects our commitment to our colleagues and their families.

This full-time position is eligible for a comprehensive benefits package designed to support the physical, emotional, and financial well-being of colleagues and their families. The benefits for this position include medical, dental, and vision coverage, paid time off, retirement savings options, wellness programs, and other resources, based on eligibility.

Additional details about available benefits are provided during the application process and on Benefits Moments (https://learn.bswift.com/cvshealth-mainland) .

We anticipate the application window for this opening will close on: 06/26/2026

Qualified applicants with arrest or conviction records will be considered for employment in accordance with all federal, state and local laws.

CVS Health is an equal opportunity/affirmative action employer, including Disability/Protected Veteran - committed to diversity in the workplace.

What CVS Health employees say

Pay

Benefits

Hours and flexibility

Workplace

Get the full story on Breakroom

Apply