1

Ml Inference Jobs in Bridgewater, NJ (NOW HIRING)

Solutions Architect

New York, NY · On-site +1

$165K - $330K/yr

Experience running or supporting benchmarks for ML inference deployments. * Familiarity with infrastructure tradeoffs relevant to inference performance and cost (for example GPU selection and latency ...

Design, build, and optimize large-scale ML training and inference systems, including distributed computing frameworks and hardware-accelerated pipelines * Develop and maintain high-performance ML ...

Senior Inference Engineer

New York, NY · Remote

$190K - $220K/yr

Hands-on generative AI experience with common ML frameworks (PyTorch, Transformers) * Good ... and inference clusters, so they can turn raw GPUs into cluster products they can sell in days ...

Significant experience building production ML systems, inference runtimes, or performance-sensitive infrastructure * Strong programming ability in Python and C++ * Deep understanding of transformer ...

Design, implement, and maintain robust microservices to host high-throughput ML inference endpoints * Architect and manage the ML infrastructure necessary to support sophisticated LLM-based features ...

Software Engineer II

New York, NY · On-site

$106K - $146K/yr

You will participate in the ML lifecycle: feature engineering on time-series data, model training on GPU clusters, real-time inference pipelines, and model improvement. You will partner with ...

Silicon Software Lead

New York, NY · On-site

$200K - $400K/yr

Deep understanding of ML inference workloads and the constraints that shape their execution on hardware * Experience with compiler frameworks such as MLIR or LLVM, or with inference runtimes and ...

Showing results 21-40

Ml Inference information

See Bridgewater, NJ salary details

$38.3K

$125.5K

$200.9K

How much do ml inference jobs pay per year?

As of Sep 12, 2026, the average yearly pay for ml inference in Bridgewater, NJ is $125,467.00, according to ZipRecruiter salary data. Most workers in this role earn between $100,700.00 and $139,000.00 per year, depending on experience, location, and employer.

What is ML inference?

ML inference refers to the process of using a trained machine learning model to make predictions or decisions based on new data. After a model has been trained on historical data, inference is the phase where that model is deployed and used in real-world applications, such as recognizing speech, detecting objects in images, or recommending products. The focus in ML inference is on speed, efficiency, and scalability to ensure quick predictions, often in real time. This process is critical for practical applications like mobile apps, web services, and embedded systems. Optimizing inference involves reducing latency, memory usage, and computational requirements.

What are the key skills and qualifications needed to thrive in ML inference?

To thrive in ML Inference, you need a solid background in machine learning principles, programming (Python or C++), and experience with deploying models at scale, often supported by a degree in computer science or a related field. Familiarity with frameworks and tools such as TensorFlow, PyTorch, ONNX, and cloud platforms like AWS SageMaker or Google AI Platform is typically required. Strong problem-solving skills, attention to detail, and effective communication are crucial soft skills for collaborating with multidisciplinary teams and optimizing model performance. These skills ensure efficient, scalable, and reliable deployment of machine learning solutions in real-world applications.

What are some common challenges faced by ML inference engineers when deploying models to production?

ML Inference Engineers often encounter challenges such as optimizing model latency and throughput to meet production requirements, ensuring compatibility with diverse hardware environments, and managing model versioning and updates without disrupting service. Additionally, balancing resource utilization and inference accuracy while monitoring real-time performance metrics is crucial. Collaboration with data scientists, DevOps, and software engineers is typically essential to streamline deployment and maintain robust, scalable inference pipelines.

What is the difference between Ml Inference vs Data Scientist?

AspectML InferenceData Scientist
Required CredentialsKnowledge of machine learning models, programming skillsDegree in data science, statistics, or related fields
Work EnvironmentDeploying models in production, real-time data processingData analysis, model development, research
Industry UsageAI product deployment, software companiesResearch institutions, tech firms, consulting

ML Inference focuses on deploying trained models to make predictions on new data, often in real-time. Data Scientists develop and analyze models, working primarily in research and development. While both roles require understanding of machine learning, ML Inference emphasizes deployment and operationalization, whereas Data Scientists focus on model creation and analysis.

What job categories do people searching Ml Inference jobs in Bridgewater, NJ look for?

The top searched job categories for Ml Inference jobs in Bridgewater, NJ are:

What cities near Bridgewater, NJ are hiring for Ml Inference jobs?

Cities near Bridgewater, NJ with the most Ml Inference job openings:

Software Development Engineer, ML Systems, Annapurna Labs

New York, NY • On-site

Amazon
IT Services • 10K+ employees

Full-time

Re-posted 29 days ago


Amazon rating

7.4

Company rating: 7.4 out of 10

Based on 7,173 frontline employees who took The Breakroom Quiz


Job description

About Amazon Annapurna Labs:
Amazon Annapurna Labs team (our organization within AWS UC) is responsible for building innovation in silicon and software for our AWS customers. We are at the forefront of innovation by combining cloud scale with the world's most talented engineers. Our team covers multiple disciplines including silicon engineering, hardware design, software and operations.

Because of our teams breadth of talent, we have been able to improve AWS cloud infrastructure in high-performance machine learning with AWS Neuron, Inferentia and Trainium ML chips, in networking and security with products such as AWS Nitro, Enhanced Network Adapter (ENA), and Elastic Fabric Adapter (EFA), and in computing with AWS Graviton and F1 EC2 instances.
About AWS Utility Computing (UC):
AWS Utility Computing (UC) provides product innovations that continue to set AWS's services and features apart in the industry. As a member of the UC organization, you'll support the development and management of Compute, Database, Storage, Platform, and Productivity Apps services in AWS, including support for customers who require specialized security solutions for their cloud services. Additionally, this role may involve exposure to and experience with Amazon's growing suite of generative AI services and other cloud computing offerings across the AWS portfolio.
About AWS
Amazon Web Services (AWS) is the world's most comprehensive and broadly adopted cloud platform

We pioneered cloud computing and never stopped innovating - that's why customers from the most successful startups to Global 500 companies trust our robust suite of products and services to power their businesses.
About AWS Neuron:
AWS Neuron is the software of Trainium and Inferentia, the AWS Machine Learning chips. Inferentia delivers best-in-class ML inference performance at the lowest cost in the cloud to our AWS customers. Trainium is designed to deliver the best-in-class ML training performance at the lowest training cost in the cloud, and it's all being enabled by AWS Neuron

Neuron is a Software that include ML compiler and native integration into popular ML frameworks. Our products are being used at scale with external customers like Anthropic and Databricks as well as internal customers like Alexa, Amazon Bedrocks, Amazon Robotics, Amazon Ads, Amazon Rekognition and many more.
You will join a dynamic team building and applying AI agents to simplify and accelerate customer adoption of Trainium and Inferentia. As an SDE II you will work with external and internal customers to identify the main obstacles and the opportunities to accelerate their adoption of the Neuron technology


Key job responsibilities
Our engineers collaborate across diverse teams, projects, and environments to have a firsthand impact on our global customer base. You'll bring a passion for innovation, data, search, analytics, and distributed systems. You'll also:
- Solve challenging technical problems, often ones not solved before, at every layer of the stack.


- Design, implement, test, deploy and maintain innovative software solutions to transform service performance, durability, cost, and security.
- Build high-quality, highly available, always-on products.
- Research implementations that deliver the best possible experiences for customers.
A day in the life
As you design and code solutions to help our team drive efficiencies in software architecture, you'll create metrics, implement automation and other improvements, and resolve the root cause of software defects. You'll also:
- Build high-impact solutions to deliver to our large customer base.
- Participate in design discussions, code review, and communicate with internal and external stakeholders.
- Work cross-functionally to help drive business decisions with your technical input. You will collaborate closely with a cross-functional team comprised of compiler, hardware, and ML engineers.
- Work in a startup-like development environment, where you're always working on the most important stuff.


What Amazon employees say

Pay

Benefits

Hours and flexibility

Workplace

Get the full story on Breakroom


Amazon logo

About Amazon

Sourced by ZipRecruiter

Amazon.com, Inc., commonly known as Amazon, is an American multinational technology company. It was founded by Jeff Bezos in 1994 and initially started as an online marketplace for books. Since then, Amazon has expanded its operations and become one of the largest e-commerce companies in the world. Amazon's primary business is its online retail platform, where customers can purchase a vast array of products, including electronics, clothing, books, home goods, and much more. The company offers a convenient and user-friendly shopping experience, with features such as fast shipping, customer reviews, and personalized recommendations. In addition to its e-commerce platform, Amazon has diversified its business into various other areas. One of its notable ventures is Amazon Web Services (AWS), a comprehensive cloud computing platform that provides services such as storage, compute power, and database management to individuals and businesses. AWS has become a leader in the cloud computing industry, powering many websites and applications worldwide. Amazon has also developed its own consumer electronics, including the popular Amazon Kindle e-reader, Fire tablets, Fire TV streaming devices, and the Alexa-powered Echo smart speakers. The Alexa voice assistant, integrated into these devices, allows users to interact with their devices using voice commands, perform tasks, and access information. Furthermore, Amazon has expanded into media and entertainment. It operates Prime Video, a streaming service that offers a wide range of movies, TV shows, and original content. Amazon Music provides a platform for streaming and purchasing digital music, while Audible offers audiobooks and other audio content. The company's commitment to customer satisfaction and convenience is demonstrated by its membership program, Amazon Prime. Prime members receive various benefits, including free two-day shipping, access to streaming services, exclusive deals, and more.

Industry

It services, book publishers, retail, real estate, computer and electronic product manufacturing and software development

Company size

10,000+ Employees

Headquarters location

Seattle, WA, US