1

Prompt Engineering Jobs in New York (NOW HIRING)

GEN AI Architect@ NYC, NY & Charlotte, NC

Manhattan, NY · On-site

$69.50 - $91.50/hr

Define solution patterns for prompt engineering, RAG (Retrieval-Augmented Generation), and fine-tuning * Collaborate with business, data, and engineering teams to identify GenAI use cases * Select ...

Requirements * 2+ years in voice AI, conversational AI, prompt engineering, speech systems, or related agentic workflows * 3+ years in platform, infrastructure, or backend engineering * Strong ...

Lead the design, training, fine-tuning, and deployment of large language models, leveraging techniques like prompt engineering, retrieval-augmented generation (RAG), and agent-based architectures.

Sr. AI/ML Engineer, Platform

Manhattan, NY · On-site

$115K - $158K/yr

Build and optimize integrations with large language models (LLMs), utilizing best practices for prompt engineering, tool use, streaming, and token management. • Distributed Systems Architecture:

Python + Gen AI Developer - New York

Manhattan, NY · On-site

$55 - $76/hr

Prompt Engineering, Workflow Design, and GenAI Optimization. Key Responsibilities: Develop and orchestrate sophisticated AI workflows using LangGraph and multi-agent architectures. Build and maintain ...

Python + Gen AI Developer - New York

Manhattan, NY · On-site

$55 - $76/hr

Prompt Engineering, Workflow Design, and GenAI Optimization. Key Responsibilities: Develop and orchestrate sophisticated AI workflows using LangGraph and multi-agent architectures. Build and maintain ...

Understanding of machine learning concepts and prompt engineering. * Experience with version control (Git). * Strong problem-solving and analytical skills. * Excellent communication and collaboration ...

New

Showing results 41-60

Prompt Engineering information

See New York salary details

$35.6K

$68.9K

$104.5K

How much do prompt engineering jobs pay per year?

As of Aug 8, 2026, the average yearly pay for prompt engineering in New York is $68,899.00, according to ZipRecruiter salary data. Most workers in this role earn between $51,400.00 and $78,800.00 per year, depending on experience, location, and employer.

What is prompt engineering?

A Prompt Engineering job involves designing, refining, and optimizing prompts to improve the performance of AI language models. Prompt engineers work with large language models (LLMs) to generate accurate, relevant, and high-quality responses. They experiment with different phrasing techniques, fine-tune AI outputs, and collaborate with developers to enhance model capabilities. This role is essential in ensuring AI systems provide reliable and useful responses for various applications.

What skills and qualifications are needed for prompt engineering?

To excel in Prompt Engineering, a strong grasp of natural language processing (NLP), machine learning concepts, and analytical thinking is essential, often supported by a degree in computer science or a related field. Familiarity with AI platforms, code repositories (such as GitHub), and prompt development tools is typically required. Excellent problem-solving, creativity, and cross-functional communication skills help Prompt Engineers effectively collaborate and refine model outputs. These capabilities enable the creation of precise, effective prompts driving high-quality AI responses in rapidly evolving technical environments.

What do you do as a prompt engineer?

A prompt engineer designs and refines prompts to improve the performance of AI language models. They analyze model responses, experiment with prompt structures, and use tools like AI development platforms to ensure accurate and relevant outputs, often requiring skills in programming and understanding of AI behavior.

What are the most common challenges faced by prompt engineers in their daily work?

Prompt Engineers frequently encounter challenges such as ensuring the clarity and relevance of prompts to achieve accurate AI responses, troubleshooting inconsistent model behavior, and staying updated with evolving AI technologies. Balancing experimentation with efficiency is often essential, as iterative testing and refinement are core parts of the workflow. Collaboration with data scientists, product managers, and other engineers is common, requiring adaptability and strong communication skills. These challenges make the role dynamic and rewarding for professionals who enjoy problem-solving and innovation.

What are the most commonly searched types of Prompt Engineering jobs in New York? The most popular types of Prompt Engineering jobs in New York are:
What cities in New York are hiring for Prompt Engineering jobs? Cities in New York with the most Prompt Engineering job openings:
Infographic showing various Prompt Engineering job openings in New York as of August 2026, with employment types broken down into 88% Full Time, 7% Part Time, 4% Contract, and 1% Nights. Highlights an 88% Physical, 3% Hybrid, and 9% Remote job distribution, with an average salary of $68,899 per year, or $33.1 per hour.

Machine Learning Operations Engineer - Remote

NAVA Software Solutions

Jersey City, NJ • On-site, Remote

$76K - $102K/yr

Full-time

Re-posted 6 days ago


Job description

NAVA Software solutions is looking for a Machine Learning Operations Engineer
Details:
Machine Learning Operations (MLOps) Engineer - AWS (with LLM Focus)
Location: Remote work
Duration: 12 months

Responsibilities:
  • LLM-Optimized MLOps Infrastructure: Design and implement MLOps infrastructure on AWS tailored for LLMs, leveraging services like SageMaker, EC2 (with GPU instances), S3, ECS/EKS, Lambda, and more.
  • LLM Deployment Pipelines: Build and manage CI/CD pipelines specifically for LLM deployment, addressing unique challenges like model size, inference optimization, and versioning.
  • LLMOps Practices: Implement LLMOps best practices for monitoring model performance, drift detection, prompt management, and feedback loops for continuous improvement.
  • RESTful API Development: Design and develop RESTful APIs to expose LLM capabilities to other applications and services, ensuring scalability, security, and optimal performance.
  • Model Optimization: Apply techniques like quantization, distillation, and pruning to optimize LLM models for efficient inference on AWS infrastructure.
  • Monitoring and Observability: Establish comprehensive monitoring and alerting mechanisms to track LLM performance, latency, resource utilization, and potential biases.
  • Prompt Engineering and Management: Develop strategies for prompt engineering and management to enhance LLM outputs and ensure consistency and safety.
  • Collaboration: Work closely with data scientists, researchers, and software engineers to integrate LLM models into production systems effectively.
  • Cost Optimization: Continuously optimize LLMOps processes and infrastructure for cost-efficiency while maintaining high performance and reliability.

Qualifications:
  • Experience: 3+ years of experience in MLOps or a related field, with hands-on experience in deploying and managing LLMs.
  • AWS Expertise: Strong proficiency in AWS services relevant to MLOps and LLMs, including SageMaker, EC2 (with GPU instances), S3, ECS/EKS, Lambda, and API Gateway.
  • LLM Knowledge: Deep understanding of LLM architectures (e.g., Transformers), training techniques, and inference optimization strategies.
  • Programming Skills: Proficiency in Python and experience with infrastructure-as-code tools (e.g., Terraform, CloudFormation), REST API frameworks (e.g., Flask, FastAPI), and LLM libraries (e.g., Hugging Face Transformers).
  • Monitoring: Familiarity with monitoring and logging tools for LLMs, such as Prometheus, Grafana, and CloudWatch.
  • Containerization: Experience with Docker and container orchestration (e.g., Kubernetes, ECS) for LLM deployment.
  • Problem Solving: Excellent problem-solving and troubleshooting skills in the context of LLMs and MLOps.
  • Communication: Strong communication and collaboration skills to effectively work with cross-functional teams

NAVA Software Solutions logo

About NAVA Software Solutions

Sourced by ZipRecruiter

NAVA is a strategic partner for companies seeking to develop or customize software and products. Our team of experts leverages cutting-edge technology and deep industry knowledge to provide customized solutions that drive business success. Whether you're looking to improve your operations, increase efficiency, or bring a new product to market, NAVA has the expertise and resources to help you achieve your goals. Trust us to be your partner in software and product development.

Industry

It services

Company size

51 - 200 Employees

Headquarters location

Rocky Hill, CT, US

Social media