Prompt Engineering Jobs in Matawan, NJ (NOW HIRING)

Vice President AI/ML Software Engineer

At BNY, our culture allows us to run our company better and enables employees' growth and success. As a leading global financial services company at the heart of the global financial system, we influence nearly 20% of the world's investible assets. Every day, our teams harness cutting-edge AI and breakthrough technologies to collaborate with clients, driving transformative solutions that redefine industries and uplift communities worldwide.

Recognized as a top destination for innovators and champions of inclusion, BNY is where bold ideas meet advanced technology and exceptional talent. Together, we power the future of finance - and this is what #LifeAtBNY is all about. Join us and be part of something extraordinary.

We are seeking a Vice President AI/ML Software Engineer to design and implement agentic AI systems, RAG pipelines, and intelligent document processing services. This is a senior individual contributor role with high autonomy -- you will own significant components of our AI platform, from embedding pipelines and vector retrieval to multi-agent extraction workflows. You will work closely with the SVP lead to translate architectural vision into production code, while mentoring mid-level engineers and driving technical excellence across the team. This role is in New York, NY

What Sets This Role Apart - You build the agent framework, not just configure one -- custom orchestration engine, not a LangChain wrapper - Production AI with real consequences -- extraction accuracy directly impacts financial operations - Full RAG ownership -- from raw OCR bytes through embedding, retrieval, and generation - Evaluation-driven culture -- golden-truth datasets, automated regression, measurable quality gates - Greenfield AI + enterprise integration -- build new AI-native systems that plug into established platforms

In this role, you'll have the opportunity to impact on our organization in the following ways:

AI Systems Development 

Implement agentic pipelines: agent loops, tool registries, memory stores, reasoning traces, and self-correction mechanisms - Build and optimize RAG systems end-to-end: - Document ingestion and preprocessing (OCR output, PDFs, structured/unstructured text) - Chunking strategies (section-aware, semantic, sliding window, hierarchical) - Embedding generation and vector index management - Retrieval orchestration: hybrid search, metadata filtering, re-ranking - Context assembly and prompt construction for downstream LLM calls - Develop vectorization pipelines -- embedding model integration, batch processing, incremental index updates, and similarity search tuning - Implement multi-agent coordination patterns: shared blackboards, inter-agent messaging, task decomposition, and consensus mechanisms - Build prompt engineering infrastructure: template management, few-shot example selection, chain-of-thought scaffolding, and output parsing - Develop evaluation harnesses: automated accuracy measurement, retrieval quality metrics, regression detection, and A/B comparison tooling 

Platform & Backend Engineering 

Build FastAPI services exposing AI capabilities as production APIs (extraction, validation, classification) - Contribute to Java/Spring Boot platform services where AI integrates with business workflow - Design and maintain database schemas for AI metadata: audit trails, pipeline runs, memory entries, knowledge graphs - Implement content policy enforcement and data governance controls within AI pipelines 

Mentorship & Collaboration 

Mentor 2-3 mid-level engineers on AI engineering practices - Participate in architecture reviews and design sessions - Document patterns, decisions, and runbooks for AI system operation - Collaborate with product and business stakeholders to translate requirements into technical solutions 

To be successful in this role, we're seeking the following: 

Bachelor's degree in Computer Science, Engineering, or related field. - Advanced degree preferred. 6+ years of professional software engineering experience - 2+ years building production AI/ML systems (not just notebooks/prototypes. Strong problem-solving skills with the ability to manage complex data processes. - Excellent collaboration and communication skills to work effectively with cross-functional teams. ) 

Strong RAG expertise: Embedding models (OpenAI, sentence-transformers, Cohere, or similar) - Vector databases (FAISS, Pinecone, Weaviate, Chroma, pgvector, or similar) - Chunking and retrieval optimization - Context window management and prompt assembly 

Agentic AI experience: Agent orchestration (custom frameworks, LangGraph, or similar) - Tool-use patterns, function calling, structured output parsing - Memory and state management for multi-turn agent interactions - Python proficiency (3.11+): FastAPI, async patterns, Pydantic, Poetry, pytest - LLM integration: prompt engineering, token management, streaming, error handling, rate limiting - NLP & document processing: OCR post-processing, text segmentation, entity extraction - Testing rigor: unit tests, integration tests, golden-truth validation, retrieval metric evaluation - API design: RESTful services, OpenAPI specifications, versioning strategies 

Preferred Qualifications

 Experience with knowledge graph construction from unstructured text - Familiarity with code AI concepts: code generation, automated testing, AI-assisted refactoring - Java/Spring Boot experience for cross-stack contribution - Angular/TypeScript for full-stack context - Experience with model evaluation: F1 scores, precision/recall for extraction, MRR/NDCG for retrieval - Exposure to fine-tuning or prompt optimization techniques - Understanding of graph RAG or hybrid retrieval architectures - Capital markets or financial services domain exposure - Experience with enterprise deployment: Docker, CI/CD, artifact repositories 

Technology Stack 

AI/Agentic: Multi-agent pipelines, tool-use, autonomous extraction, reasoning loops 

RAG: Embedding models, vector stores, hybrid search, chunking, re-ranking 

LLM: Azure OpenAI, GPT-4o, structured outputs, function calling 

Python: Python 3.12/3.13, FastAPI, Poetry, Pydantic, Gunicorn/Uvicorn 

Java: Java 21, Spring Boot 3.x (contributory) 

NLP/OCR:  Azure Document Intelligence, NLTK, document graph parsing 

Database: Oracle, PostgreSQL, vector databases 

Infrastructure: Docker, GitLab CI/CD 

Testing:  Pytest, golden-truth validation, retrieval metrics, evaluation harnesses 

Our Benefits and Rewards:

BNY offers highly competitive compensation, benefits, and wellbeing programs rooted in a strong culture of excellence and our pay-for-performance philosophy. We provide access to flexible global resources and tools for your life's journey. Focus on your health, foster your personal resilience, and reach your financial goals as a valued member of our team, along with generous paid leaves, including paid volunteer time, that can support you and your family through moments that matter. 

BNY is an Equal Employment Opportunity/Affirmative Action Employer - Underrepresented racial and ethnic groups/Females/Individuals with Disabilities/Protected Veterans.

This position is at-will and the Company reserves the right to modify base salary (as well as any other discretionary payment or compensation) at any time, including for reasons related to individual performance, change in geographic location, Company or individual department/team performance, and market factors.

At BNY, our culture allows us to run our company better and enables employees' growth and success. As a leading global financial services company at the heart of the global financial system, we influence nearly 20% of the world's investible assets. Every day, our teams harness cutting-edge AI and breakthrough technologies to collaborate with clients, driving transformative solutions that redefine industries and uplift communities worldwide.

At BNY, our culture speaks for itself, check out the latest BNY news at BNY Newsroom & BNY LinkedIn

 Here's a few of our recent awards:

• America's Most Innovative Companies, Fortune, 2025
• World's Most Admired Companies, Fortune 2025
• "Most Just Companies", Just Capital and CNBC, 2025

Our Benefits and Rewards:

BNY offers highly competitive compensation, benefits, and wellbeing programs rooted in a strong culture of excellence and our pay-for-performance philosophy. We provide access to flexible global resources and tools for your life's journey. Focus on your health, foster your personal resilience, and reach your financial goals as a valued member of our team, along with generous paid leaves, including paid volunteer time, that can support you and your family through moments that matter.

BNY is an Equal Employment Opportunity/Affirmative Action Employer - Underrepresented racial and ethnic groups/Females/Individuals with Disabilities/Protected Veterans.

BNY assesses market data to ensure a competitive compensation package for our employees. The expected base salary for this position when employment commences can be found in the Job Info section at the bottom of the posting. 

Base salary offered may vary depending on multiple individualized factors, including market location, job-related knowledge, skills, and experience. Base salary is only part of the total rewards package, which may include eligibility for an annual discretionary incentive award. Subject to the terms and conditions of the applicable plans then in effect, eligible employees may enroll in a 401(k) plan as well as participate in Company-sponsored medical, dental, vision, and basic life insurance plans for the employee and the employee's eligible dependents. Eligible employees also may receive other benefits (including various paid time off benefits, such as vacation and sick time), dependent on the position offered. Details of participation in these benefit plans will be provided if an employee receives an offer of employment.

If hired, the employee will be in an "at will" position and the Company reserves the right to modify base salary (as well as any other discretionary payments or compensation programs) at any time, including for reasons related to individual performance, Company or individual department/team performance, and market factors.