Retrieval Augmented Generation Rag Jobs in New York

Our client is one of the leading private credit investment firms in the United States, managing more than $9 billion in assets and providing bespoke financing solutions to middle-market companies navigating growth, acquisitions, operational transformation, and complex financial situations.

The firm has built a production AI platform that is transforming how investment professionals research, analyze, and evaluate investment opportunities. The platform powers AI-generated investment memoranda, due diligence questionnaire (DDQ) automation, and intelligent retrieval across highly complex financial and legal documents.

Our client is seeking an exceptional Software Engineer to help expand this platform and build the next generation of AI-enabled investment workflows.

• This is not an AI research role.
• This is not a machine learning model training role.
• This is not a prompt engineering or proof-of-concept position.

Instead, this is an opportunity to build production AI systems that become part of the daily workflow of investment professionals making real investment decisions.

As one of a small number of engineers supporting this platform, you'll have significant ownership, influence technical direction, and see your work move quickly from concept to production.

Working alongside the platform's lead engineer, you'll partner directly with investment professionals, investor relations, and operations teams to design, build, deploy, and continuously improve AI-powered applications that deliver measurable business impact.

If you're passionate about solving challenging engineering problems, shipping production software, and leveraging cutting-edge AI technologies to solve real-world business problems, this is an exceptional opportunity.

What You'll Build

You'll help design, build, and evolve production AI systems, including:

• AI-powered investment memorandum generation
• Due diligence questionnaire (DDQ) automation
• Retrieval-Augmented Generation (RAG) across complex legal and financial documents
• AI agents, reusable skills, and workflow automation
• Source attribution and provenance tracking
• AI review workflows that improve the accuracy, transparency, and trustworthiness of AI-generated content

Everything you build will be deployed into production and used by investment professionals, not developed as a proof of concept or demo.

Representative Projects

Examples of the engineering challenges you'll tackle include:

• Trace every statement in an AI-generated investment memorandum back to the exact page and source document used to support it.
• Improve retrieval across hundreds of pages of credit agreements, financial statements, and legal documentation while preserving complex tables and formatting.
• Design review workflows that identify and prevent hallucinated financial information before it reaches investment professionals.
• Build reusable AI agents and workflows that automate repetitive research and due diligence processes.
• Develop internal AI tools that reduce hours of manual investment analysis into minutes while maintaining transparency, traceability, and accuracy.
• Partner directly with business users to transform manual workflows into scalable AI-powered solutions.
• Continuously improve platform performance, reliability, and user experience as new AI capabilities emerge.

Key Responsibilities

• Design, develop, deploy, and support production AI applications that support the investment lifecycle.
• Build and enhance AI agents, reusable skills, subagents, and workflow automation.
• Develop provenance tracking, citation, and review capabilities that improve confidence in AI-generated output.
• Build scalable retrieval pipelines utilizing vector search and modern RAG architectures.
• Collaborate closely with investment, investor relations, and operations professionals to understand business workflows and deliver practical AI solutions.
• Own projects from architecture and design through implementation, deployment, testing, monitoring, and ongoing production support.
• Participate in architectural discussions and help shape the future direction of the firm's AI platform.

Qualifications

The ideal candidate will possess:

• 3–5 years of professional software engineering experience.
• Strong programming experience in one or more of the following:
• Python
• TypeScript / JavaScript
• Go
• Rust
• Experience building production AI applications using technologies such as:
• AI Agents
• Retrieval-Augmented Generation (RAG)
• Vector Search
• Prompt & Context Engineering
• Evaluation Frameworks (Evals)
• Daily experience using AI-assisted development tools such as:
• Claude Code
• Cursor
• Codex
• GitHub Copilot
• or similar AI-native development environments
• Demonstrated success independently designing, building, deploying, and supporting production software.
• A builder's mindset with a passion for solving difficult technical problems and delivering high-quality software.
• Strong verbal and written communication skills with the ability to work directly with senior business stakeholders.
• Excellent analytical, organizational, and problem-solving skills.

Preferred Experience

Experience in one or more of the following areas is highly desirable:

• Document-intensive or highly regulated industries
• Financial Services
• Private Credit
• Banking or Capital Markets
• Asset Management
• Investment Banking
• Vector databases (Qdrant, Pinecone, Weaviate, or similar)
• PDF and document parsing
• Anthropic or OpenAI APIs
• Agent SDKs
• Model Context Protocol (MCP)
• Programmatic document generation (DOCX/PPTX)
• Open-source software development

Engineering Culture

The engineering organization is intentionally small, collaborative, and highly empowered.

Engineers are trusted to:

• Own problems from design through production.

• Work directly with investment professionals and business stakeholders.

• Influence technical architecture and platform direction.

• Continuously improve products and engineering practices.

• Move quickly while maintaining high engineering standards.

The culture values:

• Shipping reliable production software

• Strong software engineering fundamentals

• Thoughtful code reviews

• Practical AI solutions over hype

• Ownership, accountability, and initiative

• Curiosity, experimentation, and continuous learning

Why This Opportunity Stands Out

This role offers the opportunity to:

• Help shape how AI is transforming investment decision-making at one of the industry's leading private credit investment firms.
• Build AI systems that directly support real investment decisions.
• Solve technically challenging problems involving large-scale financial and legal documents.
• Influence the architecture and evolution of a rapidly expanding AI platform.
• Collaborate directly with experienced investment professionals to solve meaningful business problems.
• Join a small engineering team where individual contributions have immediate visibility and impact.
• Work with cutting-edge AI technologies while building production systems—not prototypes.
• Develop solutions that fundamentally change how investment professionals research, analyze, and evaluate investment opportunities.

Compensation

Base Salary: $150,000 – $200,000

In addition to a highly competitive base salary, our client offers a comprehensive total compensation package that includes:

• Performance-based discretionary bonus
• Comprehensive medical, dental, and vision benefits
• Retirement savings plan
• Paid time off
• Additional employee wellness and benefit programs

Compensation will be commensurate with experience, technical expertise, and overall qualifications.

Location: New York City

Applicants must be legally authorized to work in the United States on a full-time basis. Sponsorship or transfer of employment visas are not available for this position.

Interested?

If you're excited or know someone who would be an outstanding fit - building production AI systems that solve complex, real-world problems- and want to work at the intersection of software engineering, artificial intelligence, and private credit- we'd welcome the opportunity to have a confidential conversation. Please contact Jerry Goldberg | jgoldberg@jharlan.com | (212) 792-0099