Generative Ai Testing Jobs in Boston, MA (NOW HIRING)

Description

Join a team where innovation meets impact. As a Senior Data Scientist, Generative AI & Agentic Systems, you will help drive the bank's AI transformation by designing, developing, and deploying Large Language Model (LLM) solutions, Retrieval-Augmented Generation (RAG) systems, AI agents, and intelligent automation capabilities. You will work across business, technology, risk, and compliance teams to deliver responsible, scalable, and production-ready GenAI solutions that improve customer experiences, enhance operational efficiency, and create measurable business value.

This role is ideal for an experienced data scientist with strong software engineering and machine learning skills, deep expertise in NLP and Generative AI, and experience developing AI solutions within highly regulated environments.

Key Responsibilities

• Design, develop, and deploy production-grade Generative AI solutions using LLMs, RAG frameworks, AI agents, and workflow orchestration platforms.
• Build intelligent document processing capabilities for information extraction, summarization, classification, question answering, and conversational AI applications.
• Develop agentic workflows capable of autonomous reasoning, task execution, tool utilization, and multi-step decision support.
• Design and implement retrieval pipelines, vector search architectures, embedding strategies, and knowledge-grounded AI systems.
• Evaluate and improve LLM performance through prompt engineering, model benchmarking, hallucination reduction, and faithfulness testing.
• Build scalable AI solutions using modern frameworks and infrastructure including vLLM, LangChain, LangGraph, MLflow, Databricks, Snowflake, and cloud-native platforms.
• Perform exploratory data analysis, feature engineering, and statistical analysis to support machine learning and GenAI model development.
• Develop model monitoring, evaluation, and observability frameworks to measure quality, reliability, fairness, and operational performance.
• Collaborate closely with Model Risk Management (MRM), Compliance, Audit, Legal, and Information Security teams to ensure responsible AI deployment.
• Create technical documentation, model development artifacts, validation packages, and executive-level presentations.
• Partner with product managers, engineers, data architects, and business stakeholders to identify and prioritize GenAI opportunities.
• Stay current with advances in Generative AI, agentic systems, multimodal AI, foundation models, and emerging industry best practices.

Qualifications

Required

• Ph.D. or Master's degree in Computer Science, Data Science, Statistics, Mathematics, Artificial Intelligence, or a related quantitative field.
• 7+ years of experience in data science, machine learning, predictive analytics, or artificial intelligence.
• 4+ years of hands-on experience developing NLP and Generative AI solutions.
• Strong proficiency in Python and modern software development practices.
• Experience developing and deploying LLM-based applications using commercial or open-source models.
• Experience with Retrieval-Augmented Generation (RAG), vector databases, embeddings, and semantic search.
• Experience with prompt engineering, prompt evaluation, and LLM performance optimization.
• Strong understanding of machine learning algorithms, deep learning, statistical modeling, and model explainability techniques.
• Experience working with structured and unstructured data at enterprise scale.
• Experience collaborating with cross-functional stakeholders and communicating technical concepts to non-technical audiences.
• Strong knowledge of model governance, validation processes, and documentation standards.

Preferred

• Experience designing and deploying AI agents and multi-agent systems.
• Experience with agent orchestration frameworks such as LangChain, LangGraph, Semantic Kernel, CrewAI, Autogen, or similar technologies.
• Experience serving open-source LLMs using vLLM, Hugging Face, or equivalent inference frameworks.
• Experience with RAG evaluation frameworks such as RAGAS or other LLM evaluation methodologies.
• Experience with model monitoring, MLOps, and production AI deployment.
• Experience with cloud AI platforms such as AWS Bedrock, Azure AI, Databricks, Snowflake Cortex.
• Experience building document intelligence solutions involving PDFs, OCR,  document extraction, knowledge extraction from images, and workflow automation.
• Experience within banking, financial services, fintech, insurance, or other regulated industries.
• Experience supporting Model Risk Management (MRM), model validation, audit reviews, or regulatory examinations.
• Familiarity with MCP (Model Context Protocol), tool calling frameworks, and AI workflow automation platforms.

Technical Skills

Generative AI & LLMs

• GPT, Claude, Llama and other foundation models
• Retrieval-Augmented Generation (RAG)
• AI Agents and Multi-Agent Systems
• Prompt Engineering and Prompt Optimization
• Fine-Tuning and Model Adaptation
• LLM Evaluation and Guardrails
• Knowledge Retrieval and Vector Search

Programming & Frameworks

• Python
• SQL
• PyTorch
• TensorFlow
• Scikit-Learn
• LangChain
• LangGraph
• Hugging Face

Data Platforms & MLOps

• Experience with cloud-based data, AI, and ML platforms (AWS, SageMaker, Databricks, Snowflake, etc.)
• Experience with distributed data processing frameworks (Spark / PySpark/Snowpark Snowflake)
• Experience with ML lifecycle, orchestration, and deployment tools (MLflow, Airflow, CI/CD)
• Experience with AI-assisted development and model monitoring solutions

NLP & Analytics

• Text Classification
• Information Extraction
• Summarization
• Topic Modeling
• Question Answering
• Sentiment Analysis
• Explainable AI

Preferred Candidate Profile

The ideal candidate needs to demonstrate success building production-scale GenAI solutions such as RAG platforms, conversational AI systems, document intelligence solutions, AI agents, and automated decision-support systems. They possess strong technical depth, understand governance requirements in regulated industries, and can bridge the gap between cutting-edge AI capabilities and practical business outcomes. This individual is comfortable operating from concept through production deployment while maintaining a strong focus on quality, compliance, explainability, and measurable impact.

Hours & Work Schedule

• Hours per Week: 40
• Work Schedule: Monday - Friday
• Hybrid: 4 days per week on-site, 1 day remote

Some job boards have started using jobseeker-reported data to estimate salary ranges for roles. If you apply and qualify for this role, a recruiter will discuss accurate pay guidance.

Equal Employment Opportunity

Citizens, its parent, subsidiaries, and related companies (Citizens) provide equal employment and advancement opportunities to all colleagues and applicants for employment without regard to age, ancestry, color, citizenship, physical or mental disability, perceived disability or history or record of a disability, ethnicity, gender, gender identity or expression, genetic information, genetic characteristic, marital or domestic partner status, victim of domestic violence, family status/parenthood, medical condition, military or veteran status, national origin, pregnancy/childbirth/lactation, colleague's or a dependent's reproductive health decision making, race, religion, sex, sexual orientation, or any other category protected by federal, state and/or local laws. At Citizens, we are committed to fostering an inclusive culture that enables all colleagues to bring their best selves to work every day and everyone is expected to be treated with respect and professionalism. Employment decisions are based solely on merit, qualifications, performance and capability.