Evaluation Engineer Jobs Near Me

hackajob is collaborating with J.P. Morgan to connect them with exceptional professionals for this role.

JOB DESCRIPTION

Employee Compute is evolving from endpoint operations to an AI-enabled compute control plane that supports a global workforce at scale. As Executive Director for Agentic Validation, Trust & Evaluation, you will own and lead the function that verifies AI-enabled employee experiences are safe, explainable, policy-compliant, and operationally ready before and during production use. 

This role sits within Risk & Controls and is accountable for setting the strategy for, building, and running the validation, trust, and evaluation operating model for agentic capabilities used across Employee Compute journeys. You will partner across engineering, cybersecurity, architecture, and control teams to establish measurable confidence in AI-assisted workflows through strong evaluation frameworks, auditable evidence, and clear governance outcomes aligned to a highly regulated environment. 

This role holds executive-level individual accountability for the effectiveness of the agentic validation and control environment within Employee Compute, including timely identification, escalation, and remediation of control gaps. The Executive Director is the senior named control owner for go/no-go release decisions on AI-enabled employee experiences and is answerable to senior leadership, audit, and regulators for the integrity of the evidence supporting those decisions. 

Job Responsibilities 

• Set and own the enterprise strategy for Agentic Validation, Trust & Evaluation for Employee Compute, translating policy and risk expectations into practical validation and control mechanisms for AI-enabled employee workflows. 

• Design and operationalize end-to-end evaluation frameworks for agentic capabilities, including pre-release validation, post-release monitoring, drift detection, and periodic control attestations. 

• Build standardized trust criteria for AI-assisted outcomes such as identity integrity, authorization correctness, data-handling boundaries, provenance, explainability, and reproducibility of decisions and actions. 

• Establish and govern quality gates and go/no-go decision practices for AI-enabled releases, ensuring evidence-based readiness across security, controls, resiliency, and operational support. 

• Own the mandate that validation and control requirements are embedded in design and delivery lifecycles, with authority to block releases that fail to meet engineered-in control standards (controls engineered in, not retrofit after launch). 

• Create and own KPI/OKR and scorecard structures for trust and evaluation outcomes, including control health, issue closure velocity, incident learnings, and model/tool performance against policy expectations. 

• Own and personally attest to examiner-ready evidence packages and control-testing results presented at risk and control forums, including executive communication on material risks, decisions, and mitigations to senior technology and control leadership. 

• Be accountable for approving or rejecting AI supplier capabilities for broad enablement, based on documented validation against internal standards and operational realities; retain sign-off authority and evidence of that decision. 

• Ensure timely and transparent escalation of material control weaknesses, validation failures, and AI-risk incidents to senior leadership and relevant risk committees. 

• Own the risk issues and corrective actions logged for the function in the firm's risk system of record, ensuring accuracy, supporting evidence, and on-time remediation. 

• Build, lead, and develop a high-performing team of validation and controls practitioners with strong technical fluency and disciplined risk-management execution, including managing managers and senior individual contributors. 

• Be accountable for control-culture outcomes within the function - where trust, control integrity, and speed coexist through clear standards, automation, and continuous improvement - evidenced by control-testing results, issue-closure performance, and audit/examiner feedback. 

Required Qualifications, Capabilities, and Skills 

• Formal training or certification on software engineering concepts and 10+ years applied experience

• 12+ years of technology, risk, controls, or governance experience, with significant time in regulated enterprise environments and at least 5 years leading programs, teams, or cross-functional workstreams. 

• Proven experience designing or operating control frameworks for modern technology platforms, such as cloud, endpoint, identity, cybersecurity, or AI-enabled systems. 

• Demonstrated experience as a senior named control owner or risk owner accountable for audit/examiner outcomes, including managing findings to closure. 

• Experience making and defending go/no-go / risk-acceptance decisions at an executive level, including documenting rationale and standing behind decisions under audit and regulatory scrutiny. 

• Hands-on experience with validation and evaluation approaches, including test design, scenario-based assessments, evidence generation, and measurable acceptance criteria. 

• Strong understanding of AI/agentic risk dimensions such as identity attribution, authorization boundaries, data governance, model/tool behavior variability, and operational misuse pathways.