Video Prompt Engineer Jobs in Minnesota (NOW HIRING)

Job Summary:
GlobalLogic, a Hitachi Group Company, is a trusted digital engineering partner to the world's largest and most forward-thinking companies. They are seeking an AI Evaluation & Benchmarking Engineer to support the productionization of an AI evaluation platform, focusing on reinforcement learning and LLM-based agents in video game environments.
Responsibilities:
• Develop, adapt, and integrate reinforcement learning algorithms and baseline approaches into the shared evaluation platform.
• Integrate LLM-based agents and/or evaluators for solving, interacting with, and benchmarking game environments.
• Integrate external or off-the-shelf algorithms into the platform using defined execution and ingestion workflows.
• Design and run benchmark experiments across games, environments, configurations, agents, and algorithm versions.
• Define evaluation strategies for comparing RL, LLM-based, hybrid, and baseline approaches.
• Define, extract, and validate meaningful performance metrics from logs, outputs, run results, and environment interactions.
• Build comparison logic, scoring approaches, rankings, verdicts, and performance summaries.
• Develop analytics and visualizations to evaluate algorithm performance across runs and environments.
• Act as a primary power user of the platform, running experiments and identifying gaps in tooling, APIs, metrics, workflows, logs, and user experience.
• Provide structured feedback to Platform and Full Stack engineers to improve execution, logging, evaluation, and reporting capabilities.
• Validate existing game environments and support development or validation of new game environments.
• Evaluate environment operability using baseline/reference frontier LLM models, harnesses, and agents.
• Collaborate with client technical teams and engineering resources within 3M-owned repositories, workflows, infrastructure, and security processes.
• Ensure all algorithms, experiments, notebooks/scripts, configuration, documentation, and outputs comply with 3M-defined standards and policies.
Qualifications:
Required:
• Hands-on reinforcement learning experience.
• Experience using LLMs for agents, evaluation, reasoning, automation, or benchmark workflows.
• Strong Python experience for ML, data workflows, experimentation, and analysis.
• Experience designing and running experiments with statistical and analytical rigor.
• Strong understanding of evaluation metrics, scoring frameworks, performance comparison, and benchmark design.
• Experience analyzing structured logs, run outputs, model/agent performance, and experiment results.
• Ability to work across APIs, logs, CLI/tools, data structures, and platform workflows.
• Strong communication skills to translate experiment findings into platform improvement requirements.
• Ability to work inside client-owned repositories, infrastructure, workflows, and security controls.
Preferred:
• Experience with game environments, simulation environments, Gym-like interfaces, RL environments, or agentic AI test harnesses.
• Experience benchmarking LLM agents, RL policies, autonomous agents, or hybrid AI systems.
• Experience with experiment tracking, run comparison tools, metrics dashboards, or evaluation pipelines.
• Experience with prompt engineering, agent orchestration, tool use, and LLM evaluation frameworks.
• Experience with data visualization and performance analytics.
• Experience working with externally developed algorithms, reproducible experiments, and version-controlled evaluation workflows.
Company:
Hitachi Rail is a provider of rail solutions across rolling stock, signalling services and turnkey. Founded in , the company is headquartered in London, GBR, with a team of 10001+ employees. The company is currently Late Stage.