Preference Model

Job Summary:
Preference Model is focused on building automated ML research engineering. They are looking for Research Engineers or Research Scientists to advance post-training on large language models, blending research and engineering to implement novel approaches and shape research directions.
Responsibilities:
• Train and evaluate models on our proprietary RL environments to validate data quality, surface gaps in task coverage, and close the feedback loop between environment design and model capability.
• Architect and optimize our RL training infrastructure, from training abstractions to distributed experiment management, using frameworks like Verl, OpenRLHF, or similar. Help scale our systems to handle increasingly complex research workflows.
• Design, implement, and test training environments, evaluations, and methodologies for RL agents.
• Profile and optimize training runs end-to-end, from data loading through reward computation, to maximize experiment throughput and shorten the research iteration cycle.
Qualifications:
Required:
• Experience running end-to-end LLM post-training pipelines
• Proficiency in Python and PyTorch or JAX
• Experience with at least one modern RL training framework
• Experience building and operating ML infrastructure at scale
Preferred:
• Have experience evaluating model outputs and building reward or evaluation signals
• Stay current on post-training research and can translate papers into running code
• Have strong opinions (loosely held) about how to structure RL training code for reproducibility and fast iteration
• Can balance research exploration with engineering rigor
• Have strong systems design and communication skills
Company:
Preference Model develops reinforcement learning environments for training AI systems to perform machine learning research. Founded in 2025, the company is headquartered in San Francisco, USA, with a team of 11-50 employees. The company is currently Early Stage.