Annotation Math Jobs in Walnut Creek, CA (NOW HIRING)

About the role:

Cobalt is seeking current PhD students working in machine learning to produce the expert reasoning and evaluation data used to train and assess frontier AI models.

This opportunity is suited to students who are actively doing ML research: designing and running experiments, training and evaluating models, working through derivations, and debugging results that do not behave as expected. You may be at any stage of your program, from first year through writing up, and you do not need to have published yet.

You do not need prior experience in data annotation or model evaluation. What matters is that you can solve non-trivial ML problems unaided and explain your reasoning clearly in writing.

What you'll do:

Depending on the project, you may:

• Produce written reasoning traces on hard ML problems, capturing how you reach a solution rather than only the solution itself, and draft expert reference answers to technical questions
• Author novel problems in your subfield that have verifiable or defensible correct answers
• Evaluate model-generated technical content: compare and rank responses, articulate what makes the stronger one stronger, and identify the specific step at which a chain of reasoning breaks down
• Assess whether stated conclusions are supported by the underlying derivation, code, or experimental evidence
• Design rubrics and partial-credit criteria for scoring multistep technical tasks

Projects follow their own guidelines, formatting conventions, and quality standards, and you will work with feedback from reviewers and lab research teams.

Required qualifications:

• Current enrollment in a PhD program in machine learning, computer science, statistics, mathematics, physics, or a closely related quantitative discipline, with research that is substantially ML focused, at any stage
• Demonstrated depth in at least one area, for example optimization, reinforcement learning, language model training and post-training, learning theory, probabilistic methods, computer vision, natural language processing, or systems for ML
• Ability to solve advanced ML problems independently, to interpret papers, derivations, code and experimental results, and to explain each step of your reasoning clearly in writing
• Strong attention to detail, a commitment to factual accuracy, and the ability to work independently to agreed timelines
• Confirmation that outside contract work is permitted under your visa status, funding terms, and institutional policies. Applicants are responsible for verifying this, and we cannot advise on it.

Publications at venues such as NeurIPS, ICML, ICLR, ACL, or CVPR are useful but not required, as is teaching assistant, grading, or peer review experience.

Why join Cobalt AI:

• Advance frontier AI where it counts. Apply your expertise to data that frontier labs cannot obtain any other way, where your reasoning directly shapes how the next generation of models works through technical problems.
• Grow professionally. Expand your influence through evaluation projects, advisory roles, and research collaborations, while developing a working understanding of how frontier models are trained and assessed.
• Work with a top-tier network. Collaborate with researchers and engineers from leading institutions and labs on high-impact, flexible work.
• Set your own schedule. Flexible 10 to 40 hour weeks that fit around your existing work and your life.
• Competitive pay. Rates vary by project and are determined by a number of factors, including scope, skillset, and experience.