Benchmark Specialist Jobs (NOW HIRING)

Applied Health & Medicine Benchmark Specialist

Position: Applied Health & Medicine Benchmark Specialist

Type: Hourly Contract

Compensation: $94–$119/hour

Location: Remote

About the Opportunity

This opportunity is for experienced medical and health science professionals to contribute their expertise to an advanced AI research and evaluation project with Mercor.

You will author and verify high-quality academic assessment content across clinical medicine, surgery, medical imaging and diagnostics, pharmacovigilance, healthcare management and economics, rehabilitation, and allied health. The work focuses on developing rigorous benchmarks that evaluate AI systems on deep conceptual understanding, clinical reasoning, accuracy, and professional judgment.

Experts with advanced knowledge in relevant health and medicine domains are encouraged to apply.

Responsibilities

• Develop original, challenging health and medicine multiple-choice questions that assess deep conceptual understanding rather than surface-level recall.
• Create realistic assessment scenarios across clinical medicine, surgery, diagnostics, pharmacovigilance, healthcare management, rehabilitation, and allied health.
• Provide 1 correct answer and 9 plausible but subtly incorrect alternatives for each question.
• Ensure questions are self-contained, unambiguous, precise, and appropriately challenging.
• Rate each question according to difficulty: Medium, Hard, or Expert.
• Develop clear, step-by-step solutions explaining the correct reasoning and answer.
• Provide 1–5 reputable academic references for each question, including peer-reviewed research and clinical guidelines.
• Review and verify pre-written questions for medical accuracy, clarity, completeness, precision, and solvability.
• Identify and correct inaccurate, ambiguous, or unsupported content.
• Document and justify any modifications made during the verification process.
• Evaluate AI-generated health and medical responses and identify technical errors, unsupported conclusions, and gaps in reasoning.
• Ensure benchmark content reflects graduate-level professional and academic medical knowledge rather than basic textbook or examination-level recall.

Required Qualifications

• MD, DO, PhD, or doctoral candidate in Medicine, Biomedical Sciences, Public Health, or a closely related field.
• Strong command of graduate-level medical knowledge and clinical reasoning.
• Strong understanding of biomedical research methodology and academic assessment design.
• Expertise in one or more relevant health and medicine domains.
• Ability to evaluate complex medical information for accuracy, rigor, and scientific validity.
• Excellent written English and ability to communicate complex medical concepts clearly and concisely.
• Strong analytical and critical-thinking skills.
• Ability to independently research and reference peer-reviewed literature, clinical guidelines, and other authoritative sources.

Preferred Qualifications

• Experience in clinical medicine, medical research, healthcare, or academic medicine.
• Board certification or equivalent professional credentials.
• Research publications in medicine, biomedical sciences, public health, or related fields.
• Advanced specialization in clinical medicine, surgery, medical imaging, diagnostics, pharmacovigilance, healthcare management, rehabilitation, or allied health.
• Experience creating academic assessments, examination questions, training materials, or evaluation rubrics.
• Experience with AI evaluation, AI training, benchmark development, or quality assurance is a plus.
• Experience reviewing or publishing peer-reviewed academic or scientific content.

Compensation

• $94–$119/hour
• Hourly contract engagement.
• Fully remote and asynchronous opportunity.
• Opportunity to apply advanced medical and health science expertise to AI research, evaluation, and benchmark development.

Eligibility

• Medical and health science professionals with advanced academic or professional expertise are encouraged to apply.
• Candidates with MD, DO, PhD, or doctoral-level training in relevant disciplines are particularly suited to the role.
• Expertise in one or more covered health and medicine domains is applicable.
• Candidates with clinical experience, research publications, or specialized professional credentials are strongly encouraged to apply.

Application Process

1. Apply for the Applied Health & Medicine Benchmark Specialist position.
2. Complete the required screening process.
3. Demonstrate relevant medical, health science, research, or clinical expertise.
4. Complete any required health and medicine evaluation or assessment.
5. Complete onboarding if selected.
6. Begin authoring, reviewing, and evaluating AI-generated health and medicine assessment content.