Associate Psychometrician Jobs in Virginia (NOW HIRING)

About the School
The University of Virginia School of Data Science, the first of its kind in the nation, advances discovery, innovation, and societal impact through collaborative, open, and responsible data science research and education. Founded in 2019, the School brings together expertise across business, computation, engineering, humanities, law, mathematics, social sciences, and statistics to address complex, real-world challenges. Its academic offerings include a B.S. in Data Science, an undergraduate minor, residential and online M.S. in Data Science programs, and a Ph.D. in Data Science, all designed to prepare students for a rapidly evolving data-driven world.
About the Position
This position leads the design, execution, and analysis of a large paired human-subjects study that links AI biosecurity benchmark scores to performance on realistic biological protocol-design and troubleshooting tasks. The researcher reports to Associate Professor Stephen D. Turner and coordinates closely with co-investigators Tom Hartvigsen and Chirag Agarwal, and a laboratory partner. The role draws on study design, psychometric and statistical analysis, and management of participant data collection. The researcher will also contribute to a parallel machine unlearning effort, supporting measurement of how capability-mitigation methods affect both benchmark scores and real-world task performance.
Key Responsibilities

• Lead day-to-day execution of the paired evaluation study, including benchmark administration, participant recruitment and scheduling, and coordination with the laboratory partner on protocol intake and outcome reporting

• Develop scoring rubrics and instruments for protocol-design, feasibility, and troubleshooting tasks

• Build and run mixed-effects and item-level analyses that estimate how benchmark scores predict real-world task performance, with quantified uncertainty

• Produce the benchmark validity report and the benchmark-to-task crosswalk and calibration framework

• Contribute to the machine unlearning and model editing components through safety-utility analyses, including within-subject comparisons of model variants on shared task batteries

• Prepare manuscripts, preprints, and documentation for open release, and support IRB compliance and data quality

Minimum Qualifications

• Doctoral degree (PhD or equivalent) in data science, statistics, biostatistics, psychometrics, quantitative psychology, a biological or biomedical science, or a related field, completed at the time of hire

• Experience designing and analyzing human subjects studies, including power analysis and study design

• Strong statistical programming in R or Python, including mixed-effects or hierarchical modeling

• Working knowledge of molecular biology or microbiology sufficient to evaluate biological protocols and collaborate with laboratory scientists

Preferred Qualifications

• Research experience in biosecurity, biodefense, or dual-use risk assessment

• Familiarity with AI or large language model evaluation, benchmarking, or psychometric validation of model capabilities

• Experience with machine learning tooling such as PyTorch or Hugging Face Transformers, and with model finetuning workflows

• Prior work under IRB oversight and with controlled or sensitive data

• Record of open-source software or reproducible-research practices

• Experience coordinating academic-industry or multi-site research collaborations

Anticipated Salary: $60,000 - $75,000 per year
Anticipated Start Date: September 1, 2026
Health and Other Benefits
(visit Health and Other Benefits for additional information)

• UVA Health Plan: the choice between 3 different health plans

• Vision Coverage

• Dental Plan

• Benefit Savings Plans

• Life Insurance

• Disability Benefits

• Paid Time Off: starting with 22 days of time off per year, 12 or more holidays, 8 weeks parental leave

Education Benefits (visit Education Benefits for additional information)
After six months of employment, full-time and part-time (20+ hours) employees in a benefits-eligible position are offered options of:

• Use of up to $5250 per calendar year towards a for-credit degree program or for-credit certificate program

• Use of up to $2000 of the total $5250 noted above per calendar year for professional development including job-related training, conferences, and initial certificate exams.

Position Details
This position will remain open until it is filled. This is a full-time in-person position at the School of Data Science at the University of Virginia in Charlottesville, VA. The initial appointment is for one year; however, the appointment may be renewed for an additional year contingent upon funding and satisfactory performance. This is an exempt level, term-limited (restricted), benefited position.
Anticipated Salary: $70,000
Anticipated Start Date: September 1, 2026
Application Process
Please apply online , and search for R0084977.
Complete an application online and attach:
Cover letter detailing your interest and relevant experience to this position Resume or CV Two letters of recommendation with contact information
Applications that do not contain all required documents will not receive full consideration.
Internal applicants: Search and apply for jobs on the UVA Internal Careers website .
References will be completed via direct reach. Please plan to provide at least three references when applying.
A background check is required and will be conducted per university policy prior to the first day of employment.
For questions about the position , contact Associate Professor Stephen D. Turner at sdt5z@virginia.edu .
For questions about the application process, please contact Daniel Strong, Senior Human Resources Recruiter, at das6zb@virginia.edu .
For more information about UVA and the Charlottesville community, please see www.virginia.edu/life/charlottesville and https://embarkcva.com/ .
The University of Virginia is an equal opportunity employer. All interested persons are encouraged to apply, including veterans and individuals with disabilities.
MINIMUM REQUIREMENTS
Education: Doctoral degree
Experience: None
Licensure: None
PHYSICAL DEMANDS
This is primarily a sedentary job involving extensive use of desktop computers. The job does occasionally require traveling some distance to attend meetings, and programs.
The University of Virginia is an equal opportunity employer. All interested persons are encouraged to apply, including veterans and individuals with disabilities. Learn more about UVA's commitment to non-discrimination and equal opportunity employment .