Reliability of Human and LLM Annotations for AI Risk Assessment The work will entail: This project focuses on using Large Language Models (LLMs) to provide annotations of evaluation data (a.k.a., LLM ...
Reliability of Human and LLM Annotations for AI Risk Assessment The work will entail: This project focuses on using Large Language Models (LLMs) to provide annotations of evaluation data (a.k.a., LLM ...
Collaborate with DoD CIO on AI ethics and RMF AI risk alignment Minimum Qualifications: * 8 + years of experience in AI, ML, or data science solutions * Bachelor's in Data Science, CS, or Engineering
Collaborate with DoD CIO on AI ethics and RMF AI risk alignment Minimum Qualifications: * 8 + years of experience in AI, ML, or data science solutions * Bachelor's in Data Science, CS, or Engineering
Collaborate with DoD CIO on AI ethics and RMF AI risk alignment Minimum Qualifications: * 8 + years of experience in AI, ML, or data science solutions * Bachelor's in Data Science, CS, or Engineering
Collaborate with DoD CIO on AI ethics and RMF AI risk alignment Minimum Qualifications: * 8 + years of experience in AI, ML, or data science solutions * Bachelor's in Data Science, CS, or Engineering
Senior Associate, Cyber Risk Specialist Identity & Access Management | Retail Bank
Mclean, VA · On-site
Certified AI Governance Professional (AIGP) certification or other relevant AI risk management certifications * Experience in a Large Financial Services or Big4 consulting organization At this time ...
Senior Associate, Cyber Risk Specialist Identity & Access Management | Retail Bank
Mclean, VA · On-site
Certified AI Governance Professional (AIGP) certification or other relevant AI risk management certifications * Experience in a Large Financial Services or Big4 consulting organization At this time ...
Senior Associate, Cyber Risk Specialist Identity & Access Management | Retail Bank
Mclean, VA · On-site
Certified AI Governance Professional (AIGP) certification or other relevant AI risk management certifications * Experience in a Large Financial Services or Big4 consulting organization At this time ...
Senior Associate, Cyber Risk Specialist Identity & Access Management | Retail Bank
Mclean, VA · On-site
Certified AI Governance Professional (AIGP) certification or other relevant AI risk management certifications * Experience in a Large Financial Services or Big4 consulting organization At this time ...
IT Risk Technical Lead
Mclean, VA · On-site
Support enterprise Artificial Intelligence (AI) adoption through the development and enhancement of critical governance technology solutions * Execute risk mitigation processes to ensure compliance ...
IT Risk Technical Lead
Mclean, VA · On-site
Support enterprise Artificial Intelligence (AI) adoption through the development and enhancement of critical governance technology solutions * Execute risk mitigation processes to ensure compliance ...
Support enterprise Artificial Intelligence (AI) adoption through the development and enhancement of critical governance technology solutions * Execute risk mitigation processes to ensure compliance ...
Support enterprise Artificial Intelligence (AI) adoption through the development and enhancement of critical governance technology solutions * Execute risk mitigation processes to ensure compliance ...
Conducts AI risk assessments, threat modeling, model and agent security reviews, and red team exercises for generative AI, predictive AI, and autonomous agent workflows * Establishes governance ...
Conducts AI risk assessments, threat modeling, model and agent security reviews, and red team exercises for generative AI, predictive AI, and autonomous agent workflows * Establishes governance ...
Conducts AI risk assessments, threat modeling, model and agent security reviews, and red team exercises for generative AI, predictive AI, and autonomous agent workflows * Establishes governance ...
Conducts AI risk assessments, threat modeling, model and agent security reviews, and red team exercises for generative AI, predictive AI, and autonomous agent workflows * Establishes governance ...
AI Governance & Data Privacy General Counsel
Washington, DC · On-site
$242 - $302/hr
The role sits at the intersection of AI innovation, data privacy, and enterprise risk management, enabling responsible AI adoption and data privacy compliance across internal operations, products ...
AI Governance & Data Privacy General Counsel
Washington, DC · On-site
$242 - $302/hr
The role sits at the intersection of AI innovation, data privacy, and enterprise risk management, enabling responsible AI adoption and data privacy compliance across internal operations, products ...
Supervisory Data Scientist (AI/ML)
Fort Belvoir, VA · On-site
$125K/yr
Conducts formal AI risk assessments and provides mitigation strategies * Certifies a model's scientific validity and readiness for the next phase of the deployment lifecycle, providing detailed test ...
Supervisory Data Scientist (AI/ML)
Fort Belvoir, VA · On-site
$125K/yr
Conducts formal AI risk assessments and provides mitigation strategies * Certifies a model's scientific validity and readiness for the next phase of the deployment lifecycle, providing detailed test ...
Support the development and operationalization of model risk management (MRM) practices for AI and traditional models across the model lifecycle (development, testing/validation, deployment ...
Support the development and operationalization of model risk management (MRM) practices for AI and traditional models across the model lifecycle (development, testing/validation, deployment ...
Support AI governance documentation, risk assessments, and authorization artifacts for AI/ML workloads consistent with the NIST AI Risk Management Framework and applicable federal guidance.
Support AI governance documentation, risk assessments, and authorization artifacts for AI/ML workloads consistent with the NIST AI Risk Management Framework and applicable federal guidance.
Data Engineer (DoD Secret | Remote)
Washington, DC · Remote
$129K - $155K/yr
Optimize data performance across distributed environments while ensuring compliance with DoD Responsible AI and NIST AI Risk Management frameworks. Who You Are * 4+ years of experience in Data ...
Quick apply
Data Engineer (DoD Secret | Remote)
Washington, DC · Remote
$129K - $155K/yr
Optimize data performance across distributed environments while ensuring compliance with DoD Responsible AI and NIST AI Risk Management frameworks. Who You Are * 4+ years of experience in Data ...
Support AI governance documentation, risk assessments, and authorization artifacts for AI/ML workloads consistent with the NIST AI Risk Management Framework and applicable federal guidance.
Support AI governance documentation, risk assessments, and authorization artifacts for AI/ML workloads consistent with the NIST AI Risk Management Framework and applicable federal guidance.
Support AI governance documentation, risk assessments, and authorization artifacts for AI/ML workloads consistent with the NIST AI Risk Management Framework and applicable federal guidance.
Support AI governance documentation, risk assessments, and authorization artifacts for AI/ML workloads consistent with the NIST AI Risk Management Framework and applicable federal guidance.
Support AI governance documentation, risk assessments, and authorization artifacts for AI/ML workloads consistent with the NIST AI Risk Management Framework and applicable federal guidance.
Quick apply
Support AI governance documentation, risk assessments, and authorization artifacts for AI/ML workloads consistent with the NIST AI Risk Management Framework and applicable federal guidance.
Solution Architect (AI)
$63.75 - $84/hr
Support the development of AI governance structures for federal customers, including model risk management, explainability standards, bias mitigation protocols, and continuous monitoring practices ...
Solution Architect (AI)
$63.75 - $84/hr
Support the development of AI governance structures for federal customers, including model risk management, explainability standards, bias mitigation protocols, and continuous monitoring practices ...
NIST AI Risk Management Framework • Experience applying: AI & Emerging Security • Experience applying: AI and model fundamentals • Experience applying: AI security risks • Experience with ...
NIST AI Risk Management Framework • Experience applying: AI & Emerging Security • Experience applying: AI and model fundamentals • Experience applying: AI security risks • Experience with ...
The engineer will support RMF, ATO, continuous monitoring, cloud compliance, control validation, AI security testing, and risk-based authorization decisions. Responsibilities include architecture ...
The engineer will support RMF, ATO, continuous monitoring, cloud compliance, control validation, AI security testing, and risk-based authorization decisions. Responsibilities include architecture ...
Ai Risk information
See Washington, DC salary details
$21.85 is the 25th percentile. Wages below this are outliers.
$16.34 - $22.47
28% of jobs
The median wage is $26.14 / hr.
$22.47 - $28.61
37% of jobs
$28.61 - $34.75
6% of jobs
$38.59 is the 75th percentile. Wages above this are outliers.
$34.75 - $40.89
6% of jobs
$40.89 - $47.03
12% of jobs
$47.03 - $53.16
0% of jobs
$53.16 - $59.30
0% of jobs
$59.30 - $65.44
8% of jobs
$65.44 - $71.58
0% of jobs
$71.58 - $77.72
0% of jobs
$77.72 - $83.86
2% of jobs
$16
$34
$83
How much do ai risk jobs pay per hour?
What is the difference between Ai Risk vs Data Scientist?
| Aspect | Ai Risk | Data Scientist |
|---|---|---|
| Required Credentials | Background in AI, risk management, certifications in AI safety | Degree in Computer Science, Statistics, or related fields; certifications in data analysis |
| Work Environment | Risk assessment teams, AI development projects, regulatory settings | Data analysis teams, research labs, tech companies |
| Employer & Industry Usage | Tech firms, AI safety organizations, regulatory agencies | Tech companies, finance, healthcare, research institutions |
| Common Search & Comparison Intent | Understanding AI risk roles, career differences | Data analysis careers, AI safety roles |
Ai Risk professionals focus on identifying and mitigating risks associated with artificial intelligence systems, often working in safety, ethics, and regulatory contexts. Data Scientists analyze large datasets to extract insights, build models, and support decision-making across various industries. While both roles require technical skills, Ai Risk emphasizes safety and ethical considerations, whereas Data Scientists focus on data analysis and modeling.

Full-time
Re-posted 24 days ago
Johns Hopkins University rating
8.0
Based on 71 frontline employees who took The Breakroom Quiz
184th of 614 rated colleges and universities
Job description
PREP Research Associate
This position is part of the National Institute of Standards and Technology (NIST) Professional Research Experience Program (PREP). NIST recognizes that its research staff may want to collaborate with researchers at academic institutions on specific projects of mutual interest and, therefore, requires those institutions to be recipients of a PREP award. The PREP program involves staff from a wide range of backgrounds conducting scientific research across various fields. Individuals in this position will perform technical work supporting the collaboration's scientific research.
Research Title:
Reliability of Human and LLM Annotations for AI Risk Assessment
The work will entail:
This project focuses on using Large Language Models (LLMs) to provide annotations of evaluation data (a.k.a., LLM as judge), and the design of an Inter-Annotator Agreement study to assess the reliability of both human and LLM annotations. The candidate will explore assessing the indicators of a given AI-related risk, determining how to identify them, and providing annotators with examples to annotate the presence of various risks. The project aims to develop an annotation framework for AI risk assessment and establish metrics for data quality in AI risk research, supporting broader work at NIST in assessing and measuring the validity and reliability of AI-related risks in data annotation.
U.S. Citizen Preferred
Key responsibilities will include but are not limited to:
- Gain familiarity with existing literature on data annotation and LLM as judge
- Understand NIST's role and ongoing efforts in assessing and measuring the validity and reliability of AI-related risks in data annotation
- Contribute to developing an annotation framework for AI risk assessment
- Collaborate effectively with cross-functional and interdisciplinary stakeholders to ensure successful project outcomes
Deliverables
- Contributions to a NIST report that supports ongoing NIST AI evaluation efforts focused on the design of an Inter-Annotator Agreement to assess the reliability of both human and LLM annotations.
Qualifications
- Background in Computer Science, Data Science, or related field.
- Education level: Bachelor's or Graduate Degree
- Strong interest in data annotation and AI risks
- Familiarity with scientific reading and technical writing
Application Instructions
Please upload the following with your application:
• CV/Resume
*Please limit C.V to 3 pages only and ONLY include a valid email address for your contact info. Your resume will not be considered if the following information is included on your CV/resume.
• Self portraits
• Phone number
• Home address/Country
• Citizenship status
• Languages spoken
• Sex/Gender
Privacy Act Statement
Authority: 15 U.S.C. § 278g-1(e)(1) and (e)(3) and 15 U.S.C. § 272(b) and (c)
Purpose: The National Institute for Standards and Technology (NIST) hosts the Professional Research Experience Program (PREP) which is designed to provide valuable laboratory experience and financial assistance to undergraduates, post-bachelor's degree holders, graduate students, master's degree holders, postdocs, and faculty.
PREP is a 5-year cooperative agreement between NIST laboratories and participating PREP Universities to establish a collaborative research relationship between NIST and U.S. institutions of higher education in the following disciplines including (but may not be limited to) biochemistry, biological sciences, chemistry, computer science, engineering, electronics, materials science, mathematics, nanoscale science, neutron science, physical science, physics, and statistics. This collection of information is needed to facilitate administrative functions of the PREP Program.
Routine Uses: NIST will use the information collected to perform the requisite reviews of the applications to determine eligibility, and to meet programmatic requirements. Disclosure of this information is also subject to all the published routine uses as identified in the Privacy Act System of Records Notices: NIST-1: NIST Associates.
Disclosure: Furnishing this information is voluntary. When you submit the form, you are indicating your voluntary consent for NIST to use of the information you submit for the purpose stated.
What Johns Hopkins University employees say
Pay
Benefits
Hours and flexibility
Workplace
Get the full story on Breakroom
About Johns Hopkins University
Sourced by ZipRecruiter
Gilman believed that teaching and research go hand in hand—that success in one depends on success in the other—and that a modern university must do both well. He also believed that sharing our knowledge and discoveries would help make the world a better place. In 145 years, we haven’t strayed from that vision. This is still a destination for excellent, ambitious scholars and a world leader in teaching and research. Distinguished professors mentor students in the arts and music, humanities, social and natural sciences, engineering, international studies, education, business, and the health professions. Those same faculty members, along with their colleagues at the university’s Applied Physics Laboratory, have made us the nation’s leader in federal research and development funding every year since 1979. That’s a fitting distinction for America’s first research university, a place that has revolutionized higher education in the U.S. and continues to bring knowledge and discoveries to the world.
Industry
Colleges, universities, and professional schools
Company size
10,000+ Employees
Headquarters location
Baltimore, MD, US
Year founded
1876