The project aims to develop an annotation framework for AI risk assessment and establish metrics ... and financial assistance to undergraduates, post-bachelor's degree holders, graduate students ...
The project aims to develop an annotation framework for AI risk assessment and establish metrics ... and financial assistance to undergraduates, post-bachelor's degree holders, graduate students ...
AI/ML Test Engineer
Springfield, VA · On-site
$100K - $150K/yr
Develop, maintain, and refine data annotation workflows supporting operational evaluation ... MTSI also offers a full range of medical, financial, and other benefits, dependent on the position ...
AI/ML Test Engineer
Springfield, VA · On-site
$100K - $150K/yr
Develop, maintain, and refine data annotation workflows supporting operational evaluation ... MTSI also offers a full range of medical, financial, and other benefits, dependent on the position ...
Biologist - Bioinformatics
Arlington, VA · On-site
$100K - $135K/yr
Apply deep expertise in bioinformatics, including genome assembly, annotation, transcriptomics, or ... Build financial stability with an industry-leading 401(k) retirement savings plan. For most ...
Biologist - Bioinformatics
Arlington, VA · On-site
$100K - $135K/yr
Apply deep expertise in bioinformatics, including genome assembly, annotation, transcriptomics, or ... Build financial stability with an industry-leading 401(k) retirement savings plan. For most ...
Enterprise Architect (Washington)
Washington, DC · On-site
$160K - $190K/yr
... annotation and tagging software, and multi-system search tools. Who We Are Comtech is an award ... In the finance and medical sector, Comtech offers IT solutions to help financial and healthcare ...
New
Enterprise Architect (Washington)
Washington, DC · On-site
$160K - $190K/yr
... annotation and tagging software, and multi-system search tools. Who We Are Comtech is an award ... In the finance and medical sector, Comtech offers IT solutions to help financial and healthcare ...
New
Experience in regulatory or financial services environment * Experience building human-in-the-loop review systems, annotation platforms, or approval workflows for AI outputs * Familiarity with event ...
Experience in regulatory or financial services environment * Experience building human-in-the-loop review systems, annotation platforms, or approval workflows for AI outputs * Familiarity with event ...
... and data annotation activities supporting advanced analytics use cases. Guide development teams ... financial datasets. Background working in fast-paced consulting, advisory, or large-scale ...
... and data annotation activities supporting advanced analytics use cases. Guide development teams ... financial datasets. Background working in fast-paced consulting, advisory, or large-scale ...
Coordinate data collection and annotation efforts for supervised training efforts * Design and ... Trusted by leading enterprises and government agencies, we partner with defense, finance, telecom ...
Coordinate data collection and annotation efforts for supervised training efforts * Design and ... Trusted by leading enterprises and government agencies, we partner with defense, finance, telecom ...
Lead Specialist, Federal Artificial Intelligence (AI) Engineering
Ashburn, VA · On-site
$104K - $138K/yr
... data / annotation processes to enrich unstructured data leveraging a variety of data sources (social media, news, internal / external documents, images, video, voice, emails, financial data, and ...
Lead Specialist, Federal Artificial Intelligence (AI) Engineering
Ashburn, VA · On-site
$104K - $138K/yr
... data / annotation processes to enrich unstructured data leveraging a variety of data sources (social media, news, internal / external documents, images, video, voice, emails, financial data, and ...
Senior Data Scientist
Reston, VA · On-site
Coordinate data collection and annotation efforts for supervised training efforts * Design and ... Trusted by leading enterprises and government agencies, we partner with defense, finance, telecom ...
Senior Data Scientist
Reston, VA · On-site
Coordinate data collection and annotation efforts for supervised training efforts * Design and ... Trusted by leading enterprises and government agencies, we partner with defense, finance, telecom ...
Imagery/Order of Battle Analyst with Security Clearance
Falls Church, VA · On-site
$79K - $134K/yr
Experience with Machine Learning training data creation, annotation, and review as well as ... and financial well-being. Regular employees scheduled to work 20+ hours per week are offered ...
Imagery/Order of Battle Analyst with Security Clearance
Falls Church, VA · On-site
$79K - $134K/yr
Experience with Machine Learning training data creation, annotation, and review as well as ... and financial well-being. Regular employees scheduled to work 20+ hours per week are offered ...
Sr Imagery Analyst- Falls Church, VA with Security Clearance
Falls Church, VA · On-site
$95K - $161K/yr
Experience with Machine Learning training data creation, annotation, and review as well as ... and financial well-being. Regular employees scheduled to work 20+ hours per week are offered ...
Sr Imagery Analyst- Falls Church, VA with Security Clearance
Falls Church, VA · On-site
$95K - $161K/yr
Experience with Machine Learning training data creation, annotation, and review as well as ... and financial well-being. Regular employees scheduled to work 20+ hours per week are offered ...
Sr Imagery Analyst - Falls Church, VA with Security Clearance
Falls Church, VA · On-site
$95K - $161K/yr
Experience with Machine Learning training data creation, annotation, and review as well as ... and financial well-being. Regular employees scheduled to work 20+ hours per week are offered ...
Sr Imagery Analyst - Falls Church, VA with Security Clearance
Falls Church, VA · On-site
$95K - $161K/yr
Experience with Machine Learning training data creation, annotation, and review as well as ... and financial well-being. Regular employees scheduled to work 20+ hours per week are offered ...
Senior Imagery / Order of Battle Analyst - AI/ML Mission Support with Security Clearance
Falls Church, VA · On-site
$95K - $161K/yr
Experience with AI/ML training data creation, annotation, validation, review, or Structured ... and financial well-being. Regular employees scheduled to work 20+ hours per week are offered ...
Senior Imagery / Order of Battle Analyst - AI/ML Mission Support with Security Clearance
Falls Church, VA · On-site
$95K - $161K/yr
Experience with AI/ML training data creation, annotation, validation, review, or Structured ... and financial well-being. Regular employees scheduled to work 20+ hours per week are offered ...
Senior Imagery / Order of Battle Analyst - AI/ML Mission Support with Security Clearance
Falls Church, VA · On-site
$95K - $161K/yr
Experience with AI/ML training data creation, annotation, validation, review, or Structured ... and financial well-being. Regular employees scheduled to work 20+ hours per week are offered ...
Senior Imagery / Order of Battle Analyst - AI/ML Mission Support with Security Clearance
Falls Church, VA · On-site
$95K - $161K/yr
Experience with AI/ML training data creation, annotation, validation, review, or Structured ... and financial well-being. Regular employees scheduled to work 20+ hours per week are offered ...
Sr Imagery Analyst- Falls Church, VA with Security Clearance
Sterling, VA · On-site
$95K - $161K/yr
Experience with Machine Learning training data creation, annotation, and review as well as ... and financial well-being. Regular employees scheduled to work 20+ hours per week are offered ...
Sr Imagery Analyst- Falls Church, VA with Security Clearance
Sterling, VA · On-site
$95K - $161K/yr
Experience with Machine Learning training data creation, annotation, and review as well as ... and financial well-being. Regular employees scheduled to work 20+ hours per week are offered ...
Annotation Finance information
What are the key skills and qualifications needed to thrive as an annotation finance specialist, and why are they important?
What is an annotation finance job?
What are common challenges faced by professionals working in annotation finance, and how can they be addressed?
What is the difference between Annotation Finance vs Data Analyst?
| Aspect | Annotation Finance |
|---|---|
| Primary Role | Annotating financial data for machine learning models in finance |
| Required Skills | Financial knowledge, data annotation, attention to detail |
| Work Environment | Data labeling teams, finance tech companies |
| Certifications | Basic financial certifications may help, but not mandatory |
Annotation Finance focuses on labeling financial data for AI applications, requiring financial understanding and data annotation skills. Data Analysts analyze and interpret data to inform business decisions, often involving data cleaning and reporting. While both roles work with data, Annotation Finance is specialized in preparing data for machine learning, whereas Data Analysts focus on data analysis and insights.
Full-time
Re-posted 27 days ago
Johns Hopkins University rating
8.0
Based on 71 frontline employees who took The Breakroom Quiz
183rd of 614 rated colleges and universities
Job description
PREP Research Associate
This position is part of the National Institute of Standards and Technology (NIST) Professional Research Experience Program (PREP). NIST recognizes that its research staff may want to collaborate with researchers at academic institutions on specific projects of mutual interest and, therefore, requires those institutions to be recipients of a PREP award. The PREP program involves staff from a wide range of backgrounds conducting scientific research across various fields. Individuals in this position will perform technical work supporting the collaboration's scientific research.
Research Title:
Reliability of Human and LLM Annotations for AI Risk Assessment
The work will entail:
This project focuses on using Large Language Models (LLMs) to provide annotations of evaluation data (a.k.a., LLM as judge), and the design of an Inter-Annotator Agreement study to assess the reliability of both human and LLM annotations. The candidate will explore assessing the indicators of a given AI-related risk, determining how to identify them, and providing annotators with examples to annotate the presence of various risks. The project aims to develop an annotation framework for AI risk assessment and establish metrics for data quality in AI risk research, supporting broader work at NIST in assessing and measuring the validity and reliability of AI-related risks in data annotation.
U.S. Citizen Preferred
Key responsibilities will include but are not limited to:
- Gain familiarity with existing literature on data annotation and LLM as judge
- Understand NIST's role and ongoing efforts in assessing and measuring the validity and reliability of AI-related risks in data annotation
- Contribute to developing an annotation framework for AI risk assessment
- Collaborate effectively with cross-functional and interdisciplinary stakeholders to ensure successful project outcomes
Deliverables
- Contributions to a NIST report that supports ongoing NIST AI evaluation efforts focused on the design of an Inter-Annotator Agreement to assess the reliability of both human and LLM annotations.
Qualifications
- Background in Computer Science, Data Science, or related field.
- Education level: Bachelor's or Graduate Degree
- Strong interest in data annotation and AI risks
- Familiarity with scientific reading and technical writing
Application Instructions
Please upload the following with your application:
• CV/Resume
*Please limit C.V to 3 pages only and ONLY include a valid email address for your contact info. Your resume will not be considered if the following information is included on your CV/resume.
• Self portraits
• Phone number
• Home address/Country
• Citizenship status
• Languages spoken
• Sex/Gender
Privacy Act Statement
Authority: 15 U.S.C. § 278g-1(e)(1) and (e)(3) and 15 U.S.C. § 272(b) and (c)
Purpose: The National Institute for Standards and Technology (NIST) hosts the Professional Research Experience Program (PREP) which is designed to provide valuable laboratory experience and financial assistance to undergraduates, post-bachelor's degree holders, graduate students, master's degree holders, postdocs, and faculty.
PREP is a 5-year cooperative agreement between NIST laboratories and participating PREP Universities to establish a collaborative research relationship between NIST and U.S. institutions of higher education in the following disciplines including (but may not be limited to) biochemistry, biological sciences, chemistry, computer science, engineering, electronics, materials science, mathematics, nanoscale science, neutron science, physical science, physics, and statistics. This collection of information is needed to facilitate administrative functions of the PREP Program.
Routine Uses: NIST will use the information collected to perform the requisite reviews of the applications to determine eligibility, and to meet programmatic requirements. Disclosure of this information is also subject to all the published routine uses as identified in the Privacy Act System of Records Notices: NIST-1: NIST Associates.
Disclosure: Furnishing this information is voluntary. When you submit the form, you are indicating your voluntary consent for NIST to use of the information you submit for the purpose stated.
What Johns Hopkins University employees say
Pay
Benefits
Hours and flexibility
Workplace
Get the full story on Breakroom
About Johns Hopkins University
Sourced by ZipRecruiter
Gilman believed that teaching and research go hand in hand—that success in one depends on success in the other—and that a modern university must do both well. He also believed that sharing our knowledge and discoveries would help make the world a better place. In 145 years, we haven’t strayed from that vision. This is still a destination for excellent, ambitious scholars and a world leader in teaching and research. Distinguished professors mentor students in the arts and music, humanities, social and natural sciences, engineering, international studies, education, business, and the health professions. Those same faculty members, along with their colleagues at the university’s Applied Physics Laboratory, have made us the nation’s leader in federal research and development funding every year since 1979. That’s a fitting distinction for America’s first research university, a place that has revolutionized higher education in the U.S. and continues to bring knowledge and discoveries to the world.
Industry
Colleges, universities, and professional schools
Company size
10,000+ Employees
Headquarters location
Baltimore, MD, US
Year founded
1876