The project aims to develop an annotation framework for AI risk assessment and establish metrics for data quality in AI risk research, supporting broader work at NIST in assessing and measuring the ...
The project aims to develop an annotation framework for AI risk assessment and establish metrics for data quality in AI risk research, supporting broader work at NIST in assessing and measuring the ...
Data Scientist [$231k/yr+] TS/SCI-FS Poly with Security Clearance
Annapolis Junction, MD · On-site
$231K/yr
... NLP) projects, leveraging expertise in Human Language Processing (HLP). • Execute advanced language data tokenization and develop automated annotation solutions. • Enhance and refine existing ...
Data Scientist [$231k/yr+] TS/SCI-FS Poly with Security Clearance
Annapolis Junction, MD · On-site
$231K/yr
... NLP) projects, leveraging expertise in Human Language Processing (HLP). • Execute advanced language data tokenization and develop automated annotation solutions. • Enhance and refine existing ...
Data Scientist [$231k/yr+] TS/SCI-FS Poly with Security Clearance
Annapolis Junction, MD · On-site
$231K/yr
... NLP) projects, leveraging expertise in Human Language Processing (HLP). • Execute advanced language data tokenization and develop automated annotation solutions. • Enhance and refine existing ...
Data Scientist [$231k/yr+] TS/SCI-FS Poly with Security Clearance
Annapolis Junction, MD · On-site
$231K/yr
... NLP) projects, leveraging expertise in Human Language Processing (HLP). • Execute advanced language data tokenization and develop automated annotation solutions. • Enhance and refine existing ...
Data Project Manager
Annapolis, MD · On-site
$51.25 - $69.25/hr
PRIMARY RESPONSIBILITY The role of the Chesapeake Data Project Manager will be to execute and manage a 5-year geospatial land characterization project between Chesapeake Conservancy's Conservation ...
Data Project Manager
Annapolis, MD · On-site
$51.25 - $69.25/hr
PRIMARY RESPONSIBILITY The role of the Chesapeake Data Project Manager will be to execute and manage a 5-year geospatial land characterization project between Chesapeake Conservancy's Conservation ...
Project Manager - Data Modernization
Silver Spring, MD · On-site
$53.25 - $72.25/hr
Brillient Corporation is seeking an experienced Project Manager (SME/Data Modernization) to join its Team supporting a federal contract in Silver Spring, MD. This position requires U.S. Citizenship ...
Project Manager - Data Modernization
Silver Spring, MD · On-site
$53.25 - $72.25/hr
Brillient Corporation is seeking an experienced Project Manager (SME/Data Modernization) to join its Team supporting a federal contract in Silver Spring, MD. This position requires U.S. Citizenship ...
Senior Data Project Manager
North Bethesda, MD · Remote
$52.75 - $71.25/hr
We're hiring a Senior Data Project Manager to drive critical data initiatives supporting our Supply Management function. You'll own delivery of complex, cross-functional data programs end-to-end.
Senior Data Project Manager
North Bethesda, MD · Remote
$52.75 - $71.25/hr
We're hiring a Senior Data Project Manager to drive critical data initiatives supporting our Supply Management function. You'll own delivery of complex, cross-functional data programs end-to-end.
Senior Data Project Manager
North Bethesda, MD · Remote
$51.75 - $70/hr
We're hiring a Senior Data Project Manager to drive critical data initiatives supporting our Supply Management function. You'll own delivery of complex, cross-functional data programs end-to-end.
Senior Data Project Manager
North Bethesda, MD · Remote
$51.75 - $70/hr
We're hiring a Senior Data Project Manager to drive critical data initiatives supporting our Supply Management function. You'll own delivery of complex, cross-functional data programs end-to-end.
DATA CENTER PROJECT MANAGER
Elkridge, MD · On-site
$90K - $105K/yr
Job Title: Data Center Project Manager Location: Elkridge, Maryland Starting Compensation Range: $90,000 to $105,000 per year The Data Center Project Manager is responsible for providing the ...
DATA CENTER PROJECT MANAGER
Elkridge, MD · On-site
$90K - $105K/yr
Job Title: Data Center Project Manager Location: Elkridge, Maryland Starting Compensation Range: $90,000 to $105,000 per year The Data Center Project Manager is responsible for providing the ...
DATA CENTER PROJECT MANAGER
$90K - $105K/yr
Job Title: Data Center Project Manager Location: Elkridge, Maryland Starting Compensation Range: $90,000 to $105,000 per year The Data Center Project Manager is responsible for providing the ...
DATA CENTER PROJECT MANAGER
$90K - $105K/yr
Job Title: Data Center Project Manager Location: Elkridge, Maryland Starting Compensation Range: $90,000 to $105,000 per year The Data Center Project Manager is responsible for providing the ...
Project Manager, Data Centers
$51.50 - $69.50/hr
We provide value across the entire project lifecycle through our core construction management ... By integrating data, artificial intelligence, and advanced technology through our Seamless Platform ...
Project Manager, Data Centers
$51.50 - $69.50/hr
We provide value across the entire project lifecycle through our core construction management ... By integrating data, artificial intelligence, and advanced technology through our Seamless Platform ...
Project Manager, Data Centers
Frederick, MD · On-site
$51.50 - $69.50/hr
We provide value across the entire project lifecycle through our core construction management ... By integrating data, artificial intelligence, and advanced technology through our Seamless Platform ...
Project Manager, Data Centers
Frederick, MD · On-site
$51.50 - $69.50/hr
We provide value across the entire project lifecycle through our core construction management ... By integrating data, artificial intelligence, and advanced technology through our Seamless Platform ...
Project Manager, Data Centers
$51.50 - $69.50/hr
We provide value across the entire project lifecycle through our core construction management ... By integrating data, artificial intelligence, and advanced technology through our Seamless Platform ...
Project Manager, Data Centers
$51.50 - $69.50/hr
We provide value across the entire project lifecycle through our core construction management ... By integrating data, artificial intelligence, and advanced technology through our Seamless Platform ...
Project Manager - Data Modernization
Silver Spring, MD · On-site
$135K - $155K/yr
Brillient Corporation is seeking an experienced Project Manager (SME/Data Modernization) to join its Team supporting a federal contract in Silver Spring, MD. This position requires U.S. Citizenship ...
Project Manager - Data Modernization
Silver Spring, MD · On-site
$135K - $155K/yr
Brillient Corporation is seeking an experienced Project Manager (SME/Data Modernization) to join its Team supporting a federal contract in Silver Spring, MD. This position requires U.S. Citizenship ...
Project Manager
Bethesda, MD · On-site
We're data geeks to our core, committed to simplifying the toughest challenges and operating with ... Project Team Engagement and Client Management: Serve as the primary point of contact for the ...
Quick apply
Project Manager
Bethesda, MD · On-site
We're data geeks to our core, committed to simplifying the toughest challenges and operating with ... Project Team Engagement and Client Management: Serve as the primary point of contact for the ...
Project Manager
Potomac, MD · On-site
$70K - $120K/yr
NACI AMDEX.ai The Art of Data Science We are a seasoned Full-spectrum data solutions firm. We ... The Project Manager shall be responsible for leading and managing projects from initiation to ...
Quick apply
Project Manager
Potomac, MD · On-site
$70K - $120K/yr
NACI AMDEX.ai The Art of Data Science We are a seasoned Full-spectrum data solutions firm. We ... The Project Manager shall be responsible for leading and managing projects from initiation to ...
Company Description Vignesh Chandran KRG Technologies Inc. vignesh.c at krgtech.com 661-367-8000 EXT- 405 12+ years, Technical project manager Should have experience with Big data Managed development ...
Company Description Vignesh Chandran KRG Technologies Inc. vignesh.c at krgtech.com 661-367-8000 EXT- 405 12+ years, Technical project manager Should have experience with Big data Managed development ...
Project Manager, Data Centers, Security
$51.50 - $69.50/hr
We provide value across the entire project lifecycle through our core construction management ... By integrating data, artificial intelligence, and advanced technology through our Seamless Platform ...
Project Manager, Data Centers, Security
$51.50 - $69.50/hr
We provide value across the entire project lifecycle through our core construction management ... By integrating data, artificial intelligence, and advanced technology through our Seamless Platform ...
Project Manager, Data Centers, Security
Frederick, MD · On-site
$51.50 - $69.50/hr
We provide value across the entire project lifecycle through our core construction management ... By integrating data, artificial intelligence, and advanced technology through our Seamless Platform ...
Project Manager, Data Centers, Security
Frederick, MD · On-site
$51.50 - $69.50/hr
We provide value across the entire project lifecycle through our core construction management ... By integrating data, artificial intelligence, and advanced technology through our Seamless Platform ...
Project Manager, Data Centers, Security
Frederick, MD · On-site
$51.50 - $69.50/hr
We provide value across the entire project lifecycle through our core construction management ... By integrating data, artificial intelligence, and advanced technology through our Seamless Platform ...
Project Manager, Data Centers, Security
Frederick, MD · On-site
$51.50 - $69.50/hr
We provide value across the entire project lifecycle through our core construction management ... By integrating data, artificial intelligence, and advanced technology through our Seamless Platform ...
Project Manager
Baltimore, MD · On-site
$83K - $85K/yr
Conduct research and data analysis. * Draft, review, and revise education-related communications ... Manage project contracts and budgets. * Develop and present reports to stakeholders and partners.
Quick apply
Project Manager
Baltimore, MD · On-site
$83K - $85K/yr
Conduct research and data analysis. * Draft, review, and revise education-related communications ... Manage project contracts and budgets. * Develop and present reports to stakeholders and partners.
Data Annotation Project Manager information
How much do data annotation project managers make?
Does data annotation actually pay?
What are the key skills and qualifications needed to thrive as a Data Annotation Project Manager, and why are they important?
How hard is it to get hired by data annotation?
What is the salary of data annotation manager?
What are some common challenges faced by Data Annotation Project Managers, and how can they be managed effectively?
What is the difference between Data Annotation Project Manager vs Data Labeling Specialist?
| Aspect | Data Annotation Project Manager | Data Labeling Specialist |
|---|---|---|
| Credentials | Typically requires project management experience, certifications in data management or related fields | Often requires basic technical skills, familiarity with labeling tools, sometimes certifications in data annotation |
| Work Environment | Oversees teams, manages projects, coordinates workflows in office or remote settings | Performs labeling tasks, often in a remote or on-site environment, focused on data tagging |
| Employer & Industry Usage | Used by tech companies, AI firms, and data service providers for managing annotation projects | Employed within similar industries, focusing on executing labeling tasks under supervision |
The main difference is that the Data Annotation Project Manager oversees and coordinates annotation projects, ensuring quality and deadlines, while the Data Labeling Specialist focuses on executing the labeling tasks themselves. Both roles are essential in the data annotation process but differ in responsibilities and scope.
What is a Data Annotation Project Manager?
Full-time
Posted 20 days ago
Johns Hopkins Medicine rating
7.5
Based on 205 frontline employees who took The Breakroom Quiz
231st of 890 rated healthcare providers
Job description
PREP Research Associate
This position is part of the National Institute of Standards and Technology (NIST) Professional Research Experience Program (PREP). NIST recognizes that its research staff may want to collaborate with researchers at academic institutions on specific projects of mutual interest and, therefore, requires those institutions to be recipients of a PREP award. The PREP program involves staff from a wide range of backgrounds conducting scientific research across various fields. Individuals in this position will perform technical work supporting the collaboration's scientific research.
Research Title:
Reliability of Human and LLM Annotations for AI Risk Assessment
The work will entail:
This project focuses on using Large Language Models (LLMs) to provide annotations of evaluation data (a.k.a., LLM as judge), and the design of an Inter-Annotator Agreement study to assess the reliability of both human and LLM annotations. The candidate will explore assessing the indicators of a given AI-related risk, determining how to identify them, and providing annotators with examples to annotate the presence of various risks. The project aims to develop an annotation framework for AI risk assessment and establish metrics for data quality in AI risk research, supporting broader work at NIST in assessing and measuring the validity and reliability of AI-related risks in data annotation.
U.S. Citizen Preferred
Key responsibilities will include but are not limited to:
- Gain familiarity with existing literature on data annotation and LLM as judge
- Understand NIST's role and ongoing efforts in assessing and measuring the validity and reliability of AI-related risks in data annotation
- Contribute to developing an annotation framework for AI risk assessment
- Collaborate effectively with cross-functional and interdisciplinary stakeholders to ensure successful project outcomes
Deliverables
- Contributions to a NIST report that supports ongoing NIST AI evaluation efforts focused on the design of an Inter-Annotator Agreement to assess the reliability of both human and LLM annotations.
Qualifications
- Background in Computer Science, Data Science, or related field.
- Education level: Bachelor's or Graduate Degree
- Strong interest in data annotation and AI risks
- Familiarity with scientific reading and technical writing
Application Instructions
Please upload the following with your application:
• CV/Resume
*Please limit C.V to 3 pages only and ONLY include a valid email address for your contact info. Your resume will not be considered if the following information is included on your CV/resume.
• Self portraits
• Phone number
• Home address/Country
• Citizenship status
• Languages spoken
• Sex/Gender
Privacy Act Statement
Authority: 15 U.S.C. § 278g-1(e)(1) and (e)(3) and 15 U.S.C. § 272(b) and (c)
Purpose: The National Institute for Standards and Technology (NIST) hosts the Professional Research Experience Program (PREP) which is designed to provide valuable laboratory experience and financial assistance to undergraduates, post-bachelor's degree holders, graduate students, master's degree holders, postdocs, and faculty.
PREP is a 5-year cooperative agreement between NIST laboratories and participating PREP Universities to establish a collaborative research relationship between NIST and U.S. institutions of higher education in the following disciplines including (but may not be limited to) biochemistry, biological sciences, chemistry, computer science, engineering, electronics, materials science, mathematics, nanoscale science, neutron science, physical science, physics, and statistics. This collection of information is needed to facilitate administrative functions of the PREP Program.
Routine Uses: NIST will use the information collected to perform the requisite reviews of the applications to determine eligibility, and to meet programmatic requirements. Disclosure of this information is also subject to all the published routine uses as identified in the Privacy Act System of Records Notices: NIST-1: NIST Associates.
Disclosure: Furnishing this information is voluntary. When you submit the form, you are indicating your voluntary consent for NIST to use of the information you submit for the purpose stated.
What Johns Hopkins Medicine employees say
Pay
Benefits
Hours and flexibility
Workplace
Get the full story on Breakroom
About Johns Hopkins Hospital
Sourced by ZipRecruiter
Industry
Health care and social assistance
Company size
10,000+ Employees
Headquarters location
Baltimore, MD, US
Year founded
1889