The PREP program involves staff from a wide range of backgrounds conducting scientific research ... Gain familiarity with existing literature on data annotation and LLM as judge * Understand NIST ...
The PREP program involves staff from a wide range of backgrounds conducting scientific research ... Gain familiarity with existing literature on data annotation and LLM as judge * Understand NIST ...
Data/AI Program Manager
Bethesda, MD · On-site
$98K - $163K/yr
Lead IC-wide data management programs aligned with the IC Data Strategy. * Develop and execute strategic plans to enhance data governance, metadata management, and data sharing across IC elements.
Data/AI Program Manager
Bethesda, MD · On-site
$98K - $163K/yr
Lead IC-wide data management programs aligned with the IC Data Strategy. * Develop and execute strategic plans to enhance data governance, metadata management, and data sharing across IC elements.
Data/AI Program Manager
$98K - $163K/yr
Lead IC-wide data management programs aligned with the IC Data Strategy. * Develop and execute strategic plans to enhance data governance, metadata management, and data sharing across IC elements.
Data/AI Program Manager
$98K - $163K/yr
Lead IC-wide data management programs aligned with the IC Data Strategy. * Develop and execute strategic plans to enhance data governance, metadata management, and data sharing across IC elements.
Program Manager
Potomac, MD · On-site
$120K - $170K/yr
We specialize in data sciences and cloud solutions, application development and maintenance, program management, and health IT operations leveraging experience and trusted talent to solve the federal ...
Quick apply
Program Manager
Potomac, MD · On-site
$120K - $170K/yr
We specialize in data sciences and cloud solutions, application development and maintenance, program management, and health IT operations leveraging experience and trusted talent to solve the federal ...
Program Manager
College Park, MD · On-site
Visit our website at: www.dcca.com Program Manager Candidate must have an active TS/SCI W/ POLYGRAPH Program : DCCA is recruiting candidates for an exciting opportunity to support leading edge ...
Program Manager
College Park, MD · On-site
Visit our website at: www.dcca.com Program Manager Candidate must have an active TS/SCI W/ POLYGRAPH Program : DCCA is recruiting candidates for an exciting opportunity to support leading edge ...
Program Manager - Networking
Baltimore, MD · Hybrid
$115K - $115K/yr
NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with ... We are currently seeking a Program Manager - Networking to join our team in Baltimore, Maryland (US ...
Program Manager - Networking
Baltimore, MD · Hybrid
$115K - $115K/yr
NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with ... We are currently seeking a Program Manager - Networking to join our team in Baltimore, Maryland (US ...
Program Manager - Networking
Baltimore, MD · On-site
$115K - $115K/yr
NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with ... We are currently seeking a Program Manager - Networking to join our team in Baltimore, Maryland (US ...
Program Manager - Networking
Baltimore, MD · On-site
$115K - $115K/yr
NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with ... We are currently seeking a Program Manager - Networking to join our team in Baltimore, Maryland (US ...
Program Manager - Networking
Baltimore, MD · Hybrid
$115K - $115K/yr
NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with ... We are currently seeking a Program Manager - Networking to join our team in Baltimore, Maryland (US ...
Program Manager - Networking
Baltimore, MD · Hybrid
$115K - $115K/yr
NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with ... We are currently seeking a Program Manager - Networking to join our team in Baltimore, Maryland (US ...
Program Manager
Greenbelt, MD · On-site
$140K - $190K/yr
Program Management and Leadership: * Overall responsibility for program execution of a $200+M ... Cloud, Earth Science, Data Collection, Modeling, etc; schedule and cost * Coordination across ...
Program Manager
Greenbelt, MD · On-site
$140K - $190K/yr
Program Management and Leadership: * Overall responsibility for program execution of a $200+M ... Cloud, Earth Science, Data Collection, Modeling, etc; schedule and cost * Coordination across ...
Program Manager
Greenbelt, MD · Hybrid
$140K - $190K/yr
Program Management and Leadership: * Overall responsibility for program execution of a $200+M ... Cloud, Earth Science, Data Collection, Modeling, etc; schedule and cost * Coordination across ...
Program Manager
Greenbelt, MD · Hybrid
$140K - $190K/yr
Program Management and Leadership: * Overall responsibility for program execution of a $200+M ... Cloud, Earth Science, Data Collection, Modeling, etc; schedule and cost * Coordination across ...
Program Manager
Greenbelt, MD · Hybrid
$140K - $190K/yr
Program Management and Leadership: * Overall responsibility for program execution of a $200+M ... Cloud, Earth Science, Data Collection, Modeling, etc; schedule and cost * Coordination across ...
Program Manager
Greenbelt, MD · Hybrid
$140K - $190K/yr
Program Management and Leadership: * Overall responsibility for program execution of a $200+M ... Cloud, Earth Science, Data Collection, Modeling, etc; schedule and cost * Coordination across ...
Program Manager
Rockville, MD · On-site
Copper River Data Solutions is seeking an experienced Program Manager to lead and oversee the successful execution of a federal records and information management program in Rockville, Maryland. The ...
Program Manager
Rockville, MD · On-site
Copper River Data Solutions is seeking an experienced Program Manager to lead and oversee the successful execution of a federal records and information management program in Rockville, Maryland. The ...
Program Manager
College Park, MD · On-site
$72K - $86K/yr
Summary The Program Manager - Safety and Health Regulatory Compliance develops, manages, and ... The role includes analyzing safety data and metrics (both lagging and leading indicators) and ...
Program Manager
College Park, MD · On-site
$72K - $86K/yr
Summary The Program Manager - Safety and Health Regulatory Compliance develops, manages, and ... The role includes analyzing safety data and metrics (both lagging and leading indicators) and ...
Program Manager
Suitland, MD · On-site
Program Manager Location: Suitland, MD Clearance Level: TS with ability to obtain an SCI Required ... Data Reports (CSDR), and Cost Data Summary Reports (CDSR) • Ensure accurate and timely monthly ...
Program Manager
Suitland, MD · On-site
Program Manager Location: Suitland, MD Clearance Level: TS with ability to obtain an SCI Required ... Data Reports (CSDR), and Cost Data Summary Reports (CDSR) • Ensure accurate and timely monthly ...
Program Manager
Suitland, MD · On-site
Program Manager Location: Suitland, MD Clearance Level: TS with ability to obtain an SCI Required ... Data Reports (CSDR), and Cost Data Summary Reports (CDSR) Ensure accurate and timely monthly ...
Program Manager
Suitland, MD · On-site
Program Manager Location: Suitland, MD Clearance Level: TS with ability to obtain an SCI Required ... Data Reports (CSDR), and Cost Data Summary Reports (CDSR) Ensure accurate and timely monthly ...
Program Manager---Managed Services
Rockville, MD · On-site
$157K - $262K/yr
Job Summary: NTT DATA is seeking an experienced Program Manager to support one of the nation's premier biomedical research organizations. This role supports the delivery of enterprise IT ...
New
Program Manager---Managed Services
Rockville, MD · On-site
$157K - $262K/yr
Job Summary: NTT DATA is seeking an experienced Program Manager to support one of the nation's premier biomedical research organizations. This role supports the delivery of enterprise IT ...
New
Program Manager
Baltimore, MD · On-site
Coordinate resources across multiple interconnected project workstreams (e.g., data conversion ... Program Management - 5 years Progressive experience in managing large scale, complex technical ...
Program Manager
Baltimore, MD · On-site
Coordinate resources across multiple interconnected project workstreams (e.g., data conversion ... Program Management - 5 years Progressive experience in managing large scale, complex technical ...
Program Manager
Columbia, MD · On-site +1
$140K - $190K/yr
... data. * Develop, maintain, and control all required program documentation, including: * Program ... Manage programs through a matrix organization, leveraging functional teams across TCOM to execute ...
Program Manager
Columbia, MD · On-site +1
$140K - $190K/yr
... data. * Develop, maintain, and control all required program documentation, including: * Program ... Manage programs through a matrix organization, leveraging functional teams across TCOM to execute ...
Program Manager
Rockville, MD · On-site
Copper River Data Solutions is seeking an experienced Program Manager to lead and oversee the successful execution of a federal records and information management program in Rockville, Maryland. The ...
Program Manager
Rockville, MD · On-site
Copper River Data Solutions is seeking an experienced Program Manager to lead and oversee the successful execution of a federal records and information management program in Rockville, Maryland. The ...
Program Manager
Suitland, MD · On-site
Minimum 12+ years of progressive experience managing large, complex federal IT programs, including multi-workstream delivery across engineering, operations, security, and data domains. * Demonstrated ...
Program Manager
Suitland, MD · On-site
Minimum 12+ years of progressive experience managing large, complex federal IT programs, including multi-workstream delivery across engineering, operations, security, and data domains. * Demonstrated ...
Data Annotation Program Manager information
What are the key skills and qualifications needed to thrive as a Data Annotation Program Manager, and why are they important?
To thrive as a Data Annotation Program Manager, you need expertise in project management, data quality assessment, and a solid understanding of machine learning or data annotation processes, typically supported by a relevant degree. Familiarity with annotation platforms, workflow management tools, and data labeling software is essential, along with knowledge of quality assurance frameworks. Strong leadership, problem-solving abilities, and effective communication are crucial soft skills that help manage diverse teams and ensure stakeholder alignment. These skills are important to maintain high data quality, meet project deadlines, and drive successful AI model training initiatives.
How does a Data Annotation Program Manager coordinate with cross-functional teams to ensure project success?
A Data Annotation Program Manager regularly collaborates with engineering, data science, and quality assurance teams to align annotation guidelines, project timelines, and quality standards. They often facilitate meetings to clarify requirements, resolve ambiguities in data labeling, and provide feedback on annotation accuracy. This role serves as a bridge between technical teams and annotation staff, ensuring open communication and timely resolution of challenges, which is critical for delivering high-quality datasets essential for machine learning and AI projects.
What are Data Annotation Program Managers?
Data Annotation Program Managers are professionals who oversee and coordinate data labeling projects, ensuring that data used for machine learning and artificial intelligence is accurately tagged and prepared. They manage teams of annotators, set project guidelines, monitor quality, and ensure deadlines are met. Their role is crucial for building high-quality datasets that enable reliable AI model training. Program Managers often collaborate with data scientists, engineers, and stakeholders to define requirements and improve annotation processes.
What are popular job titles related to Data Annotation Program Manager jobs in Maryland? For Data Annotation Program Manager jobs in Maryland, the most frequently searched job titles are:
What job categories do people searching Data Annotation Program Manager jobs in Maryland look for? The top searched job categories for Data Annotation Program Manager jobs in Maryland are:
What cities in Maryland are hiring for Data Annotation Program Manager jobs? Cities in Maryland with the most Data Annotation Program Manager job openings:
Full-time
Posted 29 days ago
Johns Hopkins Medicine rating
7.4
Based on 203 frontline employees who took The Breakroom Quiz
227th of 877 rated healthcare providers
Job description
Description
PREP Research Associate
This position is part of the National Institute of Standards and Technology (NIST) Professional Research Experience Program (PREP). NIST recognizes that its research staff may want to collaborate with researchers at academic institutions on specific projects of mutual interest and, therefore, requires those institutions to be recipients of a PREP award. The PREP program involves staff from a wide range of backgrounds conducting scientific research across various fields. Individuals in this position will perform technical work supporting the collaboration's scientific research.
Research Title:
Reliability of Human and LLM Annotations for AI Risk Assessment
The work will entail:
This project focuses on using Large Language Models (LLMs) to provide annotations of evaluation data (a.k.a., LLM as judge), and the design of an Inter-Annotator Agreement study to assess the reliability of both human and LLM annotations. The candidate will explore assessing the indicators of a given AI-related risk, determining how to identify them, and providing annotators with examples to annotate the presence of various risks. The project aims to develop an annotation framework for AI risk assessment and establish metrics for data quality in AI risk research, supporting broader work at NIST in assessing and measuring the validity and reliability of AI-related risks in data annotation.
U.S. Citizen Preferred
Key responsibilities will include but are not limited to:
Deliverables
Qualifications
Application Instructions
Please upload the following with your application:
• CV/Resume
*Please limit C.V to 3 pages only and ONLY include a valid email address for your contact info. Your resume will not be considered if the following information is included on your CV/resume.
• Self portraits
• Phone number
• Home address/Country
• Citizenship status
• Languages spoken
• Sex/Gender
Privacy Act Statement
Authority: 15 U.S.C. § 278g-1(e)(1) and (e)(3) and 15 U.S.C. § 272(b) and (c)
Purpose: The National Institute for Standards and Technology (NIST) hosts the Professional Research Experience Program (PREP) which is designed to provide valuable laboratory experience and financial assistance to undergraduates, post-bachelor's degree holders, graduate students, master's degree holders, postdocs, and faculty.
PREP is a 5-year cooperative agreement between NIST laboratories and participating PREP Universities to establish a collaborative research relationship between NIST and U.S. institutions of higher education in the following disciplines including (but may not be limited to) biochemistry, biological sciences, chemistry, computer science, engineering, electronics, materials science, mathematics, nanoscale science, neutron science, physical science, physics, and statistics. This collection of information is needed to facilitate administrative functions of the PREP Program.
Routine Uses: NIST will use the information collected to perform the requisite reviews of the applications to determine eligibility, and to meet programmatic requirements. Disclosure of this information is also subject to all the published routine uses as identified in the Privacy Act System of Records Notices: NIST-1: NIST Associates.
Disclosure: Furnishing this information is voluntary. When you submit the form, you are indicating your voluntary consent for NIST to use of the information you submit for the purpose stated.
PREP Research Associate
This position is part of the National Institute of Standards and Technology (NIST) Professional Research Experience Program (PREP). NIST recognizes that its research staff may want to collaborate with researchers at academic institutions on specific projects of mutual interest and, therefore, requires those institutions to be recipients of a PREP award. The PREP program involves staff from a wide range of backgrounds conducting scientific research across various fields. Individuals in this position will perform technical work supporting the collaboration's scientific research.
Research Title:
Reliability of Human and LLM Annotations for AI Risk Assessment
The work will entail:
This project focuses on using Large Language Models (LLMs) to provide annotations of evaluation data (a.k.a., LLM as judge), and the design of an Inter-Annotator Agreement study to assess the reliability of both human and LLM annotations. The candidate will explore assessing the indicators of a given AI-related risk, determining how to identify them, and providing annotators with examples to annotate the presence of various risks. The project aims to develop an annotation framework for AI risk assessment and establish metrics for data quality in AI risk research, supporting broader work at NIST in assessing and measuring the validity and reliability of AI-related risks in data annotation.
U.S. Citizen Preferred
Key responsibilities will include but are not limited to:
- Gain familiarity with existing literature on data annotation and LLM as judge
- Understand NIST's role and ongoing efforts in assessing and measuring the validity and reliability of AI-related risks in data annotation
- Contribute to developing an annotation framework for AI risk assessment
- Collaborate effectively with cross-functional and interdisciplinary stakeholders to ensure successful project outcomes
Deliverables
- Contributions to a NIST report that supports ongoing NIST AI evaluation efforts focused on the design of an Inter-Annotator Agreement to assess the reliability of both human and LLM annotations.
Qualifications
- Background in Computer Science, Data Science, or related field.
- Education level: Bachelor's or Graduate Degree
- Strong interest in data annotation and AI risks
- Familiarity with scientific reading and technical writing
Application Instructions
Please upload the following with your application:
• CV/Resume
*Please limit C.V to 3 pages only and ONLY include a valid email address for your contact info. Your resume will not be considered if the following information is included on your CV/resume.
• Self portraits
• Phone number
• Home address/Country
• Citizenship status
• Languages spoken
• Sex/Gender
Privacy Act Statement
Authority: 15 U.S.C. § 278g-1(e)(1) and (e)(3) and 15 U.S.C. § 272(b) and (c)
Purpose: The National Institute for Standards and Technology (NIST) hosts the Professional Research Experience Program (PREP) which is designed to provide valuable laboratory experience and financial assistance to undergraduates, post-bachelor's degree holders, graduate students, master's degree holders, postdocs, and faculty.
PREP is a 5-year cooperative agreement between NIST laboratories and participating PREP Universities to establish a collaborative research relationship between NIST and U.S. institutions of higher education in the following disciplines including (but may not be limited to) biochemistry, biological sciences, chemistry, computer science, engineering, electronics, materials science, mathematics, nanoscale science, neutron science, physical science, physics, and statistics. This collection of information is needed to facilitate administrative functions of the PREP Program.
Routine Uses: NIST will use the information collected to perform the requisite reviews of the applications to determine eligibility, and to meet programmatic requirements. Disclosure of this information is also subject to all the published routine uses as identified in the Privacy Act System of Records Notices: NIST-1: NIST Associates.
Disclosure: Furnishing this information is voluntary. When you submit the form, you are indicating your voluntary consent for NIST to use of the information you submit for the purpose stated.
What Johns Hopkins Medicine employees say
Pay
Benefits
Hours and flexibility
Workplace
Get the full story on Breakroom
About Johns Hopkins Hospital
Sourced by ZipRecruiter
Industry
Health care and social assistance
Company size
10,000+ Employees
Headquarters location
Baltimore, MD, US
Year founded
1889