1

Annotation Judge Jobs in Florida (NOW HIRING)

Perform AI/ML-related tasks such as data labeling, annotation, and content evaluation * Participate ... These tools assist our recruitment team but do not replace human judgment. Final hiring decisions ...

AI Finance Expert - Remote

Miami, FL · Remote

$100 - $200/hr

... judgment are what matter most. Key Responsibilities * Analyze, review, and edit AI-generated ... Data Annotation * Problem-Solving * Independent Research * Attention to Detail Preferred ...

Perform AI/ML-related tasks such as data labeling, annotation, and content evaluation * Participate ... These tools assist our recruitment team but do not replace human judgment. Final hiring decisions ...

Perform AI/ML-related tasks such as data labeling, annotation, and content evaluation * Participate ... These tools assist our recruitment team but do not replace human judgment. Final hiring decisions ...

Perform AI/ML-related tasks such as data labeling, annotation, and content evaluation * Participate ... These tools assist our recruitment team but do not replace human judgment. Final hiring decisions ...

Perform AI/ML-related tasks such as data labeling, annotation, and content evaluation * Participate ... These tools assist our recruitment team but do not replace human judgment. Final hiring decisions ...

Perform AI/ML-related tasks such as data labeling, annotation, and content evaluation * Participate ... These tools assist our recruitment team but do not replace human judgment. Final hiring decisions ...

next page

Showing results 1-20

Annotation Judge information

What is an annotation judge?

An Annotation Judge is a professional who evaluates the quality and accuracy of labeled data, such as text, images, or audio, which has been annotated for use in machine learning and artificial intelligence projects. Their main responsibility is to review, verify, and ensure that the data annotations meet specific guidelines and standards. Annotation Judges play a critical role in improving the reliability of training datasets, which directly impacts the performance of AI systems. They often work closely with data annotators, quality assurance teams, and project managers to maintain high data quality.

What are the key skills and qualifications needed to thrive as an annotation judge, and why are they important?

To thrive as an Annotation Judge, you need strong analytical skills, attention to detail, and subject matter expertise relevant to the data being evaluated, usually supported by a degree in a related field. Familiarity with annotation platforms, data labeling tools, and quality assurance systems is typically required. Excellent communication, impartiality, and critical thinking help you provide clear feedback and maintain high annotation standards. These skills are crucial to ensure data accuracy and consistency, which directly impact the performance of machine learning models.

What are some common challenges faced by annotation judges, and how can they effectively overcome them?

Annotation Judges often face challenges such as maintaining impartiality, handling ambiguous or subjective data, and ensuring high consistency across large volumes of work. To overcome these, it’s essential to follow established guidelines closely, communicate regularly with team members for clarification, and participate in calibration sessions. Staying detail-oriented and seeking feedback can also help maintain accuracy and fairness in their assessments.

What is the difference between Annotation Judge vs Data Annotator?

AspectAnnotation JudgeData Annotator
CredentialsTypically requires basic education, sometimes certification in data labelingUsually requires similar or less formal education, often on-the-job training
Work EnvironmentOffice or remote, working with data labeling platformsOffice or remote, performing data labeling tasks
Industry UsageUsed across AI, machine learning, and data science projectsCommon in AI, machine learning, and data preparation workflows
Search & Comparison IntentOften compared for roles involving data review and quality controlCompared for entry-level data labeling roles

The main difference between an Annotation Judge and a Data Annotator lies in their roles. Annotation Judges typically review and validate annotations made by Data Annotators, ensuring quality and accuracy. Data Annotators perform the initial labeling of data. Both roles are essential in AI data pipelines, with Annotation Judges focusing on quality control and Data Annotators on data preparation.

What job categories do people searching Annotation Judge jobs in Florida look for?

The top searched job categories for Annotation Judge jobs in Florida are:

What cities in Florida are hiring for Annotation Judge jobs?

Cities in Florida with the most Annotation Judge job openings:

Infographic showing various Annotation Judge job openings in Florida as of August 2026, with employment types broken down into 70% Full Time, and 30% Part Time. Highlights an 46% In-person, and 54% Remote job distribution.

Survey Watch Stander / Video Annotator - Level I (Mesophotic Deep Benthic Communities Survey)

Think Tank, Inc.

Panama City, FL

Contractor

Re-posted 27 days ago


Job description

*US Citizenship Required

*Position is Subject to Contract Award

Position Overview

SEFSC-PEMD personnel are critical collaborators in restoration work in response to the 2010 DWH oil spill in the Gulf of Mexico. This aspect of the project involves surveying mesophotic and deep benthic reef communities (50 - 1800 m in depth). The survey utilizes a variety of survey platforms (e.g. ROV, AUV) to detect, classify and enumerate assemblages of corals, sponges, octocorals, crustaceans, mollusks, fishes and other fauna. Tasks should include: Survey Watch Stander for Mesophotic Deep Benthic Communities (MDBC) surveys conducted on NOAA Ships and Small Research Vessels and participate on other PEMD resource surveys, as requested.

Position Title

Survey Watch Stander and Video Annotation (Level I)

Labor Category

Technical Support Services – Level I

Location

Panama City, FL

Start Date

TBD – Upon Award

Schedule

Full-Time, 40 hrs/week (extended hours during surveys)

Security

Low Risk – Public Trust Background Investigation Required

Travel

Domestic and international travel may be required for at-sea surveys, project-related workshops, and working group meetings; authorized by COR

Key Responsibilities

a) Survey Duties

Tasks shall include but is not limited to:

  • Knowledgeable in data collection MDBC SOPs, and execution of at-sea mapping and ground truthing missions as Field Party Chief or Watch Leader.
  • Pre- and post-survey inventory and maintenance of equipment and gear.
  • Conducting data collection, biological and/or environmental sampling and logging and processing data and/or samples under field conditions, as required and according to prescribed MDBC SOP.
  • Complete incident reporting procedures as necessary in accordance with applicable NOAA Small Boat Program requirements (small boat operations only)

b) Laboratory Duties

Video Annotation roles require subject matter expertise on biota associated with Mesophotic/Deep Reefs of the Gulf of America. Primary duties focus on video annotation to derive habitat and fish detection and enumeration. Collected video data is annotated using a semi-automated multi-stage process including 1) automated processing using C-Vision tools, 2) human-in-the-loop review and correction, 3) data review and QA/QC, 4) data output/ingest into GCRF Oracle databases, 4) updating GitHUB data for updates and project management.

  • Subject matter expert for identifying marine fishes in all life stages from mesophotic and deep reef habitats of the Gulf of America.
  • Video annotation, review, and data QA/QC of ROV-based footage.
  • Management of review and tracking sheets.
  • Updating project management tools for status updates.
  • Updating annotation tracking sheets.

c) Data Management Support

Video Annotation duties involve basic duties including data entry, QA/QC, and validation. In the near future these will also include determining and setting standards for automated data outputs to ensure interoperability and democratization of datasets.

  • Utilize inventory of existing bathymetric information in U.S. EEZ of Gulf of Mexico between 50 and 1800 m into MDBC database. Task to be coordinated with NCEI and NCCOS-Biogeography Branch.
  • Inputs and corrects data in MDBC database, associated data entry applications and MDBC survey SOP (including metadata).
  • Input updates into video annotation projects management tools.

d) Analytical Support

Video Annotation roles include basic duties of data preparation and validation prior to analysis, assisting in drafting reports and manuscripts, and review.

  • Maintain knowledge of relevant scientific literature, provide support for data analyses and develop data summaries.
  • Maintain updates in analysis projects using project management tool.
  • Develop basic data reports, and presentations. Assist in preparing graphs and visuals from survey outcomes.

e) Outreach and Education

Participate in public outreach and education efforts. Examples of outreach and education activities include marine science education at local schools, participating as judges for local science fairs, conducting and participating in outreach and education tours of the SEFSC facilities and NOAA ships (in conjunction with federal employees), manning of outreach booths at local fairs and festivals.

Minimum Qualifications (Required)

  • MS Degree in Marine Science or a related field of study or BS Degree in Marine Science or a related field of study plus 1 (1) years of relevant experience.
  • Computer literate and experienced with common word processing, database, statistical, and graphics programs. Capable of writing reports at the college level and making oral presentations.
  • Experience as data collector on at sea surveys. Knowledge and skills in using software that handles data streams for hydrological, and biological data.
  • Proficient in the use of fish, invertebrate, and protected species identification books and keys. Knowledge and identification of reef taxa from the Gulf of America.
  • Expert knowledge of major structure forming invertebrates found in mesophotic and deep benthic communities of the Gulf of America.
  • Knowledge of procedures when encountering endangered or threatened species.
  • Sea going contractors who participate on research cruises on Office of Marine and Aviation Operations' NOAA ships will be required to annually/biennially (age dependent) submit a NOAA Health Services Questionnaire (NOAA Form 57- 10-01) and Tuberculosis Screening Form (NOAA Form NF 57-10-02) to OMAO health services to determine fitness for sea duty. Sea going contractors must have a current tetanus vaccine (once every 10 years).

Required Training

Training Requirement

Due Date

NOAA IT security (required for all contractors)

Certificates due within 3 business days of contract/task order start

NOAA Safety training (required for all contractors working at a Government facility)

Certificates due within 3 business days of contract/task order start

Records Management 101 (required for all contractors)

Certificates due within 3 business days of contract/task order start

Department of Homeland Security (DHS) Active Threats to the Workplace Awareness Seminar

Certificates due within 3 business days of contract/task order start

Sexual Assault/Sexual Harassment Prevention and Response Training IAW NAM 1330-52.222-70(b)(6)

Certificates due no later than 5 business days after contract/task order start