Part Time Annotation Specialist Jobs (NOW HIRING)

Specialised Part-Time Consulting OpportunityWe are sharing a specialised part-time consulting opportunity for experienced federal litigation professionals with frequent hands-on use of PACER and CM/ECF for federal court docket research.This role supports an advanced AI initiative focused on evaluating how effectively AI systems perform realistic federal court research. Selected professionals will review completed AI-agent research tasks against actual dockets and records, identify procedural and retrieval errors, refine task difficulty, and help establish rigorous grading standards for PACER-based legal research.Key ResponsibilitiesPACER & Federal Docket ReviewReview AI-generated research against actual PACER and federal court docket informationVerify the correct case, court, parties, procedural posture, outcome, status, and supporting documentsTrace docket histories across filings, motions, orders, judgments, and related entriesIdentify errors involving case numbers, party names, case status, or procedural historyDistinguish open, terminated, transferred, consolidated, and related matters accuratelyFederal Court ResearchUse PACER Case Locator and individual court CM/ECF systems appropriatelyConduct targeted searches using party names, courts, date ranges, case types, and Nature of Suit classificationsAccount for name variants, corporate suffixes, d/b/a entities, subsidiaries, misspellings, and other search complicationsNavigate district, appellate, and bankruptcy court recordsAnalyse relationships between district cases, appeals, consolidated matters, and MDL proceedingsAI Evaluation & Quality AssuranceReview completed AI-agent research workflows for factual and procedural accuracyIdentify plausible but incorrect answers that may escape non-specialist reviewDetermine whether answers are fully supported by available court recordsIdentify failures involving the wrong case, wrong court, incorrect procedural posture, or inaccessible recordsAssess whether tasks genuinely require PACER research rather than being solvable through simpler public sourcesTask & Grading DevelopmentRefine research prompts to improve realism and technical difficultyHelp create authentic federal docket research scenariosDevelop and improve grading guidance for legal research tasksDefine critical failure conditions such as incorrect case identification or reliance on sealed or restricted recordsProvide clear written rationales explaining why an answer should pass or failIdeal ProfileStrong candidates may have:At least 2 years of professional federal litigation or federal court research experienceFrequent hands-on use of PACER and CM/ECF, typically weekly or more, in a current or recent professional roleExperience as a litigation paralegal, docket or court clerk, legal research librarian, litigator, or comparable federal litigation professionalAbility to read and interpret federal docket sheets fluentlyStrong understanding of procedural posture, case status, docket entries, and court structureExperience identifying and resolving party-name and case-search ambiguitiesStrong legal research judgment and attention to detailExcellent written communication and ability to explain research errors preciselyEducational BackgroundA J.D. and bar admission may strengthen an application but are not requiredExperienced litigation paralegals, docket specialists, court professionals, and legal research librarians are strongly relevantFormal training in legal studies, paralegal studies, library science, or a related field may be helpfulEquivalent research-intensive federal litigation experience may also qualifyNice to HaveBankruptcy docket experienceExperience with complex or multidistrict litigationHands-on PACER research across multiple federal courtsFamiliarity with consolidated and related-case structuresExperience tracing district court matters through appealKnowledge of sealed, restricted, and redacted filing rulesExperience with CourtLink, docket-management tools, or comparable legal research platformsPrior AI evaluation, data annotation, or structured quality-review experienceWhy This OpportunityApply specialised federal litigation research expertise to advanced AI evaluationWork with realistic PACER and federal docket research scenariosHelp identify subtle procedural and factual errors in AI-generated legal researchImprove the realism and difficulty of federal court research tasksShape grading standards for accurate, defensible legal researchParticipate in flexible remote consulting work with competitive hourly compensationContract DetailsIndependent contractor roleFully remote with flexible schedulingCompetitive rates between $65–$115 per hour depending on expertise and project scopeFrequent professional use of PACER and CM/ECF is requiredWork may include docket research, AI-agent evaluation, task refinement, grading-guidance development, and quality assuranceWeekly payments via Stripe or WiseProjects may be extended, shortened, or adjusted depending on scope and performanceWork will not involve access to confidential or proprietary information from any employer, client, or institutionAbout the PlatformThis opportunity is available through 24-MAG LLC. We connect experienced professionals with remote consulting opportunities across technical, evaluation, and project-based workstreams.By submitting this application, you acknowledge that your information may be processed by 24-MAG LLC for recruitment and opportunity matching in accordance with our Privacy Policy.