Annotation Qa Jobs (NOW HIRING)

Job Summary:
YO IT Consulting is a fast-growing AI Data Services company delivering training data for many of the world’s largest AI companies. They are seeking a Psychology Quality Assurance Lead to oversee quality and consistency across psychology-focused AI training projects, ensuring that psychology training data is accurate, ethical, and aligned with client expectations.
Responsibilities:
• Quality monitoring: Spot-check psychology items, identify quality issues, provide ongoing feedback through DMs, and escalate recurring or critical issues.
• Psychology review: Evaluate AI-generated psychology explanations, mental-health content, research summaries, behavioral guidance, theory applications, and safety-sensitive responses for accuracy and appropriateness.
• Trainer and QA communication: Update trainers and QAs on Discord about new item guidelines, project changes, workflow updates, quality expectations, and psychology-specific review standards.
• Question handling: Respond to trainer/QA questions clearly and promptly, especially around psychological concepts, evidence quality, ethical boundaries, mental-health safety, diagnostic caution, and rubric interpretation.
• Trainer/QA activation management: DM contributors who are inactive or not working, encourage activation, track follow-ups, and flag availability issues when needed.
• Documentation: Create and maintain psychology project documentation, including style guides, trackers, FAQs, quality notes, examples, honeypots, calibration tasks, and onboarding materials.
• Onboarding and training: Schedule and run onboarding/training calls with trainers and QAs to explain project expectations, workflows, rubrics, quality standards, and psychology-specific review requirements.
• Quality alignment: Ensure all trainers and QAs apply psychology-review guidelines consistently and understand updates as projects evolve.
• Safety and ethics review: Flag unsafe, overconfident, stigmatizing, pseudoscientific, diagnostically inappropriate, or crisis-insensitive psychology content.
• Process improvement: Identify recurring quality gaps, propose workflow improvements, and help build scalable QA processes for psychology AI training projects.
Qualifications:
Required:
• Bachelor’s, Master’s, PhD, PsyD, or equivalent professional background in Psychology, Clinical Psychology, Counseling, Behavioral Science, Cognitive Science, Neuroscience, Social Work, Psychiatry-adjacent research, Education, or a closely related field.
• Strong grasp of the English language to follow project guidelines, communicate with teams, and provide clear written feedback.
• 3+ years of experience in psychology research, clinical/counseling settings, behavioral science, teaching, academic writing, mental-health content review, psychological assessment, or related workflows.
• Strong understanding of psychological theories, research methods, cognitive and social psychology, developmental psychology, abnormal psychology, ethics, evidence-based practice, and mental-health safety considerations.
• Ability to evaluate psychology content against detailed rubrics and identify issues such as overdiagnosis, unsafe advice, unsupported claims, flawed study interpretation, pseudoscience, stigmatizing language, or inappropriate therapeutic recommendations.
• Highly detail-oriented and organized, with the ability to maintain style guides, FAQs, trackers, onboarding materials, calibration tasks, and documentation.
Preferred:
• Familiarity with areas such as CBT, developmental psychology, personality, trauma-informed communication, behavioral interventions, mental-health literacy, research design, statistics, and ethical guidelines.
• Experience leading or supporting remote teams of researchers, clinicians, educators, reviewers, annotators, or QAs.
• Comfortable working in fast-moving remote environments using tools such as Discord, Google Sheets, Google Docs, trackers, dashboards, and project management systems.
• Experience with AI training, data annotation, LLM evaluation, psychology QA, mental-health content review, or rubric-based review.
Company:
Our Core mission is to develop, deploy, or integrate artificial intelligence (AI) — including machine learning (ML), data analytics, automation, natural language processing (NLP), computer vision, and related technologies — to solve real-world problems, improve decision-making, automate repetitive tasks, and deliver intelligent solutions across industries. Founded in 2018, the company is headquartered in Abu Dhabi, Abu Dhabi Emirate, AE, , with a team of 51-200 employees. The company is currently Growth Stage.