Content Moderation Companies Jobs (NOW HIRING)

Job Summary:
YO IT Consulting is a fast-growing AI Data Services company that delivers training data for major AI companies and foundation-model labs. They are seeking a Red-Teaming Quality Assurance Lead to oversee quality and consistency in AI red-teaming projects, ensuring that safety training data is realistic and policy-aligned. The role involves evaluating outputs, providing feedback, managing quality workflows, and supporting onboarding for contributors.
Responsibilities:
• Quality monitoring: Spot-check red-teaming items, identify quality issues, provide ongoing feedback through DMs, and escalate recurring or critical issues.
• Safety and red-team review: Evaluate adversarial prompts, model responses, risk classifications, safety analyses, policy explanations, and vulnerability reports for accuracy, realism, and usefulness.
• Trainer and QA communication: Update trainers and QAs on Discord about new item guidelines, project changes, workflow updates, quality expectations, and red-teaming-specific review standards.
• Question handling: Respond to trainer/QA questions clearly and promptly, especially around risk categories, adversarial strategy, policy boundaries, edge cases, severity, and rubric interpretation.
• Trainer/QA activation management: DM contributors who are inactive or not working, encourage activation, track follow-ups, and flag availability issues when needed.
• Documentation: Create and maintain red-teaming project documentation, including style guides, trackers, FAQs, quality notes, examples, honeypots, calibration tasks, and onboarding materials.
• Onboarding and training: Schedule and run onboarding/training calls with trainers and QAs to explain project expectations, workflows, rubrics, quality standards, and red-teaming-specific review requirements.
• Quality alignment: Ensure all trainers and QAs apply red-teaming and safety-review guidelines consistently and understand updates as projects evolve.
• Risk review: Flag unsafe, low-quality, unrealistic, policy-inconsistent, or insufficiently documented red-team items.
• Process improvement: Identify recurring quality gaps, propose workflow improvements, and help build scalable QA processes for AI red-teaming projects.
Qualifications:
Required:
• Bachelor’s, Master’s, or professional experience in Computer Science, Cybersecurity, AI Safety, Trust & Safety, Public Policy, Psychology, Linguistics, Law, Security Studies, Risk Analysis, or a related field.
• Strong grasp of the English language to follow project guidelines, communicate with teams, and provide clear written feedback.
• 3+ years of experience in AI safety, red-teaming, cybersecurity, trust and safety, content policy, risk analysis, adversarial testing, model evaluation, content moderation, or related workflows.
• Strong understanding of AI risk categories, adversarial prompting, jailbreak patterns, harmful-content taxonomies, misuse scenarios, policy interpretation, model behavior, and safety evaluation principles.
• Ability to evaluate red-teaming content against detailed rubrics and identify issues such as weak adversarial design, unrealistic scenarios, poor risk categorization, policy misinterpretation, unsafe outputs, or superficial vulnerability testing.
• Highly detail-oriented and organized, with the ability to maintain style guides, FAQs, trackers, onboarding materials, calibration tasks, and documentation.
Preferred:
• Familiarity with areas such as prompt injection, social engineering, cybersecurity abuse, fraud, self-harm safety, extremist content, misinformation, privacy risk, illicit behavior, bias, and model refusal behavior.
• Experience leading or supporting remote teams of red-teamers, reviewers, policy analysts, annotators, researchers, or QAs.
• Comfortable working in fast-moving remote environments using tools such as Discord, Google Sheets, Google Docs, trackers, dashboards, and project management systems.
• Experience with AI training, LLM evaluation, safety evaluations, content moderation QA, policy QA, or rubric-based review.
Company:
Our Core mission is to develop, deploy, or integrate artificial intelligence (AI) — including machine learning (ML), data analytics, automation, natural language processing (NLP), computer vision, and related technologies — to solve real-world problems, improve decision-making, automate repetitive tasks, and deliver intelligent solutions across industries. Founded in 2018, the company is headquartered in Abu Dhabi, Abu Dhabi Emirate, AE, , with a team of 51-200 employees. The company is currently Growth Stage.