Remote Machine Learning Qa Jobs (NOW HIRING)

Job Summary:
YO IT Consulting is a fast-growing AI Data Services company delivering training data for many of the world’s largest AI companies and foundation-model labs. They are seeking a Python(Machine learning) Quality Assurance Lead to oversee quality and consistency across Python machine learning AI training projects, evaluate output quality, and ensure contributors follow expected quality standards.
Responsibilities:
• Quality monitoring: Spot-check Python ML items, identify quality issues, provide feedback through DMs, and escalate recurring or critical issues.
• Code and ML review: Evaluate AI-generated Python code, ML pipelines, data-preprocessing steps, model training workflows, evaluation logic, debugging responses, and explanations for correctness and reproducibility.
• Trainer and QA communication: Update contributors on Discord about guideline changes, workflow updates, and Python/ML-specific review standards.
• Question handling: Respond to questions around Python syntax, package usage, data leakage, model validation, metrics, statistical assumptions, reproducibility, notebooks, and rubric interpretation.
• Trainer/QA activation management: DM inactive contributors, encourage activation, track follow-ups, and flag availability issues.
• Documentation: Create and maintain Python ML style guides, trackers, FAQs, examples, honeypots, calibration tasks, and onboarding materials.
• Onboarding and training: Run onboarding/training calls for Python ML contributors.
• Risk review: Flag misleading, overconfident, statistically invalid, non-reproducible, insecure, or non-production-ready Python ML recommendations.
• Process improvement: Identify recurring quality gaps and build scalable QA processes.
Qualifications:
Required:
• Bachelor’s, Master’s, or PhD degree in Computer Science, Machine Learning, Data Science, Statistics, Mathematics, Engineering, or a closely related quantitative field.
• Strong grasp of English to follow guidelines, communicate with teams, and provide clear technical feedback.
• 3+ years of professional experience in Python development, machine learning, data science, ML engineering, model evaluation, research engineering, technical review, or ML education.
• Strong understanding of Python fundamentals such as data structures, functions, classes, iterators, comprehensions, exception handling, virtual environments, package management, testing, and debugging.
• Strong understanding of ML topics such as supervised/unsupervised learning, feature engineering, train/test splits, cross-validation, model selection, data leakage, regression, classification, clustering, metrics, bias/variance, regularization, and reproducibility.
• Ability to evaluate ML content against detailed rubrics and identify issues such as flawed methodology, wrong metrics, data leakage, non-reproducible code, invalid assumptions, hallucinated APIs, misleading conclusions, or incomplete explanations.
• Comfortable working in fast-moving remote environments using Discord, Google Sheets, Google Docs, trackers, dashboards, GitHub, and project management systems.
• Highly organized and able to maintain style guides, trackers, FAQs, onboarding materials, honeypots, calibration tasks, and quality documentation.
Preferred:
• Familiarity with NumPy, pandas, scikit-learn, PyTorch, TensorFlow/Keras, XGBoost/LightGBM, Jupyter, matplotlib, seaborn, MLflow, Hugging Face, SQL, GitHub, Docker, and CI/CD.
• Experience leading or supporting remote teams of trainers, annotators, reviewers, engineers, data scientists, ML researchers, coding mentors, or QAs.
• Experience with AI training, data annotation, LLM evaluation, code QA, ML QA, or rubric-based technical review.
Company:
Our Core mission is to develop, deploy, or integrate artificial intelligence (AI) — including machine learning (ML), data analytics, automation, natural language processing (NLP), computer vision, and related technologies — to solve real-world problems, improve decision-making, automate repetitive tasks, and deliver intelligent solutions across industries. Founded in 2018, the company is headquartered in Abu Dhabi, Abu Dhabi Emirate, AE, , with a team of 51-200 employees. The company is currently Growth Stage.