WI · On-site
$110 - $165/hr
Use A/B tests and offline/online evaluation frameworks to measure model impact and guide iterative improvement Requirements * Bachelor's degree in Computer Science or a related technical field; or ...
New
Willing to complete R3C's on-line Evaluation training.
Willing to complete R3C's on-line Evaluation training.
Willing to complete R3C's on-line Evaluation training.
Willing to complete R3C's on-line Evaluation training.
Senior Machine Learning Engineer, User Signal & Ads
Bellevue, WA · On-site
$185 - $235/hr
Own the model lifecycle : data preparation, feature engineering, training, offline/online evaluation, deployment, monitoring, and iteration -- with rigorous A/B testing and clear business metrics ...
New
Senior Machine Learning Engineer, User Signal & Ads
Bellevue, WA · On-site
$185 - $235/hr
Own the model lifecycle : data preparation, feature engineering, training, offline/online evaluation, deployment, monitoring, and iteration -- with rigorous A/B testing and clear business metrics ...
New
... online evaluation infrastructure -- measure the gap between them, don't assume it. • Publish ranking and matching APIs for product surfaces, with latency and quality SLOs. • Instrument model ...
... online evaluation infrastructure -- measure the gap between them, don't assume it. • Publish ranking and matching APIs for product surfaces, with latency and quality SLOs. • Instrument model ...
Senior AI Agent & Evaluations Engineer
$107K - $146K/yr
... online evaluation systems and automated scoring of live traffic • Experience integrating AI systems with Salesforce, AWS Connect, or customer engagement platforms • Background in customer-facing ...
Senior AI Agent & Evaluations Engineer
$107K - $146K/yr
... online evaluation systems and automated scoring of live traffic • Experience integrating AI systems with Salesforce, AWS Connect, or customer engagement platforms • Background in customer-facing ...
Founding Machine Learning Engineer
Boston, MA · On-site
$180 - $240/hr
Stand up offline and online evaluation infrastructure -- measure the gap between them, don't assume it. * Publish ranking and matching APIs for product surfaces, with latency and quality SLOs.
Founding Machine Learning Engineer
Boston, MA · On-site
$180 - $240/hr
Stand up offline and online evaluation infrastructure -- measure the gap between them, don't assume it. * Publish ranking and matching APIs for product surfaces, with latency and quality SLOs.
Engineering Manager, Evals
San Francisco, CA · On-site
$130 - $160/hr
The team also owns online evaluation systems that track agent quality in production, and the close integration between online and offline evaluations. The evaluation systems that this team builds ...
New
Engineering Manager, Evals
San Francisco, CA · On-site
$130 - $160/hr
The team also owns online evaluation systems that track agent quality in production, and the close integration between online and offline evaluations. The evaluation systems that this team builds ...
New
... and online evaluation. * Drive quality, cost, and latency trade-off decisions with data - selecting models, routing strategies, and inference configurations that meet product requirements within ...
... and online evaluation. * Drive quality, cost, and latency trade-off decisions with data - selecting models, routing strategies, and inference configurations that meet product requirements within ...
Stand up offline and online evaluation infrastructure - measure the gap between them, don't assume it. * Publish ranking and matching APIs for product surfaces, with latency and quality SLOs.
Stand up offline and online evaluation infrastructure - measure the gap between them, don't assume it. * Publish ranking and matching APIs for product surfaces, with latency and quality SLOs.
Staff Software Engineer AI/ML
San Jose, CA · On-site
$163 - $253/hr
... for offline and online evaluation. * Drive quality, cost, and latency trade‑off decisions with data--selecting models, routing strategies, and inference configurations that meet product ...
Staff Software Engineer AI/ML
San Jose, CA · On-site
$163 - $253/hr
... for offline and online evaluation. * Drive quality, cost, and latency trade‑off decisions with data--selecting models, routing strategies, and inference configurations that meet product ...
... for offline and online evaluation. * Drive quality, cost, and latency trade-off decisions with data -- selecting models, routing strategies, and inference configurations that meet product ...
Quick apply
... for offline and online evaluation. * Drive quality, cost, and latency trade-off decisions with data -- selecting models, routing strategies, and inference configurations that meet product ...
Engineering Manager, Evals
New York, NY · On-site
The team also owns online evaluation systems that track agent quality in production, and the close integration between online and offline evaluations. The evaluation systems that this team builds ...
Engineering Manager, Evals
New York, NY · On-site
The team also owns online evaluation systems that track agent quality in production, and the close integration between online and offline evaluations. The evaluation systems that this team builds ...
Staff Software Engineer AI/ML
San Jose, CA · On-site
$163 - $253/hr
... online evaluation. * Drive quality, cost, and latency trade‑off decisions with data -- selecting models, routing strategies, and inference configurations that meet product requirements within ...
New
Staff Software Engineer AI/ML
San Jose, CA · On-site
$163 - $253/hr
... online evaluation. * Drive quality, cost, and latency trade‑off decisions with data -- selecting models, routing strategies, and inference configurations that meet product requirements within ...
New
Machine Learning Scientist III - Whole Trip AI
California, MO · On-site
$110 - $150/hr
Use A/B tests and offline/online evaluation frameworks to measure model impact and guide iterative improvement Requirements * Bachelor's degree in Computer Science or a related technical field; or ...
New
Machine Learning Scientist III - Whole Trip AI
California, MO · On-site
$110 - $150/hr
Use A/B tests and offline/online evaluation frameworks to measure model impact and guide iterative improvement Requirements * Bachelor's degree in Computer Science or a related technical field; or ...
New
German Bilingual Lyric Reviewer
Charleston, WV · Remote
$17 - $18/hr
Experience with online evaluation tools or platforms.
Quick apply
German Bilingual Lyric Reviewer
Charleston, WV · Remote
$17 - $18/hr
Experience with online evaluation tools or platforms.
Build and run programmatic evaluations for agents: metrics, offline/online evals, and failure-mode analysis. * Apply context engineering practices: context assembly, memory/state management, and ...
Build and run programmatic evaluations for agents: metrics, offline/online evals, and failure-mode analysis. * Apply context engineering practices: context assembly, memory/state management, and ...
Own evaluation end to end: define what "good" means, build offline and online evaluation harnesses, and detect quality regressions in production. * Build the data and MLOps foundations that keep ...
Own evaluation end to end: define what "good" means, build offline and online evaluation harnesses, and detect quality regressions in production. * Build the data and MLOps foundations that keep ...
AI Platform Engineer
Dallas, TX · On-site
Knowledge of evaluation & safety practices: PII/PHI masking, audit trails, feedback-loop instrumentation, and offline + online evaluation. * Strong programming skills in Python 3.11+, FastAPI, async ...
AI Platform Engineer
Dallas, TX · On-site
Knowledge of evaluation & safety practices: PII/PHI masking, audit trails, feedback-loop instrumentation, and offline + online evaluation. * Strong programming skills in Python 3.11+, FastAPI, async ...
... online evaluation plans for AI-enabled products Preferred Profile Familiarity with legal, policy, regulatory, or business-information datasets Relevant bachelor's degree required; master's-level ...
... online evaluation plans for AI-enabled products Preferred Profile Familiarity with legal, policy, regulatory, or business-information datasets Relevant bachelor's degree required; master's-level ...
Online Evaluation information
See salary details
$17.5K - $23.7K
12% of jobs
$28.2K is the 25th percentile. Wages below this are outliers.
$23.7K - $30K
18% of jobs
$30K - $36.2K
15% of jobs
The median wage is $37.1K / yr.
$36.2K - $42.4K
33% of jobs
$42.4K - $48.6K
12% of jobs
$48.6K - $54.9K
0% of jobs
$54.9K - $61.1K
0% of jobs
$61.1K - $67.3K
0% of jobs
$67.3K - $73.5K
0% of jobs
$73.5K - $79.8K
0% of jobs
$79.8K - $86K
9% of jobs
$17.5K
$40.6K
$86K
How much do online evaluation jobs pay per year?
As of Aug 21, 2026, the average yearly pay for online evaluation in the United States is $40,596.00, according to ZipRecruiter salary data. Most workers in this role earn between $25,000.00 and $43,500.00 per year, depending on experience, location, and employer.
What is online evaluation?
Online evaluation refers to the process of assessing digital content, services, or systems through remote, internet-based methods. This often involves tasks like rating search engine results, reviewing website usability, or analyzing the performance of online ads. Online evaluators typically work from home and follow specific guidelines to provide feedback that helps improve the quality and relevance of online platforms. The job requires attention to detail, strong analytical skills, and a good understanding of internet navigation.
What are the key skills and qualifications needed to thrive as an online evaluator, and why are they important?
To thrive as an Online Evaluator, you need strong analytical skills, attention to detail, and proficiency in written communication, typically supported by a high school diploma or higher education. Familiarity with online search engines, evaluation platforms, and quality assurance tools is commonly required, and some roles may seek specific certifications in digital literacy. Excellent time management, critical thinking, and the ability to work independently are valuable soft skills in this position. These abilities ensure accurate, unbiased assessments and help maintain the quality and reliability of online content or digital products.
What are some common challenges faced by online evaluators, and how can they effectively overcome them?
Online Evaluators often encounter challenges such as ambiguous guidelines, repetitive tasks, and maintaining objectivity while assessing digital content. To overcome these, it's helpful to regularly review updated instructions, participate in available training sessions, and actively communicate with team leads or peers when clarification is needed. Staying organized and taking short breaks can also help maintain focus and accuracy throughout their assignments.
What is the difference between Online Evaluation vs Quality Assurance Specialist?
| Aspect | Online Evaluation | Quality Assurance Specialist |
|---|---|---|
| Credentials | Typically requires a degree in education, psychology, or related fields | Requires a degree in quality management, engineering, or related fields |
| Work Environment | Primarily remote, online platforms, educational or testing organizations | Office settings, manufacturing, or software development environments |
| Industry Usage | Used in education, testing, and e-learning sectors | Common in manufacturing, software, and product development industries |
| Job Focus | Assessing and scoring online tests or educational content | Ensuring product quality and process improvements |
Online Evaluation and Quality Assurance Specialist roles share a focus on assessment and quality, but differ in industry, credentials, and work environment. Online Evaluation is centered on educational and testing contexts, often remote, while Quality Assurance Specialists work across industries to improve product quality in more physical or technical settings.
How to become a remote online evaluation?
To become a remote online evaluator, typically you need strong reading and comprehension skills, attention to detail, and familiarity with online tools. Many positions require completing an application, passing a skills assessment, and having a reliable internet connection. Some roles may also require prior experience or specific training in evaluation or quality assurance.
More about Online Evaluation jobs
What cities are hiring for Online Evaluation jobs?
Cities with the most Online Evaluation job openings:
What are the most commonly searched types of Evaluation jobs?
The most popular types of Evaluation jobs are:
What states have the most Online Evaluation jobs?
States with the most job openings for Online Evaluation jobs include:
What job categories do people searching Online Evaluation jobs look for?
The top searched job categories for Online Evaluation jobs are:
$110 - $165/hr
Other
Posted 3 days ago
New
Job description
Responsibilities
#J-18808-Ljbffr- Design and implement end-to-end model pipelines to production across multiple product domains, including ranking, recommendations, search, and personalization
- Develop and maintain scalable data pipelines, data quality checks, and model monitoring to ensure reliability, performance, and responsible behavior of ML systems in production
- Collaborate with cross-functional partners (product, analytics, engineering) to translate ambiguous business needs into well-scoped ML projects, communicate findings, and influence decision making with data-driven insights
- Use A/B tests and offline/online evaluation frameworks to measure model impact and guide iterative improvement
- Bachelor’s degree in Computer Science or a related technical field; or Equivalent related professional experience
- 2+ years of relevant professional experience
- Professional industry experience applying machine learning or statistical modeling to real business problems, including end-to-end model development from data exploration through evaluation and deployment
- Proficiency in Python and with ML frameworks and libraries for model development, training, and evaluation
- Demonstrated ability to translate problem statements into well-defined ML tasks, design appropriate model and data structures (including APIs and data models), and own solutions within a defined product, service, or feature area, including familiarity with AI-driven systems, tools, or workflows and applying AI/ML concepts to real world products with attention to safety and reliability.