Rather than relying solely on human preference data, we can ground reinforcement learning in the ... Drive human-in-the-loop evaluation with high annotation quality and sound scientific methodology
Rather than relying solely on human preference data, we can ground reinforcement learning in the ... Drive human-in-the-loop evaluation with high annotation quality and sound scientific methodology
Senior Applied Scientist, Search & Information Retrieval
Toronto, ON · Hybrid
Medical
Dental
Vision
Life
Retirement
PTO
Build evaluation frameworks - component-level and end-to-end - using expert annotation and synthetic data generation * Drive independent technical decisions on retrieval architecture, indexing ...
Senior Applied Scientist, Search & Information Retrieval
Toronto, ON · Hybrid
Medical
Dental
Vision
Life
Retirement
PTO
Build evaluation frameworks - component-level and end-to-end - using expert annotation and synthetic data generation * Drive independent technical decisions on retrieval architecture, indexing ...
Data Annotation information
What does a data annotation do?
A typical workday as a Data Annotator involves reviewing datasets—such as images, audio, text, or video—and accurately labeling or categorizing information according to specific project guidelines. Most Data Annotators work independently, but they often collaborate with project managers or data scientists to clarify requirements and resolve ambiguities. Tasks may be repetitive, but adhering to precise standards is vital for maintaining data quality. Work environments can range from technology companies to remote or freelance settings, and advancement opportunities exist as team leads or quality assurance specialists for those who excel in consistency and reliability.
How much money can I make doing data annotation?
What is a data annotation?
A Data Annotation job involves labeling and categorizing data, such as text, images, audio, or video, to help train machine learning models. Annotators apply tags, bounding boxes, or classifications to data based on specific guidelines. This process improves the accuracy of AI systems in recognizing patterns and making predictions. Many data annotation jobs require attention to detail and familiarity with specific domains. It is commonly used in applications like autonomous driving, natural language processing, and computer vision.
What are the key skills and qualifications needed to thrive in data annotation?
To thrive in Data Annotation, you need strong attention to detail, accuracy, and basic data handling skills, often supported by a high school diploma or equivalent. Familiarity with annotation platforms, data labeling software, or content management systems is frequently required, though specific certifications are rare. Excellent communication, time management, and the ability to focus on repetitive tasks distinguish top performers in this role. These skills are crucial because accurate and consistent data annotation directly impacts the quality of machine learning models and AI applications.

Full-time
Re-posted 17 days ago
Autodesk rating
9.1
Based on 8 frontline employees who took The Breakroom Quiz
25th of 244 rated software companies
Job description
Job Requisition ID #
Position Overview
Autodesk's domains - architecture, engineering, construction, manufacturing, media & entertainment - provide a distinctive research environment: rich structured data, long-horizon reasoning tasks, and real-world evaluation grounded in professional workflows. Uniquely, decades of investment in physics simulation engines, CAD kernels, and computational design tools give us something most labs don't have: high-fidelity, domain-grounded verifiers that can serve as reward signals for post-training. Rather than relying solely on human preference data, we can ground reinforcement learning in the laws of physics and the constraints of real engineering. These are exactly the kinds of challenges - and assets - that make post-training and alignment research here genuinely distinctive.
We publish at NeurIPS, ICML, ICLR, CVPR, and SIGGRAPH. We collaborate with leading academic and industry labs. And we have a direct line from research advances to product impact at scale. This is not a role where research sits behind a wall from engineering - you will see your work matter.
Responsibilities
Post-training for model development - from RLHF and preference optimization to agentic systems and long-horizon reasoning
Develop novel algorithms that improve model reliability, controllability, and alignment
Make principled architectural decisions about when to address challenges at the pre-training, post-training, or system level
Design and run experiments that shape model behavior, robustness, and reasoning quality
Partner with infrastructure teams to build scalable, reproducible post-training workflows
Contribute to publications, patents, and Autodesk's external research visibility
Design evaluation frameworks for long-horizon reasoning, tool use, agentic behavior, safety, and real-world workflow completion
Lead rigorous model analysis and interpretability efforts
Drive human-in-the-loop evaluation with high annotation quality and sound scientific methodology
Establish model readiness criteria and provide go/no-go recommendations for releases
Communicate technical risks, limitations, and trade-offs clearly to leadership
Minimum Requirements
Deep hands-on expertise in reinforcement learning for foundation models, and fluency with post-training methods (RLHF, RLAIF, DPO, PPO, or adjacent approaches)
Proven experience leading or mentoring technical research teams - whether in an academic lab, AI research organization, or industry setting
Strong intuition for model behavior, alignment challenges, and post-training trade-offs
Experience designing evaluation systems and thinking rigorously about what it means for a model to be ready
Ability to communicate complex technical trade-offs clearly to both technical and non-technical audiences
A PhD or equivalent depth of industry research experience in ML, RL, AI, or a related field
Experience at a frontier model lab or advanced applied AI organization
A strong publication record at leading ML or AI venues
Background in alignment research, preference learning, or agentic AI
Experience deploying or supporting production AI systems
Familiarity with large-scale training infrastructure and compute trade-offs
Learn More
About Autodesk
Welcome to Autodesk! Amazing things are created every day with our software - from the greenest buildings and cleanest cars to the smartest factories and biggest hit movies. We help innovators turn their ideas into reality, transforming not only how things are made, but what can be made.
We take great pride in our culture here at Autodesk - it's at the core of everything we do. Our culture guides the way we work and treat each other, informs how we connect with customers and partners, and defines how we show up in the world.
When you're an Autodesker, you can do meaningful work that helps build a better world designed and made for all. Ready to shape the world and your future? Join us!
Salary transparency
Salary is one part of Autodesk's competitive compensation package. For Canada based roles, we expect a starting base salary between $0 and $0. Offers are based on the candidate's experience and geographic location, and may exceed this range. In addition to base salaries, our compensation package may include annual cash bonuses, commissions for sales roles, stock grants, and a comprehensive benefits package.Diversity & Belonging
We take pride in cultivating a culture of belonging where everyone can thrive. Learn more here: https://www.autodesk.com/company/diversity-and-belonging
Are you an existing contractor or consultant with Autodesk?
Please search for open jobs and apply internally (not on this external site).
About Autodesk
Sourced by ZipRecruiter
Autodesk is changing how the world is designed and made. Our technology spans architecture, engineering, construction, product design, manufacturing, media, and entertainment, empowering innovators everywhere to solve challenges big and small. From greener buildings to smarter products to more mesmerizing blockbusters, Autodesk software helps our customers to design and make a better world for all. For more information visit autodesk.com or follow @autodesk.
Industry
Software development
Company size
10,000+ Employees
Headquarters location
San Rafael, CA, US
Year founded
1982