Contract Data Annotation Analyst Jobs in Chicago, IL

Who We Are

Arrive Logistics is a leading transportation and technology company in North America with plans to grow significantly year over year. Our success is a testament to our remarkable team and what we’re building together. We’re committed to providing employees with a meaningful work experience and have established an award-winning culture that supports personal and career development in a fun, casual, and collaborative environment.

Who We Want

The Data Scientist II will work closely with Data Science, Product, and Engineering to build and improve ML and AI systems that drive operational value. This role is a great fit for a hands-on practitioner with applied experience in NLP and LLM-based systems who is ready to take on meaningful technical ownership. You'll contribute to the full lifecycle of production ML systems — from evaluation and measurement through development, deployment, and iteration — with a particular focus on text and language-based applications. The ideal candidate is comfortable operating in ambiguous problem spaces, can translate loosely defined business needs into concrete technical approaches, and communicates findings clearly to both technical and non-technical audiences.

• Develop, evaluate, and iterate on NLP and LLM-based systems, including text classification, information extraction, and context retrieval pipelines.
• Build measurement and evaluation frameworks — both offline and online — to assess where and why systems are underperforming and quantify the impact of improvements.
• Develop golden test datasets and define methodologies for creating and maintaining them over time, including designing annotation guidelines and ensuring label quality.
• Evaluate and apply the appropriate approach for language tasks — whether prompt engineering, fine-tuning, or classical NLP methods — including modern retrieval and RAG architectures and LLM evaluation methodologies, based on the problem and available data.
• Perform structured analysis of system performance to surface failure modes, data gaps, and high-value areas for investment, applying sound statistical reasoning to evaluation results.
• Partner with engineers to support deployment, integration, and monitoring of ML and AI systems in production.
• Contribute to standards and best practices around deploying, evaluating, and monitoring text and language-based ML systems.
• Document work clearly and maintain knowledge artifacts that make systems understandable and maintainable over time.
• Collaborate with senior data scientists and cross-functional partners to translate business needs into well-scoped technical solutions, including communicating findings and recommendations to non-technical stakeholders.

• Bachelor's or Master's degree in a quantitative field (computer science, statistics, linguistics, or related) and 2–4 years of applied ML or data science experience, or equivalent practical experience.
• Hands-on experience building or improving NLP or LLM-based systems in applied settings.
• Familiarity with text classification, information extraction, or other NLP tasks — and an understanding of where these systems fail.
• Experience with both prompt engineering and fine-tuning approaches for language tasks, with the judgment to know when to apply each.
• Familiarity with modern retrieval strategies and RAG architectures and how they affect LLM system performance.
• Experience with Hugging Face Transformers for text classification or related NLP tasks.
• Experience contributing to evaluation frameworks, test sets, or performance diagnostics for ML systems, including comfort with statistical methods for measuring model performance.
• Proficiency in Python and SQL, and comfort working with structured and unstructured data.
• Ability to operate effectively in ambiguous problem spaces — scoping technical approaches when requirements are not fully defined.
• Strong written communication skills; able to document systems and findings clearly and present recommendations to non-technical stakeholders.
• Experience designing data annotation workflows, labeling guidelines, or label quality processes is a plus.
• Experience with model deployment, monitoring, or production ML workflows is a plus.
• Familiarity with LangChain and LangSmith or similar LLM orchestration and observability tooling is a plus.
• Transportation or logistics industry experience is a plus.

• Take advantage of our comprehensive benefits package, including medical, dental, vision, life, disability, and supplemental coverage.
• Invest in your future with our matching 401(k) program.
• Build relationships and take part in learning opportunities through our Employee Resource Groups.
• Enjoy office wide engagement activities, team events, happy hours and more!
• Leave the suit and tie at home; our dress code is casual.
• Work in the heart of downtown Chicago, IL!
• Sweat it out at the LifeStart gym in our office building that includes brand new Peloton bikes, top-of-the-line equipment and personal training options.
• Maximize your wellness with free counseling sessions through our Employee Assistance Program
• Take time to manage your physical and mental health - we offer company paid holidays, paid vacation time and wellness days.
• Receive 100% paid parental leave when you become a new parent.
• Get paid to work with your friends through our Referral Program!
• Get relocation assistance! If you are not local to the area, we offer relocation packages.