Columbus, OH · On-site +1
$80 - $100/hr
What You'll Be Doing Design and build the coding benchmarks and evaluation pipelines used to test frontier AI models on real software engineering work: * Design coding benchmarks that evaluate ...
Columbus, OH · On-site +1
$80 - $100/hr
What You'll Be Doing Design and build the coding benchmarks and evaluation pipelines used to test frontier AI models on real software engineering work: * Design coding benchmarks that evaluate ...
Columbus, OH · On-site +1
$30 - $70/hr
What You'll Need * 3+ years of professional software engineering experience in TypeScript (constraint programming experience is a bonus, but not required) * Strong code-review instincts - you can ...
Columbus, OH · On-site +1
$30 - $70/hr
What You'll Need * 3+ years of professional software engineering experience in TypeScript (constraint programming experience is a bonus, but not required) * Strong code-review instincts - you can ...
Columbus, OH · On-site +1
$30 - $70/hr
What You'll Need * 3+ years of professional software engineering experience in Go (constraint programming experience is a bonus, but not required) * Strong code-review instincts - you can spot logic ...
Columbus, OH · On-site +1
$30 - $70/hr
What You'll Need * 3+ years of professional software engineering experience in Go (constraint programming experience is a bonus, but not required) * Strong code-review instincts - you can spot logic ...
Columbus, OH · On-site +1
$30 - $70/hr
What You'll Need * 3+ years of professional software engineering experience in Ruby (constraint programming experience is a bonus, but not required) * Strong code-review instincts - you can spot ...
Columbus, OH · On-site +1
$30 - $70/hr
What You'll Need * 3+ years of professional software engineering experience in Ruby (constraint programming experience is a bonus, but not required) * Strong code-review instincts - you can spot ...
Columbus, OH · On-site +1
$30 - $70/hr
What You'll Need * 3+ years of professional software engineering experience in Go (constraint programming experience is a bonus, but not required) * Strong code-review instincts - you can spot logic ...
Columbus, OH · On-site +1
$30 - $70/hr
What You'll Need * 3+ years of professional software engineering experience in Go (constraint programming experience is a bonus, but not required) * Strong code-review instincts - you can spot logic ...
Columbus, OH · On-site +1
$30 - $70/hr
What You'll Need * 3+ years of professional software engineering experience in Rust (constraint programming experience is a bonus, but not required) * Strong code-review instincts - you can spot ...
Columbus, OH · On-site +1
$30 - $70/hr
What You'll Need * 3+ years of professional software engineering experience in Rust (constraint programming experience is a bonus, but not required) * Strong code-review instincts - you can spot ...
Columbus, OH · On-site +1
$30 - $70/hr
What You'll Need * 3+ years of professional software engineering experience in Rust (constraint programming experience is a bonus, but not required) * Strong code-review instincts - you can spot ...
Columbus, OH · On-site +1
$30 - $70/hr
What You'll Need * 3+ years of professional software engineering experience in Rust (constraint programming experience is a bonus, but not required) * Strong code-review instincts - you can spot ...
Columbus, OH · On-site +1
$30 - $70/hr
What You'll Need * 3+ years of professional software engineering experience in Java (constraint programming experience is a bonus, but not required) * Strong code-review instincts - you can spot ...
Columbus, OH · On-site +1
$30 - $70/hr
What You'll Need * 3+ years of professional software engineering experience in Java (constraint programming experience is a bonus, but not required) * Strong code-review instincts - you can spot ...
Columbus, OH · On-site +1
$30 - $70/hr
What You'll Need * 3+ years of professional software engineering experience in C++ (constraint programming experience is a bonus, but not required) * Strong code-review instincts - you can spot logic ...
Columbus, OH · On-site +1
$30 - $70/hr
What You'll Need * 3+ years of professional software engineering experience in C++ (constraint programming experience is a bonus, but not required) * Strong code-review instincts - you can spot logic ...
Columbus, OH · On-site +1
$30 - $70/hr
What You'll Need * 3+ years of professional software engineering experience in Java (constraint programming experience is a bonus, but not required) * Strong code-review instincts - you can spot ...
Columbus, OH · On-site +1
$30 - $70/hr
What You'll Need * 3+ years of professional software engineering experience in Java (constraint programming experience is a bonus, but not required) * Strong code-review instincts - you can spot ...
Columbus, OH · On-site +1
$30 - $70/hr
What You'll Need * 3+ years of professional software engineering experience in TypeScript (constraint programming experience is a bonus, but not required) * Strong code-review instincts - you can ...
Columbus, OH · On-site +1
$30 - $70/hr
What You'll Need * 3+ years of professional software engineering experience in TypeScript (constraint programming experience is a bonus, but not required) * Strong code-review instincts - you can ...
Columbus, OH · On-site +1
$30 - $70/hr
What You'll Need * 3+ years of professional software engineering experience in Python (constraint programming experience is a bonus, but not required) * Strong code-review instincts - you can spot ...
Columbus, OH · On-site +1
$30 - $70/hr
What You'll Need * 3+ years of professional software engineering experience in Python (constraint programming experience is a bonus, but not required) * Strong code-review instincts - you can spot ...
Columbus, OH · On-site +1
$30 - $70/hr
What You'll Need * 3+ years of professional software engineering experience in Python (constraint programming experience is a bonus, but not required) * Strong code-review instincts - you can spot ...
Columbus, OH · On-site +1
$30 - $70/hr
What You'll Need * 3+ years of professional software engineering experience in Python (constraint programming experience is a bonus, but not required) * Strong code-review instincts - you can spot ...
San Francisco, CA · On-site
$180K - $220K/yr
Founding Mobile Engineer (React Native / iOS) Onsite - San Francisco (Pac Heights) • Full-time • $180K-$220K + Equity About the Opportunity G2i is partnering with a fast-growing, early-stage ...
San Francisco, CA · On-site
$180K - $220K/yr
Founding Mobile Engineer (React Native / iOS) Onsite - San Francisco (Pac Heights) • Full-time • $180K-$220K + Equity About the Opportunity G2i is partnering with a fast-growing, early-stage ...
Richmond, VA · Remote
$70/hr
G2i Inc. Location: Richmond, VA Remote Pay: $70/hr Type: Full-time Posted: 17 hours ago Software Engineer, AI -- Code Evaluation & Training (Remote) Help train large-language models (LLMs) to write ...
Richmond, VA · Remote
$70/hr
G2i Inc. Location: Richmond, VA Remote Pay: $70/hr Type: Full-time Posted: 17 hours ago Software Engineer, AI -- Code Evaluation & Training (Remote) Help train large-language models (LLMs) to write ...
Manhattan, NY · Hybrid
$116.90K - $158.70K/yr
Build systems that improve developer experience and data accessibility across teams * Collaborate ... G2i Technical Screening * Background & Technical Deep Dive * System Design Interview * Onsite ...
Manhattan, NY · Hybrid
$116.90K - $158.70K/yr
Build systems that improve developer experience and data accessibility across teams * Collaborate ... G2i Technical Screening * Background & Technical Deep Dive * System Design Interview * Onsite ...
Miami, FL · On-site
$80 - $100/hr
What You'll Be Doing Design and build the coding benchmarks and evaluation pipelines used to test frontier AI models on real software engineering work: * Design coding benchmarks that evaluate ...
Miami, FL · On-site
$80 - $100/hr
What You'll Be Doing Design and build the coding benchmarks and evaluation pipelines used to test frontier AI models on real software engineering work: * Design coding benchmarks that evaluate ...
Miami, FL · On-site
$30 - $70/hr
What You'll Need * 3+ years of professional software engineering experience in TypeScript (constraint programming experience is a bonus, but not required) * Strong code-review instincts - you can ...
Miami, FL · On-site
$30 - $70/hr
What You'll Need * 3+ years of professional software engineering experience in TypeScript (constraint programming experience is a bonus, but not required) * Strong code-review instincts - you can ...
Miami, FL · On-site
$30 - $70/hr
What You'll Need * 3+ years of professional software engineering experience in Rust (constraint programming experience is a bonus, but not required) * Strong code-review instincts - you can spot ...
Miami, FL · On-site
$30 - $70/hr
What You'll Need * 3+ years of professional software engineering experience in Rust (constraint programming experience is a bonus, but not required) * Strong code-review instincts - you can spot ...
Miami, FL · On-site
$30 - $70/hr
What You'll Need * 3+ years of professional software engineering experience in Rust (constraint programming experience is a bonus, but not required) * Strong code-review instincts - you can spot ...
Miami, FL · On-site
$30 - $70/hr
What You'll Need * 3+ years of professional software engineering experience in Rust (constraint programming experience is a bonus, but not required) * Strong code-review instincts - you can spot ...
What are the key skills and qualifications needed to thrive as a Software Developer, and why are they important?
What are some common challenges software developers face when working on team-based projects?
What is a Software Developer?
What is the difference between Software Developer vs Software Engineer?
| Aspect | Software Developer | Software Engineer |
|---|---|---|
| Credentials | Bachelor's degree in CS or related field; coding certifications | Bachelor's or higher in CS or Software Engineering; certifications may vary |
| Work Environment | Develops, tests, and maintains software applications | Designs and develops software systems; often involved in architecture |
| Industry Usage | Commonly used in tech companies, startups, and IT firms | Used interchangeably with Software Developer in many organizations |
While the roles often overlap, Software Developers focus on coding and building applications, whereas Software Engineers typically work on designing and architecting complex systems. Both roles require similar credentials and are integral to software creation, but their scope and responsibilities can differ based on project size and company structure.
Columbus, OH • On-site, Remote
$80 - $100/hr
Contractor
Posted 14 days ago
This role is open to contractors in accepted locations only. Please confirm your country is on the list before applying - we're unable to process applications from unlisted locations. List of accepted countries and locations.
For US applicants: This is a 1099 independent contractor role. It is not compatible with F-1 OPT, STEM OPT, or any visa status that requires W-2 employment, guaranteed hours, or employer sponsorship. We are unable to provide offer letters or employment verification for this role.
What You'll Be DoingDesign and build the coding benchmarks and evaluation pipelines used to test frontier AI models on real software engineering work:
Design coding benchmarks that evaluate frontier models on real-world programming tasks - reasoning, debugging, and production-quality code
Build and maintain scalable data pipelines for evaluation workflows
Analyze model-generated code for correctness, reliability, and edge-case failures
Construct structured evaluation scenarios across large repos and multi-language environments
Provide detailed technical feedback on model performance and failure patterns
Contribute to evaluation frameworks that set the bar for how coding ability is measured
End result: benchmarks that meaningfully separate what frontier models can and can't do - and shape how the next generation is trained and improved.
AI coding evaluation in one line: Design task build harness run model analyze failures feed findings back into the benchmark evaluations that actually distinguish strong models from weak ones.
What You'll Need4+ years of professional software engineering experience (non-negotiable)
Expert Python - clean, performant, well-tested code
Hands-on experience working in large, complex codebases
Proven experience designing and implementing LLM coding benchmarks and evaluation data pipelines
Strong command of Git and modern development workflows
Track record at a high-growth tech company or top-tier software organization
Strong written English communication
Identity verification: Applicants will be required to verify their identity and confirm they have valid documentation to work as an independent contractor in their country of residence.
Nice to haveSenior or Lead-level profile with a history of technical ownership
Bachelor's or Master's in CS, ML, or related field (or equivalent professional experience)
Proficiency in additional languages: JavaScript, Go, C++, or others
CI/CD experience and writing robust unit tests (pytest, Mocha, JUnit)
Background in security engineering or significant open-source contributions
Familiarity with AI/ML evaluation methodologies or model benchmarking
Location: Fully remote - work from anywhere on the accepted locations list
Compensation: $80-$100/hr based on location and seniority
Contract length: 3 months, with potential for extension
Hours: Full-time availability preferred - hours vary by project and are not guaranteed week to week
Engagement: 1099 independent contractor
Payment: Weekly via PayPal or Stripe
Important: Hours are project-dependent and can vary week to week. We recommend keeping other work options open alongside this engagement rather than relying on it as your sole source of income.
Sourced by ZipRecruiter
Software development
11 - 50 Employees
Delray Beach, FL, US
2012