Annotation Judge Jobs in Connecticut (NOW HIRING)

Job Description
Location: Hybrid in Westbrook, CT or Remote - EST
Job Description:
Responsibilities:
Your primary focus:
Predicate & invariant framework for data contracts - the core of the role.
Design and implement declarative contract classes that attach to Python methods (design-by-contract decorators - no relation to the ML data annotations below) and trigger verification of the code inside, using AST-level analysis.
Predicates enforce data contracts: they state what a method must guarantee about the data it produces or consumes, and the verifier checks the implementation against those statements.
Invariants constrain evolution: they state properties of the codebase that must survive change, so that modifications - human- or AI-authored - that would break them fail at verification time, not in production.
You'll shape the vocabulary of predicates and invariants together with the architect, build the verifier and its diagnostics, and make violation messages clear enough that they teach the contract they enforce.
Your secondary focus:
Annotation data platform evolution.
Extend a shipped canonical schema (Avro) and adapter layer that normalize ML annotation data from multiple commercial labeling platforms into a shared representation.
Add adapters for new platforms, evolve the schema under a versioned spec and ADR process, and keep validation utilities and Python typing overlays in sync with the schema.
Design and implement the predicate/invariant framework: contract classes, the AST-based verifier, and CI integration.
Turn abstract contract concepts into APIs and diagnostics that working engineers adopt willingly - making the ideas graspable is part of the job, not an afterthought.
Extend and evolve schemas, adapters, and validation layers for the annotation platform under its established change process.
Investigate verification and validation failures and determine whether the fix belongs in the contract, the code, or the source system, documenting your reasoning.
Document the framework thoroughly and transfer knowledge continuously - by the end of the engagement, the team must be able to own and extend it without you.
Work closely with a senior architect on initial designs, then independently own implementation in your areas.
Qualifications:
We're flexible on background, but you should be able to demonstrate:
Comfort with formal and abstract structures - logic, type systems, program analysis, algebraic thinking - demonstrated by working software you built from them. Vision and execution together; neither alone is enough.
Deep production Python: decorators, descriptors, metaclasses, type hints, and the standard library.
Strong analytical reasoning: comfort working from ambiguous or underspecified ideas and finding structure.
Ability to communicate technical ideas clearly in writing (design docs, code reviews, documentation, async messaging).
Independence in scoping and delivering work, with the judgment to escalate complex design questions.
Bonus Qualifications:
A computer-science degree, or any particular number of years of experience.
Prior data engineering or ML experience (the role is adjacent to ML, not part of model training).
Experience with our exact stack (Avro, Databricks, Spark, dbt, etc. can be learned on the job).
Experience in any of these areas is a genuine plus:
Contracts and verification
Design-by-contract tooling (icontract, deal, Eiffel, JML, Dafny) or other program-verification exposure.
Property-based testing (Hypothesis or similar).
Code-as-data work
Parsing or analyzing source code (Python ast / libcst, tree-sitter, or equivalents); codemods; mypy plugins or typing internals.
Code generation, templating, or compiler back-ends - especially if you've maintained a code generator in production.
Rule and constraint systems
DSLs, OPA/Rego, rule engines, or knowledge-representation/constraint languages (OWL, RDF, SHACL, Datalog).
Translating declarative business rules into executable validation logic.
Schema and validation tooling
Avro, JSON Schema, OpenAPI/Swagger, LinkML, CUE, or similar; Pydantic, Marshmallow, or attrs with validators.
What success looks like:
In your first 30 days, you'll internalize the contract model and the platform's spec/ADR process, and ship a first working predicate end-to-end - decorator, verification, diagnostics.
By 90 days, the framework core will be enforcing real data contracts in CI on at least one system, and teammates will be writing predicates without your help.
By end of term, the framework will be documented, adopted, and owned by the team; invariants will be guarding codebase evolution; and the extension conversation will be about what to build next, not whether it worked.