Extremism Jobs in California (NOW HIRING)

Reflection is a research lab making intelligence open and accessible for everyone to use, customize, and build on. We build open models that let anyone control their intelligence and help shape the future of AI. Our mission: make intelligence open and accessible to all.

As a Subject Matter Expert, Election Integrity, Violent Extremism, & Fraud, you are our internal authority on preventing, detecting, and responding to election-related manipulation (e.g., coordinated influence operations, voter suppression content, fabricated or misleading claims about voting), terrorist and violent extremist content (TVEC), and fraud and deception (e.g., scams, phishing, impersonation, financial fraud, and other deceptive schemes) generated or facilitated through Reflection's models and products. You will define how the lab thinks about these harms, translate that expertise into concrete enforcement workflows and detection strategies, and act as a critical gatekeeper ensuring our open releases meet our commitments in these areas.

You will own election integrity, violent extremism, and fraud evaluations, human grading, auto-grading, and enforcement end to end: standing up and scaling review workflows, serving as the primary point of contact for those who conduct content review, as well as conducting them yourself, driving model launch evals on novel or ambiguous cases, and partnering closely with our technical Safety team, Alignment, Policy, and Legal teams to close gaps as the threat landscape evolves. You will also represent Reflection's work in these areas externally with reporting bodies, NGOs, researchers, election officials, financial-sector and anti-fraud partners, and, where appropriate, law enforcement.

In this position you will regularly be exposed to and engage with content that may be violent, graphic, hateful, or psychologically disturbing, including terrorist and violent extremist material and depictions of real-world violence. You may also review deceptive and fraudulent material, including content designed to defraud or harm vulnerable people. Reflection takes the wellbeing of team members working in this area seriously. We ask candidates to consider this aspect of the role carefully before applying.

• Serve as Reflection's internal subject matter expert on election integrity, coordinated influence operations, terrorism and violent extremism, and fraud and deception, and set the standards, definitions, and escalation thresholds the lab operates against.

• Partner with Safety research,Safety postraining, and Safety Evals teams to create new automated evaluations.

• Own the day-to-day operation of election integrity, violent extremism, and fraud content evaluation and review workflows, including task routing, queue management, quality assurance, escalation handling, and SLA monitoring.

• Serve as the primary point of contact for review partners conducting this content review, including onboarding, training, calibration, and ongoing support.

• Review novel or ambiguous flagged content to drive enforcement decisions, and surface policy and product gaps to the Alignment, Safety, and Policy teams.

• Partner with engineering to design, evaluate, and improve detection models, hash-matching pipelines (e.g., known-terrorist-content databases), fraud and abuse detection systems, and automated enforcement systems, and to build these considerations into our pre-release evaluation and red-teaming process.

• Validate that models meet Reflection's election integrity, violent extremism, and fraud risk thresholds before open weights ship.

• Develop and maintain internal documentation, decision trees, and review guidelines that enable accurate, consistent enforcement at scale.

• Identify and report emerging misuse patterns and threat actor behavior to internal stakeholders across the company.

• Coordinate reporting and information-sharing obligations to relevant external bodies (e.g., GIFCT, election authorities, anti-fraud and financial-sector partners) in accordance with applicable law and Reflection policy.

• Stay current on evolving legal frameworks, enforcement best practices, and threat-detection technology, and use these to keep our workflows ahead of the threat landscape, including through major election cycles worldwide and fast-moving fraud and scam trends.

• Deep subject matter expertise in one or more of: election integrity, disinformation and influence operations, counterterrorism and violent extremism, or fraud and deception (e.g., scams, phishing, impersonation, financial fraud).

• Experience in trust & safety, content moderation operations, or policy enforcement with direct exposure to election-related harms, terrorist or violent extremist content, fraud, or related harm areas.

• Experience managing or coordinating content review operations, including quality assurance and workflow management.

• Experience standing up and scaling policy enforcement or content review workflows.

• Proficiency in SQL and/or other data analysis tools to monitor workflow health, review-queue metrics, and enforcement trends.

• Track record of identifying emerging risks and communicating findings clearly to cross-functional stakeholders across Product, Policy, Engineering, and Legal.

• Sound judgment under time pressure and comfort making high-stakes decisions, including ones that affect whether a model ships.

• Familiarity with content classification frameworks for terrorist and violent extremist material, with influence-operation taxonomies (e.g., ABCDE / DISARM frameworks), and with common fraud and scam typologies.

• Familiarity with relevant legal and regulatory frameworks, including the EU Digital Services Act, the EU TCO Regulation, election-related transparency laws, consumer-protection and anti-fraud regulations, or equivalent international frameworks.

• Experience with generative AI products and an understanding of how AI systems can be misused to generate or facilitate influence operations, violent extremism, or fraud.

• Experience with hash-matching or perceptual hashing technologies used in detecting known violent extremist content.

• Familiarity with provenance and content-authenticity technologies (e.g., C2PA) and their role in election integrity and anti-fraud efforts.

• Experience designing or evaluating trauma-informed support structures and wellness protocols for content reviewers.

• Proficiency in Python for workflow automation or data analysis.

• Additional language proficiency beyond English.

We encourage you to apply even if you don't meet every qualification. Not all strong candidates will meet every item listed, and we'd rather hear from you than have you exclude yourself prematurely.

We believe that to make intelligence open and accessible to all, you need to start at the foundation. Joining Reflection means building from the ground up as part of a talent-dense team. You will help define our future as a company, and help define the future of open foundational models.

We want you to do the most impactful work of your career with the confidence that you and the people you care about most are supported.

• Top-tier compensation: Salary and equity structured to recognize and retain our talent globally.

• Stock options: Everyone who joins and contributes to Reflection's success gets to share in the upside through stock options.

• Health & wellness: Comprehensive medical, dental, vision, and life, with an annual wellness allowance.

• Meals: Lunch and dinner are provided in the office daily.

• Life & family: 22 weeks paid parental leave for all new birthing and non-birthing parents, including adoptive and surrogate journeys.

• Vacation days: Unlimited paid time off in the U.S. and 30 days in the U.K.

• Sponsorship support: We sponsor visas to help exceptional talent join our team and support long-term immigration pathways where applicable.

• Team building: We have regular off-sites, happy hours, and team celebrations.

Export Control Notice: This position may require access to technology or source code subject to the U.S. Export Administration Regulations. Any offer of employment for this role may be conditioned on the Company's ability to provide the candidate with access to such technology or source code in compliance with applicable U.S. export control laws, which may require the Company to seek government authorization.