Manager Ai Rater Jobs in Philadelphia, PA (NOW HIRING)

At Five Below our growth is a result of the people who embrace our purpose: We know life is way better when you are free to Let Go & Have Fun in an amazing experience, filled with unlimited possibilities, priced so low, you can always say yes to the newest, coolest stuff! Just ask any of our over 27,000 associates who work at Five Below and they'll tell you there's no other place like it. It all starts with our purpose and then, The Five Below Way, which is our values and behaviors that each and every associate believes in.

It's all about culture at Five Below, making this a place that can inspire you as much as you inspire us with big ideas, super energy, passion, and the ability to make the workplace a WOWplace!

1. AI Architecture & Strategy

• Define and own the enterprise AI architecture for retail use cases, aligning with business priorities and technology strategy.
• Develop reference architectures, patterns, and standards for AI/ML and Generative AI solutions, with an emphasis on open-source-first design principles.
• Translate retail business problems - across merchandising, supply chain, stores, marketing, and e-commerce - into scalable AI solution blueprints.
• Partner with business and product leaders to identify and prioritize high-impact AI opportunities.
• Champion open-source AI frameworks and tooling (e.g., Hugging Face, LangChain, LlamaIndex, Ray, MLflow, Feast) as the default approach before evaluating commercial alternatives.

2. Open-Source AI Architecture

• Lead the selection, evaluation, and integration of open-source AI and ML frameworks into Company's enterprise architecture.
• Design reusable patterns for open-source LLM deployment, fine-tuning, and serving (e.g., vLLM, Ollama, llama.cpp, OpenLLM).
• Establish governance standards for open-source model usage, including licensing review, security scanning, and model provenance tracking.
• Build internal capability around open-source foundations to reduce vendor lock-in and accelerate experimentation velocity.
• Evaluate and adopt emerging open-source agentic frameworks (e.g., AutoGen, CrewAI, LangGraph) for retail automation use cases.

3. AI Solutioning & Design

• Architect end-to-end AI solutions, including data ingestion, feature engineering, model training, inference, and system integration.
• Design AI systems for core retail domains such as:
• Search, recommendations, and personalization
• Demand forecasting, inventory optimization, replenishment, and allocation
• Pricing and markdown optimization
• AI assistants and copilots for store, merchandising, and supply-chain teams
• Define integration patterns between AI services and retail platforms (POS, OMS, WMS, CRM, e-commerce).
• Lead architectural reviews, ensuring solutions meet performance, scalability, security, cost, and reliability requirements.

4. AI Observability

• Define and implement an AI observability framework covering model performance monitoring, data drift detection, prediction quality tracking, and system health across all production AI systems.
• Establish real-time and batch monitoring pipelines for model inference using tools such as Evidently AI, Arize, WhyLogs, Fiddler, or equivalent open-source platforms.
• Design standardized dashboards and alerting for model degradation, data skew, latency SLO breaches, and feature store anomalies.
• Build feedback loop infrastructure to capture ground-truth labels and enable continuous model evaluation in production.
• Define observability standards for GenAI and LLM systems, including hallucination rate tracking, prompt/response logging, latency percentiles, and cost-per-query attribution.
• Partner with MLOps and Platform Engineering to embed observability as a first-class requirement in every AI system from Day 1.

5. AI Security

• Serve as the AI security authority for Company, owning the threat model for all AI and ML systems in production.
• Define and enforce secure-by-design standards for model development, training data handling, inference APIs, and GenAI integrations.
• Architect defenses against AI-specific attack vectors, including prompt injection, model inversion, adversarial inputs, data poisoning, and supply chain risks in open-source model adoption.
• Establish data privacy controls for AI pipelines, ensuring compliance with applicable regulations (e.g., CCPA) and internal data governance policies.
• Lead AI red-teaming and adversarial testing exercises to proactively identify and remediate security gaps before production deployment.
• Partner with Information Security, Legal, and Enterprise Risk to maintain an AI risk register and align AI security posture with the organization's broader cybersecurity framework.
• Define guardrails, content filtering, and human-in-the-loop safeguards for all customer-facing and associate-facing GenAI applications.

6. MLOps, GenAI & Governance

• Establish MLOps and AIOps practices, including CI/CD for models, automated retraining, monitoring, drift detection, and cost controls.
• Define standards for Generative AI and LLM usage, including multi-RAG architectures, MCP, and vector search.
• Define prompt orchestration, tool-calling, and agentic workflow patterns.
• Ensure AI solutions comply with data privacy, security, and responsible AI principles.
• Partner with Security, Legal, and Enterprise Architecture to align AI solutions with governance and risk standards.

7. AI Productivity Tooling Mandate

• Personally mandate and model the daily use of AI-native productivity tools across all architecture and delivery work.
• Evaluate, recommend, and govern the enterprise use of tools including:
• Microsoft Copilot - for productivity, code assistance, and enterprise knowledge retrieval
• Cursor - for AI-assisted development and code generation within engineering workflows
• Glean - for enterprise search, institutional knowledge management, and AI-powered information retrieval
• Claude (Anthropic) - for complex reasoning, document synthesis, and agentic task automation
• Equivalent or emerging AI productivity platforms as the market evolves
• Define standards and guardrails for enterprise AI tool adoption, including data classification policies governing what information may be shared with each platform.
• Train and upskill engineering and cross-functional teams on effective use of AI productivity tooling to multiply output and reduce time-to-delivery.

8. Technology Evaluation, Implementation & Delivery

• Work closely with AI Engineers, ML Engineers, Data Engineers, and platform teams to ensure architectures are production-ready and executable.
• Provide hands-on guidance during implementation, including reference code, pipelines, schemas, and infrastructure patterns.
• Evaluate and recommend AI SaaS solutions, cloud services, and frameworks (AWS, Azure, GCP, Databricks, Snowflake, etc.).
• Lead build vs. buy vs. open-source decisions and support vendor selection for AI capabilities.

• 9+ years of experience in software, data, or AI engineering, with 5+ years in AI/ML architecture roles.
• Proven experience designing and delivering production AI solutions specifically in retail, e-commerce, supply chain, or consumer-facing industries - this is a non-negotiable requirement.
• Deep hands-on expertise with open-source AI/ML ecosystem: Hugging Face Transformers, LangChain, LlamaIndex, MLflow, Ray, Feast, Evidently, or equivalent.
• Strong proficiency in Python and ML frameworks (PyTorch, TensorFlow, scikit-learn).
• Experience with modern data architectures: lakehouse, streaming, batch pipelines; platforms such as Databricks and Snowflake.
• Demonstrated experience designing AI observability systems - including model monitoring, drift detection, and production feedback loops.
• Working knowledge of AI security threat models, including prompt injection, adversarial attacks, and secure LLM deployment practices.
• Hands-on experience with cloud platforms and managed AI/ML services (AWS SageMaker, Azure ML, Vertex AI, or equivalent).
• Established practice of using AI productivity tools (e.g., Copilot, Cursor, Claude, Glean, or similar) in daily engineering and architecture work.
• Excellent communication skills with the ability to explain complex architectures to both technical and business stakeholders.

• Experience building or scaling enterprise AI platforms or AI Centers of Excellence.
• Contributions to open-source AI projects or published architecture patterns.
• Experience with AI red-teaming, adversarial testing, or formal AI risk assessment frameworks.
• Familiarity with retail-specific platforms: Manhattan WMS, Blue Yonder, Aptos POS, Salesforce Commerce Cloud, or equivalent.
• Cloud or AI certifications (AWS ML Specialty, Azure AI Engineer, GCP Professional ML Engineer).

Explore our benefits site to discover all the perks and support we offer! From health coverage to financial and personal wellness, we've got you covered-check it out today! benefits.fivebelow.com/public/welcome

Five Below is an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, age, national origin, disability, protected veteran status, gender identity or any other factor protected by applicable federal, state, or local laws.

Five Below is committed to working with and providing reasonable accommodations for individuals with disabilities. If you need a reasonable accommodation because of a disability for any part of the employment process, please submit a request and let us know the nature of your request and your contact information. crewservices.zendesk.com/hc/en-us/requests/new

BE AWARE OF FRAUD! Please be aware of potentially fraudulent job postings or suspicious recruiter activity by persons that are posing as a Five Below recruiters. Please confirm that the person you are working with has an @fivebelow.com email address. Additionally, Five Below does NOT request financial information or payments from candidates at any point during the hiring process. If you suspect fraudulent activity, please visit Five Below's Career Site to verify the posting. fivebelow.com/info/careers