Job Summary:
Diverse Lynx is a company focused on providing equal employment opportunities and fostering a diverse workforce. They are seeking a Python Spark Developer to work on large-scale data processing, building data pipelines, and integrating APIs. The role requires strong expertise in Python and Spark, along with experience in AWS-based data platforms.
Responsibilities:
• Strong experience in Python backend development (FastAPI preferred)
• Proven expertise in Spark (Scala or PySpark) for large-scale data processing
• Experience building data pipelines, ETL, and distributed processing systems
• Hands-on exposure to LLM/Agentic frameworks (LangGraph, Claude Code, or similar)
• Knowledge of API integration and schema transformation techniques
• Strong understanding of data validation, quality frameworks, and testing strategies
• Experience working on AWS-based data platforms
Qualifications:
Required:
• Strong experience in Python backend development (FastAPI preferred)
• Proven expertise in Spark (Scala or PySpark) for large-scale data processing
• Experience building data pipelines, ETL, and distributed processing systems
• Hands-on exposure to LLM/Agentic frameworks (LangGraph, Claude Code, or similar)
• Knowledge of API integration and schema transformation techniques
• Strong understanding of data validation, quality frameworks, and testing strategies
• Experience working on AWS-based data platforms
Preferred:
• Experience with agentic AI systems, multi-step reasoning workflows, or code generation pipelines
• Exposure to spec-driven development or model-driven architecture
• Familiarity with large enterprise ecosystems and tools (data validation platforms, ingestion frameworks, etc.)
• Experience handling high-volume datasets (1M+ records)
• Knowledge of data onboarding, vendor integration, or marketplace data standardisation
Company:
Diverse Lynx is a WBENC- and NMSDC-certified partner, helping organizations turn diversity goals into measurable impact through staffing and contingent workforce solutions. Founded in 2002, the company is headquartered in Princeton, New Jersey, US, , with a team of 1001-5000 employees. The company is currently Late Stage.