Job Summary:
Fireworks AI is a leading company in generative AI infrastructure, known for its high-quality models and fast inference capabilities. The Software Engineer on the AI Infrastructure team will design and build core systems to ensure the reliability and performance of the generative AI platform, collaborating closely with various teams to deliver effective infrastructure solutions.
Responsibilities:
• Contribute to the design and development of scalable backend infrastructure that supports distributed training, inference, and data pipelines
• Build and maintain core backend services such as LLM CI/CD pipeline, control plane, and model serving systems
• Support performance optimization, cost efficiency, and reliability improvements across compute, storage, and networking layers
• Building frameworks and safeguards to ensure Fireworks AI has the best model quality in the industry
• Collaborate with performance, training, and product teams to translate research and product needs into infrastructure solutions
• Participate in code reviews, technical discussions, and continuous integration and deployment processes
Qualifications:
Required:
• Bachelor’s degree in Computer Science, Engineering, or a related technical field (or equivalent practical experience)
• 3 years of experience in software engineering, with a focus on infrastructure or machine learning systems
• Strong programming skills in Python, Go, or a similar language
• Proven experience in ML infrastructure and tooling (e.g., PyTorch, MLflow, Vertex AI, SageMaker, Kubernetes, etc.)
• Basic understanding of LLM knowledge (e.g., context length, disaggregated prefill, KV cache memory estimation, etc)
Preferred:
• 5+ years of experience in software engineering, with a focus on infrastructure or machine learning systems
• Experience with open source inference engine like vLLM, Sglang, or TRT-LLM
• Contributions to open-source infrastructure or ML projects
• Experience in building large scale ML/MLOps infrastructure
Company:
Fireworks AI is an advanced platform that enables users to build, tune, and scale AI applications using open-source models Founded in 2022, the company is headquartered in Redwood City, USA, with a team of 51-200 employees. The company is currently Growth Stage.