ZipRecruiter

Log In

1

Audio Speech Machine Learning Jobs (NOW HIRING)

previous page

Showing results 41-60

All JobsAudio Speech Machine Learning Jobs

Audio Speech Machine Learning information

What are some common challenges faced when developing machine learning models for audio speech applications?

A key challenge in audio speech machine learning roles is dealing with diverse and noisy audio data, which can significantly affect model accuracy. Additionally, models must be robust to different accents, languages, and speaking styles, requiring large and varied datasets for training and validation. Collaboration with data engineers, linguists, and software developers is often necessary to ensure high-quality data pipelines and model integration into production systems. Staying updated with the latest research and optimizing models for real-time performance are also ongoing aspects of the role.

What is an audio speech machine learning engineer?

An Audio Speech Machine Learning Engineer is a specialized professional who designs, develops, and implements machine learning models that process and analyze audio and speech data. Their work involves tasks like speech recognition, speaker identification, and audio event detection by leveraging algorithms and large datasets. These engineers collaborate with data scientists, software developers, and linguists to create applications such as voice assistants, transcription tools, and automated customer service systems. Expertise in signal processing, deep learning frameworks, and programming languages like Python is crucial for this role.

What is the difference between Audio Speech Machine Learning vs Speech Data Analyst?

AspectAudio Speech Machine LearningSpeech Data Analyst
Required CredentialsDegree in Computer Science, Data Science, or related fields; knowledge of ML frameworksDegree in Data Analysis, Statistics, or related fields; experience with data tools
Work EnvironmentResearch labs, tech companies, AI startupsData analysis teams, research institutions, tech firms
Industry UsageDeveloping speech recognition, voice assistants, NLP applicationsAnalyzing speech datasets, improving speech models, reporting insights

Audio Speech Machine Learning focuses on developing algorithms for speech recognition and processing, often involving model training and AI development. Speech Data Analysts interpret speech data, generate insights, and support model improvements. Both roles require strong analytical skills, but their core tasks differ: one builds models, the other analyzes data.

What are the key skills and qualifications needed to thrive as an audio speech machine learning engineer, and why are they important?

To thrive as an Audio Speech Machine Learning Engineer, you need a solid background in machine learning, signal processing, and programming (typically Python), along with a relevant degree in computer science or a related field. Familiarity with tools like TensorFlow or PyTorch, audio processing libraries (such as Librosa), and experience with speech datasets and ASR systems are commonly required. Critical soft skills include problem-solving, innovation, and effective communication for collaborating with cross-functional teams. These skills are essential to develop accurate, scalable speech recognition systems that advance voice-driven technology.
More about Audio Speech Machine Learning jobs
What cities are hiring for Audio Speech Machine Learning jobs? Cities with the most Audio Speech Machine Learning job openings:
What states have the most Audio Speech Machine Learning jobs? States with the most job openings for Audio Speech Machine Learning jobs include:
What job categories do people searching Audio Speech Machine Learning jobs look for? The top searched job categories for Audio Speech Machine Learning jobs are:
Audio Speech Machine Learning jobs near you
Infographic showing various Audio Speech Machine Learning job openings in the United States as of August 2026, with employment types broken down into 100% Full Time. Highlights an 100% In-person job distribution.

Machine Learning Engineer, Speech - Joint Audio-Video Modeling

Cantina

Remote

Full-time

Medical, Dental, Vision, Retirement, PTO

Posted 7 days ago

Job description

About Cantina:
Cantina Labs is a social AI company, developing a suite of advanced real-time models that push the boundaries of expression, personality, and realism. We bring characters to life, transforming how people tell stories, connect, and create. We build and power ecosystems. Cantina, our flagship social AI platform, is just the beginning.
If you're excited about the potential AI has to shape human creativity and social interactions, join us in building the future!
About the Role:
We're looking for a Research / ML Engineer to join our Speech Team to build state-of-the-art speech and audio generation systems end-to-end from data specs through production inference with a focus on joint audio-video modeling.
You'll own the audio side of multimodal generation: the representations (audio VAEs, neural codecs), the generative backbone (diffusion / flow-matching transformers), and the conditioning and alignment machinery that makes characters speak, sing, and emote in sync with what's on screen. That includes voice cloning and multi-speaker conditioning inside joint AV models, cinematic dialogue with music and sound design, and adjacent speech tasks (controllable TTS, voice conversion) that feed the same stack.
You'll drive the model ↔ data ↔ eval flywheel, partnering closely with research, video, data, and infra to ship fast, reliable, and cost-aware models. In this role you'll work at the intersection of cutting-edge research and practical engineering, contributing to the development of safe, steerable, and trustworthy AI systems.
You will thrive in this role if you:
  • See research and engineering as two sides of the same coin and enjoy owning work end-to-end.
  • Are excited to work across modalities and collaborate closely with a video generation team rather than staying inside audio.
  • Are results-oriented, flexible, and willing to pick up whatever moves the needle.
  • Like collaborating closely with infra, data, and product to ship measurable improvements.
  • Enjoy designing experiments, listening tests, and metrics that correlate with user-perceived quality.
  • Are eager to learn every day, and to find and solve unique large-scale problems.

What You'll Do:
  • Audio Representations: Design, train, and improve the audio VAEs, neural codecs, and vocoders our generative models sit on top of latent design, reconstruction and perceptual objectives, compression-vs-fidelity tradeoffs.
  • Model Building: Architect, implement, pre-train, fine-tune, and post-train/alignment (e.g., GRPO/DPO) diffusion and flow-matching transformers for large-scale audio and video generation.
  • Joint Audio-Video Modeling: Design the audio conditioning and cross-modal alignment inside joint AV models, audio latents alongside video latents, reference-audio and multi-speaker conditioning, multi shot generation audio/video modeling.
  • Experimental Design: Design, run, and analyze scientific experiments to advance our understanding of the models.
  • Data Ownership: Define data requirements and collaborate on acquisition, curation, AV-sync and quality filtering, annotation quality, and synthetic data strategies for paired audio-video and speech corpora.
  • Rigorous Evaluation: Design automated objective/subjective evaluations audio fidelity and intelligibility metrics, AV-sync, listening and viewing tests, robustness & bias checks, and red-team studies.
  • Inference Efficiency: Drive distillation, step-count reduction, quantization, and kernel/memory optimization to meet interactive latency and cost targets.
  • Pipeline Delivery: Harden the training → evaluation → inference pipeline; profile latency, memory, and cost; and meet production SLAs with robust monitoring and rollback.
  • GPU Scaling: Partner with infrastructure to run distributed training/inference on cloud fleets and productionize models with reliability and observability.
  • Project Leadership: Independently lead small research projects while collaborating on larger team initiatives, including cross-team work with video generation.
  • Tool Development: Develop and improve dev tooling to enhance team productivity.
  • Safety & Responsibility: Contribute to safety/consent guardrails, watermarking, and misuse/abuse mitigation for responsible voice and likeness technology.

What You'll Bring:
  • Exceptional research/development experience with large-scale audio models (>8B parameters, >500k hours of data).
  • Deep hands-on experience with diffusion and/or flow-matching transformers, including practical knowledge of samplers, schedules, conditioning mechanisms, and distillation.
  • Deep hands-on experience training audio VAEs, neural audio codecs, and vocoders latent/tokenizer design, reconstruction and perceptual objectives, adversarial training.
  • Strong experience with multi-node, multi-GPU distributed training (FSDP/DeepSpeed or equivalent).
  • Strong software engineering skills with a proven track record of building complex systems.
  • Strong with PyTorch and performance work (profiling, CUDA/Triton/C++ as needed) and writing reliable production-quality code.
  • Shipped large-scale speech/audio or multimodal generative models to production.
  • Background in working with large-scale ML data, and the ability to iterate on data and triangulate quality using both subjective and objective signals.
  • Experience with voice cloning, speech control/steerability, or expressive speech generation.
  • Notable publications and/or open-source contributions in speech/audio/ML.
  • Strongly preferred:
    • Experience with multimodal audio-video modeling: joint AV generation of multi-shot, multi-speaker scenes with dialogue, music, and sound design generated jointly with video, and the cross-modal alignment that keeps them in sync.
    • Experience with video generation: video diffusion/flow-matching transformers, video VAEs, conditioned and multi-shot generation, building data pipelines for video models.
    • Streaming or real-time generation, causal distillation (e.g., Self Forcing / Self Forcing++).

Compensation:
The anticipated annual base salary range for this role is between $200,000-$220,000 (€170,000-€190,000). When determining compensation, a number of factors will be considered, including skills, experience, job scope, location, and competitive compensation market data.
Benefits for U.S.-based roles:
  • Competitive salary and generous company equity
  • Medical, dental, and vision insurance - 99.99% of premiums covered by Cantina
  • 42 days of paid time off, including:
    • 15 PTO days
    • 10 sick days
    • 15 company holidays
    • 2 floating holidays
  • Generous parental leave & fertility support
  • 401(k) retirement savings plan
  • Lifestyle spending account - $500/month to use however you'd like
  • Complimentary lunch and snacks for in-office employees
  • One Medical membership, and more!

Apply