ZipRecruiter

Log In

1

Large Dataset Jobs (NOW HIRING)

Lumafield

Backend Software Engineer

San Francisco, CA · On-site

Responsibilities : • Use Python, Django and Django Rest Framework (DRF) to implement cloud backend APIs • Use Python, Django, and Celery to implement highly parallelized large dataset processing ...

Cacheflow

Geophysicist - AI Trainer

Manhattan, NY · On-site +1

You'll draw on your hands‐on experience with scientific visualization, fluid dynamics simulation, or large dataset analysis to evaluate AI-generated content and provide feedback that helps AI ...

Sciforium

LLM Dataset Engineer

San Francisco, CA · On-site

$155K - $210K/yr

Foundation Dataset Strategy: Own the end-to-end creation of pre-training datasets for LLMs. This ... Experience building large-scale image or video datasets from scratch (e.g., LAION-style pipelines)

Teserac, Inc.

Full Stack Developer

Sunnyvale, CA · On-site

Have worked with time-series or streaming data and understand the performance implications of large dataset rendering. * Learn quickly and enjoy working at the intersection of software, data, and ...

TUPPL Technology Inc

Senior Data Analyst Lead

Dallas, TX

$85K - $107K/yr

Required Skills & Experience • 8-10 years of experience as a Data Analyst, with strong exposure to Banking. • Advanced proficiency in SQL (query optimization and large dataset handling). • ...

New

next page

Showing results 1-20

People also search for

All JobsLarge Dataset Jobs

Large Dataset information

See salary details

$47K

$98.6K

$140.5K

How much do large dataset jobs pay per year?

As of Jun 6, 2026, the average yearly pay for large dataset in the United States is $98,572.00, according to ZipRecruiter salary data. Most workers in this role earn between $84,000.00 and $113,000.00 per year, depending on experience, location, and employer.

What are the key skills and qualifications needed to thrive as a Data Scientist working with large datasets, and why are they important?

To thrive as a Data Scientist handling large datasets, you need strong analytical skills, proficiency in statistics, and a background in computer science or a related field. Expertise in programming languages like Python or R, familiarity with big data frameworks (such as Hadoop or Spark), and experience using data visualization tools are typically required. Strong problem-solving ability, attention to detail, and effective communication skills help in translating complex data findings into actionable insights. These skills are essential for extracting meaningful information from massive datasets, supporting data-driven decision-making, and driving business value.

What is the difference between Large Dataset vs Data Analyst?

AspectLarge DatasetData Analyst
Required CredentialsOften no formal degree, but knowledge of data managementBachelor's degree in data science, statistics, or related field
Work EnvironmentData storage, database management, data processingData interpretation, reporting, visualization
Industry UsageUsed across industries for storing and managing dataApplied in business, finance, healthcare for analysis
Search & Comparison IntentUnderstanding data volume managementAnalyzing data to generate insights

Large Dataset refers to the volume of data stored and managed, often requiring data engineering skills. Data Analysts focus on interpreting and visualizing data to support decision-making. While large datasets are the raw material, data analysts turn that data into actionable insights.

What are large datasets?

Large datasets are collections of data that are so vast in size or complexity that traditional data processing software struggles to manage, process, or analyze them efficiently. They are often associated with 'big data' and can include structured, semi-structured, or unstructured data from sources such as social media, sensors, business transactions, and more. Handling large datasets typically requires specialized tools and techniques for storage, computation, and analysis, such as distributed computing frameworks like Hadoop or Spark. These datasets are crucial in fields like data science, machine learning, and analytics, enabling deeper insights and data-driven decision-making.

What are some common challenges when working with large datasets, and how can professionals overcome them?

Professionals handling large datasets often face challenges such as ensuring data quality, managing storage and processing constraints, and optimizing data retrieval times. To overcome these, it's important to leverage scalable data storage solutions, such as distributed databases or cloud platforms, and utilize data processing frameworks like Hadoop or Spark. Regular data validation, efficient indexing, and collaborating closely with data engineers and analysts can also help maintain accuracy and streamline workflows.
Infographic showing various Large Dataset job openings in the United States as of May 2026, with employment types broken down into 1% As Needed, 91% Full Time, and 8% Part Time. Highlights an 79% Physical, 6% Hybrid, and 15% Remote job distribution, with an average salary of $98,572 per year, or $47.4 per hour.
Backend Software Engineer

Backend Software Engineer

Lumafield

San Francisco, CA • On-site

Full-time

Posted 2 days ago

Job description

Job Summary:
Lumafield is a company founded to upgrade manufacturing through innovative engineering solutions. The Backend Software Engineer will work on the core of the cloud platform, implementing APIs and processing large datasets while collaborating with research teams and product managers.
Responsibilities:
• Use Python, Django and Django Rest Framework (DRF) to implement cloud backend APIs
• Use Python, Django, and Celery to implement highly parallelized large dataset processing tasks ranging up to 100s of GBs of data
• Design for and deploy your code to our AWS environment leveraging EKS, S3, CloudFront, and other AWS technologies
• Work closely with our research and algorithms team to incorporate cutting edge algorithms into our production codebases
• Collaborate closely with product managers and engineering leadership to align technical objectives with business deliverables.
• Get your hands dirty and build – expect to be hands-on and building regularly
Qualifications:
Required:
• Bachelor's Degree in Engineering or related field
• 2+ years of experience with Python in a production backend setting using major web frameworks (Flask, Django, FastAPI, etc.)
• Experience with large dataset processing using numpy
• Strong software engineering fundamentals including git, unit testing, pull request reviews, module/interface design, and applications using parallelism and concurrency.
• Strong team collaboration, communication and interpersonal skills
• Experience with Linux server administration, network troubleshooting, docker deployments, productionizing systems
Preferred:
• Experience with Agile Development practices
• Experience with AWS including EKS, S3, CloudFront, or similar
• Experience with image processing pipelines and/or image acquisition
• Experience in configuring Linux systems, applying best practices, and automating workflows with Ansible and scripting.
Company:
Lumafield develops industrial computed tomography (CT) solutions for non-destructive testing and inspection for engineers. Founded in 2019, the company is headquartered in Cambridge, USA, with a team of 51-200 employees. The company is currently Growth Stage.

Apply