ZipRecruiter

Log In

1

Big Data Testing Jobs (NOW HIRING)

Software Guidance & Assistance

Big Data Engineer

Rockville, MD · On-site

$56.75 - $75.25/hr

Develop automated testing frameworks and implement continuous testing for data quality assurance ... Big Data technologies * Experience with Big data technologies such as Hadoop, Spark, Hive & Trino

Conflux Systems

Big Data Engineer

Charlotte, NC · On-site

$55/hr

Participate in code reviews, testing, deployment, and production support. Required Skills Strong experience with Big Data technologies and the Hadoop ecosystem (HDFS, Hive, Spark, MapReduce, YARN ...

ACI Infotech

Big Data Engineer

$57 - $75.50/hr

... testing activities. • At least 4-7+ years of industry experience in Big Data Domain Education: • MS/BS minimum in the areas of Computer Science, Computer engineering, or other related fields or ...

Software Guidance & Assistance

Big Data Engineer

Rockville, MD · On-site

$56.75 - $75.25/hr

Develop automated testing frameworks and implement continuous testing for data quality assurance ... Big Data technologies * Experience with Big data technologies such as Hadoop, Spark, Hive & Trino

EROS Technologies Inc

Big Data Developer

Mooresville, NC · On-site

$50.25 - $65/hr

They are seeking a Big Data Developer to code, test, and analyze application software while ... Responsibilities : • Coding, testing and analyzing application software • Improve existing code ...

ClifyX

Bigdata/ETL Test Lead

Richardson, TX · On-site

The role involves leading testing efforts in Big Data and ETL environments, ensuring the quality of data through various testing strategies and methodologies. Responsibilities : • At least 10 to 12 ...

1 point system

QA Engineer (Data)

Fort Mill, SC · Remote

$30/hr

Some background in PySpark or Hadoop based data testing would be a big plus for us. A background mostly on the web and mobile automation side, won't be a great right fit for this role. If you can ...

NAVA Software Solutions

Big Data Engineer

Atlanta, GA · On-site

$53.50 - $71/hr

... building, testing, and optimizing 'Big Data' data ingestion pipelines, architectures, and data sets * 2+ years of experience with Python (and/or Scala) and PySpark/Scala-Spark * 3+ years of ...

TechWish

Big Data Developer

Mclean, VA · On-site

$54.25 - $70.50/hr

Develop automated testing frameworks and implement continuous testing for data quality assurance ... Experience with Big data technologies such as Hadoop, Spark, Hive & Trino * Evaluate understanding ...

previous pagenext page

Showing results 21-40

All JobsTechnology JobsData JobsData Software Engineer JobsBig Data Testing Jobs

Big Data Testing information

See salary details

$15

$62

$88

How much do big data testing jobs pay per hour?

As of Sep 2, 2026, the average hourly pay for big data testing in the United States is $62.98, according to ZipRecruiter salary data. Most workers in this role earn between $53.61 and $70.91 per hour, depending on experience, location, and employer.

What is big data testing?

Big Data Testing is the process of verifying and validating the quality, accuracy, and reliability of large and complex data sets and the systems that process them. It involves testing data ingestion, processing, storage, and retrieval to ensure that big data applications function as expected. This type of testing also checks for data integrity, performance, and scalability issues in big data environments such as Hadoop, Spark, or NoSQL databases. The goal is to ensure that data-driven applications deliver correct and meaningful insights.

What are the key skills and qualifications needed to thrive as a big data tester, and why are they important?

To thrive as a Big Data Tester, you need a solid understanding of data analytics, software testing methodologies, and a background in computer science or information technology. Familiarity with big data tools such as Hadoop, Spark, Hive, SQL, as well as automation frameworks and scripting languages like Python or Java, is typically required. Attention to detail, analytical thinking, and strong problem-solving abilities are crucial soft skills for identifying data issues and ensuring data quality. These skills are essential to validate large-scale data systems effectively, ensuring reliability, accuracy, and performance in data-driven projects.

What are some common challenges faced by professionals in big data testing, and how can they be addressed?

One common challenge in Big Data Testing is ensuring data quality and accuracy when dealing with vast, complex datasets that often span multiple sources and formats. Testers must also handle performance and scalability issues, as big data platforms process information at high volumes and speeds. Collaborating closely with data engineers and developers is essential to understand data flows and system architecture. To address these challenges, testers often use automation tools, develop robust test strategies, and continually update their skills to keep pace with evolving big data technologies.

What is the difference between Big Data Testing vs Data Analyst?

AspectBig Data TestingData Analyst
Required SkillsData validation, scripting, understanding of big data toolsData interpretation, SQL, visualization skills
Work EnvironmentTesting environments, big data platforms like Hadoop, SparkData analysis platforms, BI tools, databases
CertificationsBig Data certifications, ISTQB, testing-focused credentialsData analysis certifications, SQL, Tableau
Industry UsageQuality assurance in big data projectsBusiness insights, reporting, decision-making

Big Data Testing focuses on validating data quality, performance, and integrity within big data systems, requiring testing skills and knowledge of big data tools. Data Analysts interpret and visualize data to support business decisions, emphasizing analytical skills and data visualization. While both roles work with large datasets, Big Data Testing ensures system reliability, whereas Data Analysts focus on deriving insights from data.

Is big data testing a good career option?

Big Data Testing is a specialized field involving validating large datasets and data processing systems, often requiring knowledge of tools like Hadoop, Spark, and SQL. It offers strong job growth due to increasing data volumes and the need for data quality assurance, making it a viable career choice for those with technical skills in data management and testing methodologies.

What is the salary of big data tester?

The salary of a Big Data Tester typically ranges from $70,000 to $120,000 annually, depending on experience, location, and certifications. Entry-level positions may start lower, while experienced professionals with skills in tools like Hadoop or Spark can earn higher salaries.
More about Big Data Testing jobs

What states have the most Big Data Testing jobs?

States with the most job openings for Big Data Testing jobs include:

What job categories do people searching Big Data Testing jobs look for?

The top searched job categories for Big Data Testing jobs are:

Big Data Testing jobs near you
Infographic showing various Big Data Testing job openings in the United States as of August 2026, with employment types broken down into 1% As Needed, 85% Full Time, 11% Part Time, and 3% Contract. Highlights an 86% Physical, 3% Hybrid, and 11% Remote job distribution, with an average salary of $131,001 per year, or $63 per hour.

Big Data Engineer

Software Guidance & Assistance

Rockville, MD • On-site

$56.75 - $75.25/hr

Other

Re-posted 3 days ago

Job description

Software Guidance & Assistance, Inc., (SGA), is searching for a Big Data Engineer for a CONTRACT assignment with one of our premier Regulatory clients in the DMV area.
We are seeking a highly skilled and experienced Big Data Engineer to design, develop, and optimize large-scale data processing systems. In this role, you will work closely with cross-functional teams to architect data pipelines, implement data integration solutions, and ensure the performance, scalability, and reliability of big data platforms. The ideal candidate will have deep expertise in distributed systems, cloud platforms, and modern big data technologies such as Hadoop, Spark etc.
Responsibilities :
  • Design, develop, and maintain large-scale data processing pipelines using Big Data technologies (e.g., Hadoop, Spark, Python, Scala).
  • Implement data ingestion, storage, transformation, and analysis of solutions that are scalable, efficient, and reliable.
  • Stay current with industry trends and emerging Big Data technologies to continuously improve the data architecture
  • Collaborate with cross-functional teams to understand business requirements and translate them into technical solutions.
  • Optimize and enhance existing data pipelines for performance, scalability, and reliability.
  • Develop automated testing frameworks and implement continuous testing for data quality assurance.
  • Conduct unit, integration, and system testing to ensure the robustness and accuracy of data pipelines.
  • Work with data scientists and analysts to support data-driven decision-making across the organization.
  • Ability to write and maintain automated unit, integration, and end-to-end tests
  • Monitor and troubleshoot data pipelines in production environments to identify and resolve issues.
Required Skills :
  • Bachelor's degree in Computer Science, Information Systems or related discipline with at least five (5) years of related experience, or equivalent training and/or work experience; Master's degree and past Financial Services industry experience preferred.
  • Demonstrated technical expertise in Object Oriented and database technologies/concepts which resulted in deployment of enterprise quality solutions.
  • Past experience with developing enterprise quality solutions in an iterative or Agile environment.
  • Extensive knowledge of industry leading software engineering approaches including Test Automation, Build Automation and Configuration Management frameworks.
  • Strong written and verbal technical communication skills.
  • Demonstrated ability to develop effective working relationships that improved the quality of work products.
  • Should be well organized, thorough, and able to handle competing priorities.
  • Ability to maintain focus and develop proficiency in new skills rapidly.
  • Ability to work in a fast paced environment.
  • Experience with object oriented programming languages such as Java, Scala or Python.
  • Essential Technical Skills:
    • AI Tool Proficiency: Hands-on experience with AI development tools (GitHub Copilot, Q Developer, ChatGPT, Claude, etc.)
    • Technical Background: Strong software development background with ability to contribute to technical discussions
    • Agile Methodology: Extensive experience with Scrum, Kanban, and continuous improvement practices
  • Big Data technologies
    • Experience with Big data technologies such as Hadoop, Spark, Hive & Trino
    • Evaluate understanding of common issues like:
      • Data skew and strategies to mitigate it.
      • Working with massive data volumes in PetaBytes.
      • Troublehsooting job failures due to resource limitations, bad data, scalability challenged.
    • Look for real-world debugging and mitigation stories.
  • AI Skills
    • Prompt Engineering: Proficiency in crafting effective prompts for AI coding assistants and analysis tools
    • AI Workflow Design: Experience redesigning development processes to leverage AI capabilities
    • Data Analysis: Ability to interpret AI-generated insights and translate them into actionable team improvements
    • Change Management: Experience leading teams through AI adoption and workflow transformation
  • SQL Skills (Window Functions, Joins, Complex Queries)
    • Assess comfort with SQL window functions, multi-table joins, aggregations.
    • Provide examples or ask them to write/optimize SQL queries on the spot.
    • Probe how they handle edge cases like NULLs, duplicates, ordering, etc.
  • Apache Spark (Development, Internals & Tuning)
    • Test their understanding of Spark's core architecture - executors, tasks, stages, DAG.
    • Focus on Spark performance tuning techniques: partitioning, caching, broadcast joins, etc.
    • Ask scenario-based questions on troubleshooting slow running/stuck jobs or resource issues in Spark.
    • Explore their experience optimizing Spark jobs for large-scale datasets.
  • Cloud Technologies
    • Check exposure to AWS services like S3, EMR, Glue, Lambda, Athena, etc.
    • Ask how they've used S3 with Spark (e.g., dealing with file formats, consistency issues).
    • EKS, Serverless knowledge, etc.
  • Programming - Python or Scala
    • Assess ability to write clean, modular, and performant code.
    • Look for experience in functional programming concepts (e.g., immutability, higher-order functions).
    • Ask about real-world use cases where they wrote scalable data processing code.
    • Evaluate understanding of collections, concurrency, and memory management.
  • Good to have:
    • Experience with managing production data pipelines/ETL systems
    • Experience with CI/CD
    • Experience writing test cases
    • AWS certifications
By applying for a job with SGA, you agree to allow SGA to process your application for this and future opportunities in accordance with our Privacy Policy. Also, to ensure timely processing, you agree to be contacted by our AI recruiter via email, text, or phone. Message frequency varies and data rates may apply, but you can reply STOP to any SMS message to opt-out of texts and may contact SGA at to opt-out of AI communications. The choice not to engage with AI will not adversely impact your consideration for placement. AI is not used to make any hiring determinations.
SGA is a technology and resource solutions provider driven to stand out. We are a women-owned business. Our mission: to solve big IT problems with a more personal, boutique approach. Each year, we match consultants like you to more than 1,000 engagements. When we say let's work better together, we mean it. You'll join a diverse team built on these core values: customer service, employee development, and quality and integrity in everything we do. Be yourself, love what you do and find your passion at work. Please find us at .
SGA is an Equal Opportunity Employer and does not discriminate on the basis of Race, Color, Sex, Sexual Orientation, Gender Identity, Religion, National Origin, Disability, Veteran Status, Age, Marital Status, Pregnancy, Genetic Information, or Other Legally Protected Status. We are committed to providing access, equal opportunity, and reasonable accommodation for individuals with disabilities in employment, and our services, programs, and activities. Please visit our company to request an accommodation or assistance regarding our policy.
#LI-AC1

Apply