ZipRecruiter

Log In

1

Big Data Spark Developer Jobs (NOW HIRING)

Orison

Spark Developer

Mclean, VA · On-site

Familiar with big data processing tools and techniques * Proven experience as a Spark Developer or a related role * Strong programming skills in Java, Scala, or Python * Familiarity with big data ...

Syntricate Technologies

Big data Engineer

Phoenix, AZ · On-site

$55.25 - $73.25/hr

... Big Data (Spark(Pyspark), HIVE). • Experience with CICD pipelines, Automated test frameworks, DevOps and source code management tools (XLR, Jenkins, Git, Maven). • Strong communication and ...

SmartIPlace

Big Data Engineer[Hybrid]

Dallas, TX · On-site

$55.25 - $73/hr

Skills Needed: Big Data, Spark SQL, Scala code, GCP Local to FL/TX/GA Position: Big Data Engineer Location: Temple Terrace, FL / Irving, Dallas TX / Alpharetta, GA (Hybrid - 3 Days in a Week Onsite ...

ClifyX

Scala Spark Developer

Richardson, TX · On-site

ClifyX is a company specializing in data solutions, and they are seeking a Scala Spark Developer to enhance their big data operations. The role involves hands-on experience with Spark, Hive, and ...

Ruri Software Technologies LLC

Big Data Engineer

Tampa, FL · On-site

$52.75 - $69.75/hr

TOP REQUIREMENTS: Big Data (Spark, Scala, Hive, Hadoop Ecosystem) Data Streaming (Kafka ... Experience with DevOps, Continuous Integration, and Continuous Delivery. * Expertise in performance ...

Triveni IT

BigData Spark Developer

Tampa, FL · On-site

$48.25 - $62.50/hr

We are seeking highly motivated engineers to join our team. The successful candidate will focus on ... Hands-on Experience in Big Data technologies (Hadoop, Spark, NoSQL) * Understanding Business ...

Info Way Solutions

Scala & Spark Developer

Austin, TX · On-site

$52.50 - $68.25/hr

Info Way Solutions is seeking a Scala & Spark Developer with significant experience in building and supporting scalable Big Data applications. The role involves developing modern data pipelines and ...

krg technology inc

Big Data spark Lead

New York, NY · On-site

A at krgtech.com Must have worked on atleast 1-3 big data projects with SPARK using JAVA and the Project should have gone Live to Production Must have supported atleast 2 or More projects as ...

Syntricate Technologies

Bigdata engineer

Phoenix, AZ · On-site

$55.25 - $73.25/hr

... Big Data (Spark(Pyspark), HIVE). • Experience with CICD pipelines, Automated test frameworks, DevOps and source code management tools (XLR, Jenkins, Git, Maven). • Strong communication and ...

Georgia IT, Inc.

Scala/ Spark Developer - Boston

Boston, MA · On-site

$57.50 - $74.75/hr

Georgia IT, Inc. is seeking a Scala/Spark Developer to join their team in Boston ... The role requires extensive experience in big data technologies, particularly with Scala and Spark ...

Hirekeyz Inc

Senior Java Spark Developer

Austin, TX · On-site

$56.75 - $72.25/hr

Senior Java Spark Developer Location: Austin, TX Job Type: Contract/W2/Full Time Job Summary ... Implement Java-based microservices and APIs for data integration. ✅ Big Data & Cloudera Ecosystem:

Hirekeyz Inc

Senior Java Spark Developer

Austin, TX · On-site

$56.75 - $72.25/hr

Senior Java Spark Developer Location: Austin, TX and Sunnyvale, CA --Hybrid (3 days Onsite ... Big Data & Cloudera Ecosystem: * Work with Cloudera Hadoop components such as HDFS, Hive, Impala ...

next page

Showing results 1-20

All JobsBig Data Spark Developer Jobs

Big Data Spark Developer information

What cities are hiring for Big Data Spark Developer jobs?

Cities with the most Big Data Spark Developer job openings:

What states have the most Big Data Spark Developer jobs?

States with the most job openings for Big Data Spark Developer jobs include:

What are popular job titles related to Big Data Spark Developer jobs?

For Big Data Spark Developer jobs, the most frequently searched job titles are:

Big Data/Spark Engineer

Rockville, MD • On-site

Software Guidance & Assistance
Health Care and Social Assistance • 51 - 200 employees
Learn more about Software Guidance & Assistanceexternal

$56.75 - $75.25/hr

Other

Re-posted 16 days ago

Key responsibilities

  • Design, develop, and maintain large-scale data processing pipelines using Big Data technologies (e.g., Hadoop, Spark, Python, Scala).

  • Implement data ingestion, storage, transformation, and analysis solutions that are scalable, efficient, and reliable.

  • Monitor and troubleshoot data pipelines in production environments to identify and resolve issues.

Job description

Software Guidance & Assistance, Inc., (SGA), is searching for a Big Data/Spark Engineer for a contract assignment with one of our premier Regulatory clients. Must be local to one of these office locations: Rockville MD or Tysons VA.
Hybrid work is required - non local candidates will not be accepted.
2-step interview process - onsite interview mandatory!
High-volume AWS environment, extremely large data.
Top Skills: Spark, SQL, Python, AI assisted workflows, Automation validation.
Seeking a Big Data Engineer to design, develop, and optimize large-scale data processing systems. Will work closely with cross-functional teams to architect data pipelines, implement data integration solutions, and ensure the performance, scalability, and reliability of big data platforms. Ideally will have deep expertise in distributed systems, cloud platforms, and modern big data technologies such as Hadoop, Spark etc.
Responsibilities :
  • Design, develop, and maintain large-scale data processing pipelines using Big Data technologies (e.g., Hadoop, Spark, Python, Scala).
  • Implement data ingestion, storage, transformation, and analysis of solutions that are scalable, efficient, and reliable.
  • Stay current with industry trends and emerging Big Data technologies to continuously improve the data architecture.
  • Collaborate with cross-functional teams to understand business requirements and translate them into technical solutions.
  • Optimize and enhance existing data pipelines for performance, scalability, and reliability.
  • Develop automated testing frameworks and implement continuous testing for data quality assurance.
  • Conduct unit, integration, and system testing to ensure the robustness and accuracy of data pipelines.
  • Work with data scientists and analysts to support data-driven decision-making across the organization.
  • Ability to write and maintain automated unit, integration, and end-to-end tests.
  • Monitor and troubleshoot data pipelines in production environments to identify and resolve issues.
Required Skills :
  • Bachelor's degree in Computer Science, Information Systems or related discipline with at least five (5) years of related experience, or equivalent training and/or work experience; Master's degree and past Financial Services industry experience preferred.
  • Demonstrated technical expertise in Object Oriented and database technologies/concepts which resulted in deployment of enterprise quality solutions.
  • Past experience with developing enterprise quality solutions in an iterative or Agile environment.
  • Extensive knowledge of industry leading software engineering approaches including Test Automation, Build Automation and Configuration Management frameworks.
  • Experience with object oriented programming languages such as Java, Scala or Python.
Essential Technical Skills:
  • AI Tool Proficiency: Hands-on experience with AI development tools (GitHub Copilot, Q Developer, ChatGPT, Claude, etc.).
  • Technical Background: Strong software development background with ability to contribute to technical discussions.
  • Agile Methodology: Extensive experience with Scrum, Kanban, and continuous improvement practices.
Big Data technologies
  • Experience with Big data technologies such as Hadoop, Spark, Hive & Trino.
  • Understanding of common issues like:
    • Data skew and strategies to mitigate it.
    • Working with massive data volumes in PetaBytes.
    • Troublehsooting job failures due to resource limitations, bad data, scalability challenged.
  • Real-world debugging and mitigation stories.
AI Skills
  • Prompt Engineering: Proficiency in crafting effective prompts for AI coding assistants and analysis tools.
  • AI Workflow Design: Experience redesigning development processes to leverage AI capabilities.
  • Data Analysis: Ability to interpret AI-generated insights and translate them into actionable team improvements.
  • Change Management: Experience leading teams through AI adoption and workflow transformation.
SQL Skills (Window Functions, Joins, Complex Queries)
  • Comfort with SQL window functions, multi-table joins, aggregations.
  • Provide examples of write/optimize SQL queries.
  • Handle edge cases like NULLs, duplicates, ordering, etc.
Apache Spark (Development, Internals & Tuning)
  • Understanding of Spark's core architecture - executors, tasks, stages, DAG.
  • Spark performance tuning techniques: partitioning, caching, broadcast joins, etc.
  • Troubleshooting slow running/stuck jobs or resource issues in Spark.
  • Experience optimizing Spark jobs for large-scale datasets.
Cloud Technologies
  • Exposure to AWS services like S3, EMR, Glue, Lambda, Athena, etc.
  • Use S3 with Spark (e.g., dealing with file formats, consistency issues).
  • EKS, Serverless knowledge, etc.
Programming - Python or Scala
  • Ability to write clean, modular, and performant code.
  • Functional programming concepts (e.g., immutability, higher-order functions).
  • Real-world use cases where they wrote scalable data processing code.
  • Understanding of collections, concurrency, and memory management.
Good to have:
  • Experience with managing production data pipelines/ETL systems.
  • Experience with CI/CD.
  • Experience writing test cases.
  • AWS certifications.
SGA is a technology and resource solutions provider driven to stand out. We are a women-owned business. Our mission: to solve big IT problems with a more personal, boutique approach. Each year, we match consultants like you to more than 1,000 engagements. When we say let's work better together, we mean it. You'll join a diverse team built on these core values: customer service, employee development, and quality and integrity in everything we do. Be yourself, love what you do and find your passion at work. Please find us at .
SGA is an Equal Opportunity Employer and does not discriminate on the basis of Race, Color, Sex, Sexual Orientation, Gender Identity, Religion, National Origin, Disability, Veteran Status, Age, Marital Status, Pregnancy, Genetic Information, or Other Legally Protected Status. We are committed to providing access, equal opportunity, and reasonable accommodation for individuals with disabilities in employment, and our services, programs, and activities. Please visit our company to request an accommodation or assistance regarding our policy.
#LI-SG1

Apply