ZipRecruiter

Log In

1

Senior Pyspark Developer Jobs (NOW HIRING)

Photon

Sr PySpark developer - Tampa

Orlando, FL

Requirements and Qualifications: * 8+ years of relevant experience in the Data Engineering, ApacheSpark * Strong proficiency in PySpark and Apache Spark. * Solid experience with SQL, NoSQL databases ...

Photon

Sr PySpark developer - Tampa

Orlando, FL · On-site

Requirements and Qualifications: * 8+ years of relevant experience in the Data Engineering, ApacheSpark * Strong proficiency in PySpark and Apache Spark. * Solid experience with SQL, NoSQL databases ...

Sarian, Inc.

Sr. Pyspark Data Engineer

Irving, TX · On-site

$109.90K - $132K/yr

Role: Sr. PySpark Data Engineer - Fulltime Location: Irving, TX We are seeking a skilled PySpark Data Engineer to join our team. The ideal candidate will have expertise in big data processing, ETL ...

next page

Showing results 1-20

All JobsSenior Pyspark Developer Jobs

Senior Pyspark Developer information

What cities are hiring for Senior Pyspark Developer jobs? Cities with the most Senior Pyspark Developer job openings:
What are the most commonly searched types of Pyspark Developer jobs? The most popular types of Pyspark Developer jobs are:
Senior Pyspark Developer jobs near you

Senior PySpark Developer

Accord Technologies Inc.

Iselin, NJ • On-site

Contractor

Posted 15 days ago

Job description

Job Title: Senior PySpark Developer
Location: Iselin, NJ (onsite)
Duration: 12 months
Job Type: W2 contract.
 
 
We are seeking an experienced Senior PySpark Developer to join our Data Engineering team in Iselin, NJ. In this role, you will design, develop, and optimize large-scale data processing pipelines using PySpark, Databricks, and AWS/Azure cloud services. You will collaborate with cross-functional teams to deliver high-performance data solutions that drive business intelligence, analytics, and reporting for our financial services platforms.
Key Responsibilities

  • Design & Development:

    • Develop and optimize PySpark applications for batch and real-time data processing.

    • Build scalable ETL/ELT pipelines using Databricks, Spark SQL, and Delta Lake.

    • Integrate data from multiple sources (databases, APIs, streaming platforms) into cloud-based data lakes/warehouses.

    • Implement data transformations, aggregations, and joins efficiently in distributed environments.

  • Performance Tuning & Optimization:

    • Optimize Spark jobs for performance (partitioning, caching, broadcast joins).

    • Troubleshoot and resolve data skew, memory issues, and job failures.

    • Monitor and fine-tune Databricks clusters for cost efficiency.

  • Cloud & Big Data Technologies:

    • Work with AWS (EMR, Glue, S3) or Azure (Synapse, Data Lake, ADF) for data storage and processing.

    • Implement data governance, security, and compliance best practices.

  • Collaboration & Leadership:

    • Partner with data scientists, analysts, and business teams to deliver actionable insights.

    • Mentor junior developers and enforce coding standards, testing, and CI/CD practices.

Required Skills & Experience

  • 5+ years of hands-on PySpark development experience.

  • Strong expertise in Apache Spark (SQL, DataFrames, RDDs) and Databricks.

  • Proficiency in Python, SQL, and shell scripting.

  • Experience with cloud platforms (AWS/Azure) and big data tools (Hive, Kafka, Snowflake).

  • Knowledge of data modeling, partitioning, and performance optimization.

  • Bachelor’s or Master’s degree in Computer Science, Engineering, or related field.

Preferred Qualifications

  • Financial services or banking industry experience.

  • Certifications in AWS/Azure, Databricks, or Spark.

  • Familiarity with CI/CD (Jenkins, GitLab) and infrastructure-as-code (Terraform).