Python Developer
Charlotte, NC (onsite 3x a week)
1 year + contract
Must Have:
Python
Pyspark
ETL
Continuous Integration
JD for Python Full stack engineer.
Seeking a full stack Engineer to join our team, with expertise in Cloud, Python/ETL and Continuous Integration. This role focuses on designing, deploying and managing scalable data processing solutions in a cloud-native environment. You will work closely with data scientists, software engineers, and DevOps team to ensure robust, high-performance data pipelines and analytics platforms.
Responsibilities:
• Data Pipeline Development: Design and implement large-scale data processing workflows using Apache Spark
• Container Development: Design and implement docker images
• Optimization: Tune Spark jobs for performance, leveraging OpenShift's/Cloud resource management capabilities
• Integration: Integrate spark with other data sources (e.g., Kafka, s3, cloud storage) and sinks (e.g., databases, data lakes)
Qualifications:
• Experience:
- 4+ years working on Python.
- 1+ years of PySpark
- 1+ years of ETL
- Experience with Continuous Integration
- Ability to pick up and learn new technologies such as Axiom
- Knowledge of data, Continuous Integration of data, data elements, understanding what a PI is.
Technical Skills:
• Proficiency in Spark frameworks(Python/PySpark)
• Familiarity with Docker and Kubernetes concerts(e.g., pods, deployments, services and images)
• Hands on working experience on distributed systems, cloud platforms(AWS, GCP, Azure), and data storage solutions (e.g., S3, HDFS)
• Programing: Strong coding skills in Python, Airflow, Django; experience with shell scripting is a plus.
• Education: Bachelor's degree in Computer Science, Engineering or related filed.