Job Title: AWS Data Engineer
Location: Remote
Employment Type: Contract / Full-Time
Job Summary
We are seeking a highly skilled AWS Data Engineer with strong experience in building scalable cloud-native data platforms and ETL pipelines on AWS. The ideal candidate should have hands-on expertise with AWS Glue, Spark ETL, Step Functions, Amazon EKS, Kubernetes, Lambda, S3 Data Lake architectures, SQS, KEDA, Glue Data Catalog, Glue Data Quality, and CloudWatch.
Key Responsibilities
·Design, develop, and maintain scalable ETL pipelines using AWS Glue and Apache Spark (PySpark).
·Build and orchestrate data workflows using AWS Step Functions.
·Design and implement S3 Data Lake architectures following AWS best practices.
·Develop and deploy containerized applications on Amazon EKS using Kubernetes.
·Build event-driven data processing solutions using Amazon SQS, AWS Lambda, and KEDA for auto-scaling.
·Manage metadata using AWS Glue Data Catalog.
·Implement data validation and governance using AWS Glue Data Quality.
·Monitor applications and data pipelines using Amazon CloudWatch.
·Optimize ETL jobs, Spark workloads, and Kubernetes deployments for performance and scalability.
·Collaborate with architects, developers, DevOps, and business stakeholders to deliver robust cloud data solutions.
·Participate in Agile ceremonies including sprint planning, code reviews, and production deployments.
Required Skills
·5+ years of experience in Data Engineering or Cloud Engineering.
·Strong hands-on experience with AWS Glue.
·Experience building Spark ETL pipelines using PySpark.
·Experience with AWS Step Functions.
·Strong knowledge of Amazon S3 Data Lake architecture and storage patterns.
·Hands-on experience with Amazon EKS and Kubernetes.
·Experience implementing event-driven architectures using Amazon SQS, AWS Lambda, and KEDA.
·Experience with AWS Glue Data Catalog and Glue Data Quality.
·Strong experience monitoring AWS workloads using Amazon CloudWatch.
·Good understanding of distributed data processing and cloud-native application design.
·Experience working in Agile/Scrum environments.
Preferred Skills
·Python / PySpark
·SQL
·Terraform or CloudFormation
·Docker
·Git
·CI/CD (Jenkins, GitHub Actions, CodePipeline)
·Kafka
·Apache Airflow
·Delta Lake / Iceberg / Lake Formation
·IAM, VPC, CloudTrail
Education
·Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field.
Mandatory Skills
·AWS Glue
·Spark ETL / PySpark
·AWS Step Functions
·Amazon S3 Data Lake
·Amazon EKS
·Kubernetes
·Amazon SQS
·KEDA
·AWS Lambda
·Glue Data Catalog
·Glue Data Quality
·Amazon CloudWatch
Nice to Have
·Terraform
·Docker
·Python
·SQL
·CI/CD
·Kafka
·AWS Certifications (Solutions Architect, Developer, Data Engineer, or DevOps Engineer)