R230500760 - Senior Cloud Data Platform Engineer

Role- Senior Cloud Engineer / Senior Cloud Data Platform Engineer

Duration- 12 months

Locations- New York, NY / Charlotte, NC/ Iselin, NJ (hybrid)

This is a Hybrid role must be in office 2 days a week, based in our Client hubs New York , NY / Charlotte, NC/ Iselin, NJ.

The individual will be responsible for designing and engineering Cloud Data platforms. The individual will require to ensure high level of customer satisfaction for all Business users of the Data platforms. The role designs datastore systems that are appropriate for applications, customer needs and consistent with the overall design of the organization's information systems architecture. Under limited supervision, this job is responsible for the solution engineering and design, provisioning, delivery, service management, continuous automations of the organization's datastore systems. This role is hybrid.

Key Functions:

Design, develop and deliver cloud datastore solutions and develop automation pipelines to migrate data sets from On-prem to Cloud platforms. Practice Infrastructure as code to develop automation routines and integration flows to manage state of the datastore platform systems Provision secures from start datastores and enable them with required security controls including encryption, masking, certificate/keys rotation etc. Collaborates with developers, analysts, various system administrators to identify business requirements in designing efficient datastore solutions and interfaces. Identifies and documents all system constraints, implications, and consequences of various proposed system changes. Reviews technical documentation to guide system users and to assist with the ongoing operation, maintenance, and development of the system. Evaluates the efficiency and effectiveness of application operations and troubleshooting problems. Provide expert level IT technical lead services, including the direction, evaluation, selection, configuration, implementation, and integration of new and existing technologies and tools in a cloud platform. Responsible for development of cloud integrations and data migrations to support operations of Cloud infrastructure, provisioning, monitoring with (IaaS) and (PaaS) models Deploy, automate, maintain, and manage AWS cloud-based production system, to ensure the availability, performance, scalability, and security of productions systems. Manage the governance framework for DB-Services specific to private, hybrid and public cloud platform adhering to standards and integration with existing tools. Ability to anticipate technology changes within a rapidly evolving environment.

Requirements (Education and Experience):

• Master's or Bachelor's degree in computer science or related discipline Manage and ensure delivery of project tasks as required
• Excellent documentation skills to create and manage design, implementation and automation related documentation
• Implementation experience for Hadoop distribution platforms like Cloudera or AWS EMR.
• Extensive experience in designing, engineering and managing data lake ingestion, validation, transformation and consumption services leveraging cloud data tools like Hive, Spark, EMR, Glue ETL and Catalog, Snowflake etc.
• Experience on implementing solutions with Workflow orchestration/scheduling tools such as Airflow, Autosys etc.
• Experience building CI/CD pipelines for infrastructure implementations with Terraform, Gitlab etc.
• Must be ability to multi-task and prioritize tasks for self Must experience working in agile environment with multiple priorities
• Preferred Skills:
• 5+ years of implementing data applications or data platforms with Bigdata/Hadoop, Python/Java/Spark full stack, etc.
• 5+ year support experience of Big Data technologies in Hadoop ecosystem Hive, HDFS, MapReduce, Spark, Yarn, Kafka, Pig, HBase, Sqoop, Elastic Search, Kerberos.
• 5+ year experience with ETL/ELT tools such as AWS Glue ETL, Talend, DataStage etc.
• 5+ year experience with Orchestration tools such as Airflow, Autosys etc.
• Experience with Jira and Agile methodology is a plus; Experience with understanding DevOps or CI/CD process w.r.t to infrastructure management activities