Responsibilities may include:
•Solution consulting and building proof of concepts on
-Message based integration
•Establishing, Documenting and evangelizing development standards
•Monitor the performance of the systems and ensure high uptime and provide 24/7 support.
•Deploy new and maintain existing Kafka cluster environments on Cloud or on-prem.
•Handle all Kafka environment builds, including design, capacity planning, cluster setup, performance tuning and ongoing monitoring.
•Research and recommend innovative, and where possible, automated approaches for system administration tasks.
•Creation of key performance metrics, measuring the utilization, performance and overall health of the cluster.
•Proactively monitor and setup alerting mechanism for Kafka Cluster and supporting hardware to ensure system health and maximum availability
•Proven ability to lead multiple high priorities initiative with aggressive timelines leveraging an agile framework.
•Work closely with the various teams to make sure that all the Kafka topics are highly available and performing as expected.
•Work with Analytics Innovations COE and business analysts on designing and running technology proof of concepts on Kafka platforms
•Responsible for the security management of the platforms.
•5+ years of solid Hands-on Kafka Administration experience in managing critical 24/7 applications
•Design, build, assemble, and configure the application or technical architecture components using business requirements
•Proficiency in Confluent Kafka platform design, installation, operation, and the best practices for the below components:
-Confluent Kafka Connect/Connectors
-Confluent REST Proxy
-Confluent Schema Registry
-Confluent Control Center
-Confluent Security Settings
-Confluent RBAC for Active Directory Authentication/Authorization
•Experience in using JMX for Kafka monitoring and performance tuning.
•Familiar with the Kafka message level, transportation level and data at rest security.
•Experience in implementing security & authorization (permission-based) on Kafka cluster.
•Experience in open-source Kafka, Zookeepers, Kafka connect, schema registry Avro schemas.
•High availability cluster setup, maintenance, and ongoing support
•Create topics, setup redundancy cluster, deploy monitoring tools, alerts Has good knowledge of best practices
•Hands-on experience in creating a backup & mirroring of Kafka Cluster brokers, broker sizing, topic sizing, h/w sizing, performance monitoring, broker security, topic security, consumer/producer access management (ACL).
•Knowledge of Kafka API (development experience is a plus)
•Kafka DR/HA cluster setup experience including Cluster Replication settings is a plus
•Hands-on experience in Elastic ELK Stack Cluster setup, administration for Kafka Cluster Monitoring including integration with other systems like ServiceNow using the Elastic Webhook.
•Open to learning new technologies at a fast pace.
•Ability to document complex technical details about Kafka and other technologies as part of solution delivery.
•Bachelor's Degree in Computer Science, MIS, Information Technology or other related technical discipline
•1+ years of experience with AWS analytics stack
•2+ years of experience architecting streaming applications/Solutions
•1+ years of Oil and Gas Industry Experience
•Confluent Kafka Certification is a plus
•Knowledge of enterprise databases and/or data warehouse platforms such as Oracle, SQL Server or Teradata
•Automation experience with Python, PowerShell or a similar technology
•Experience with source control/automated deployment using CI/CD technologies. Useful technologies include Git/Jenkins/Ansible
•Experience with complex networking infrastructure including firewalls, VLANs, and load balancers