Kafka Administrator - Diverse Linx
Jersey City, NJ
About the Job
Kafka Administrator - Jersey City, NJ (Onsite)
Job Summary
We are seeking a highly motivated Kafka Administrator to join our team in Jersey City, NJ. In this role, you will be responsible for the day-to- day administration, monitoring, and optimization of our Kafka clusters. You will work closely with engineers to ensure the smooth operation and scalability of our real-time data streaming platform.
Responsibilities
Install, configure, and manage Apache Kafka clusters (single and multi-node) on-premise, potentially using VMs, Docker, or Kubernetes.
Experience with Confluent Platform is a plus.
Perform cluster builds, including design, infrastructure planning, high availability, and disaster recovery.
Implement security measures such as wire encryption (SSL), authentication (SASL/LDAP), and authorization (Kafka ACLs) in Zookeeper, Broker/Client, Connect clusters/connectors, Schema Registry, REST API, Producers/Consumers, and KSQL.
Perform high-level monitoring of Kafka clusters using appropriate tools and identify and troubleshoot potential issues.
Manage topics, partitions, replicas, and quotas.
Stay up-to-date on the latest Kafka developments and best practices.
Work collaboratively with engineers to understand data pipelines and ensure efficient data flow.
Document procedures and processes for Kafka administration.
Qualifications
Proven experience administering Apache Kafka clusters (minimum 2 years).
Strong understanding of distributed systems concepts.
Experience with scripting languages (e.g., Bash, Python) for automation.
Experience with monitoring tools (e.g., Prometheus, Grafana) preferred.
Excellent communication and collaboration skills.
Ability to work independently and as part of a team.