We are seeking an experienced Big Data Administrator to manage and support enterprise Big Data infrastructure and production services. The role involves Hadoop cluster administration, Kafka management, OpenSearch/Elasticsearch monitoring, AWS EMR, system performance optimization, security, and incident management. The position requires strong technical expertise in Big Data technologies, ensuring high availability, system stability, data security, and efficient service delivery.

Mandatory Skill(s)

• Min 10 years of experience in Big Data or Data Platform Engineering within enterprise environments;
• Strong hands-on experience in the Hadoop ecosystem, including HDFS, YARN, Spark, MapReduce, HBase, and related components;
• Strong experience in Hadoop administration, cluster management, provisioning, scaling, patching, and troubleshooting;
• Hands-on experience in Apache Kafka administration and knowledge of Kafka internals;
• Experience managing and maintaining OpenSearch/Elasticsearch clusters;
• Strong knowledge of Linux system administration and scripting, preferably Shell or Python;
• Experience with AWS EMR and good knowledge of AWS Cloud;
• Experience in managing high-volume, low-latency data processing systems;
• Knowledge of data security, access governance, authentication, and cluster monitoring;
• Experience in performance tuning, capacity planning, and ensuring high availability;
• Strong troubleshooting, communication, analytical, and problem-solving skills.

Desirable Skill(s)

• AWS Cloud certification;
• Knowledge of access control and security mechanisms such as PAM and Kerberos;
• Experience with hardware configurations, racks, disk topology, and RAID;
• Knowledge of Virtual Machines, including deployment and configuration;
• Knowledge of JVM;
• Proficiency in Python, Java, or Scala;
• Knowledge of SQL, Hive, or other SQL-on-Hadoop tools;
• Experience with ETL processes and software.

Responsibilities

• Provide L3 technical support for Big Data infrastructure and production services;
• Manage and maintain Hadoop clusters and associated ecosystem components;
• Administer Kafka environments and ensure reliable, high-throughput data streaming;
• Monitor and manage OpenSearch/Elasticsearch clusters and optimize indexing and query performance;
• Ensure high availability, stability, and reliability of production systems;
• Troubleshoot and resolve complex technical issues and high-severity incidents;
• Perform cluster lifecycle management, capacity planning, and performance optimization;
• Implement security controls, user provisioning, authentication, and access management;
• Ensure data consistency, fault tolerance, and system resilience across data pipelines;
• Maintain technical documentation, operational procedures, and support runbooks;
• Review technology changes, identify potential risks, and recommend service improvements;
• Collaborate with technical teams, architects, and project stakeholders on technology roadmaps and infrastructure projects;
• Support production readiness and operational handover of new solutions;
• Provide technical guidance, coaching, and knowledge sharing within the team;
• Drive continuous improvement initiatives to enhance service quality and operational efficiency.

If you are interested in this role, click on the “Apply to this job” button below or you could also write in with your CV to Nabanita Poddar at nabanita.p@sciente.com quoting the job title.