Hadoop Administrator

Hadoop Administrator

Position Summary:

Responsible for the implementation, configuration, and ongoing administration of Hadoop infrastructure to support scalable data environments. This role involves close collaboration with system engineering, data delivery, and analytics teams to ensure the performance, security, and reliability of the Hadoop ecosystem.

Key Responsibilities:

Hadoop Infrastructure Management:

Install, configure, and maintain Hadoop clusters.

Add or remove cluster nodes using tools like Cloudera Manager, Nagios, Ganglia, and Dell OpenManage.

Perform cluster maintenance, including upgrades and patches.

User Management & Access Configuration:

Coordinate with data delivery teams to onboard new Hadoop users.

Set up Linux system users, Kerberos principals, and validate access to HDFS, Hive, Pig, and MapReduce.

Monitoring & Performance Tuning:

Monitor cluster performance, security, and connectivity.

Tune Hadoop MapReduce jobs and overall cluster performance.

Perform capacity planning and monitor usage patterns.

Log Management & Troubleshooting:

Analyze and manage Hadoop log files for proactive troubleshooting.

Support HDFS issues and perform file system monitoring and maintenance.

Collaboration & Cross-Team Integration:

Work closely with infrastructure, network, database, and BI teams to ensure seamless data availability and system stability.

Collaborate with application teams for OS and Hadoop version upgrades and patches.

Preferred Qualifications:

Technical Skills:

Strong hands-on experience with Hadoop ecosystem (HDFS, Hive, Pig, MapReduce).

Proficiency with Linux administration and Kerberos authentication.

Experience with monitoring and admin tools like Nagios, Ganglia, Cloudera Manager, and Dell OpenManage.

Familiarity with scripting and automation (Bash, Python, etc.) is a plus.

Soft Skills:

Strong troubleshooting and performance tuning skills.

Ability to work cross-functionally and support 24/7 production environments when needed.

Domain Knowledge:

Healthcare data experience (especially claims and member data) is highly preferred.

Travel Requirement: [Minimal/As Needed]