Data Platform Infrastructure Engineer

Location: Scottsdale, AZ (100% Onsite)

Experience: 8+ Years

Employment Type: Full-Time / Contract

Position Overview

We are seeking a highly skilled Data Platform Infrastructure Engineer to design, deploy, and manage scalable data platform infrastructure across both on-premises and cloud environments. The ideal candidate will have strong expertise in Cloudera, Databricks, AWS, Terraform, and Ansible, with hands-on experience managing distributed data platforms and infrastructure automation.

This role is responsible for building reliable, secure, and high-performing data infrastructure that supports enterprise-scale analytics and data engineering workloads.

Key Responsibilities

• Design, deploy, and manage data platform infrastructure across on-premises (Cloudera) and cloud (AWS, Databricks) environments.
• Build, configure, and maintain distributed data clusters, ensuring high availability, scalability, and performance.
• Automate infrastructure provisioning and configuration using Terraform and Ansible.
• Manage and optimize Cloudera Hadoop ecosystems, including HDFS, Hive, Spark, and YARN.
• Deploy, configure, and administer Databricks workspaces, clusters, and integrations on AWS.
• Implement Infrastructure-as-Code (IaC) and configuration management best practices.
• Monitor platform performance, troubleshoot infrastructure issues, and ensure system reliability.
• Collaborate with Data Engineering, Architecture, and DevOps teams to support enterprise data pipelines and analytics workloads.
• Ensure platform security, governance, compliance, and operational excellence across data environments.

Must-Have Skills

• 8+ years of experience in Data Platform Infrastructure, Platform Engineering, or DevOps.
• Strong experience administering Cloudera Hadoop environments (HDFS, Hive, Spark, YARN).
• Hands-on experience with Databricks administration and cluster management.
• Strong AWS infrastructure experience.
• Expertise in Terraform and Ansible for infrastructure automation.
• Experience managing distributed data clusters in production environments.
• Strong understanding of Infrastructure-as-Code (IaC) principles.
• Experience with Linux system administration and shell scripting.
• Excellent troubleshooting and performance tuning skills.
• Strong communication and collaboration skills.

Required Qualifications

• Bachelor’s degree in Computer Science, Information Technology, Engineering, or a related field.
• Proven experience supporting enterprise-scale data infrastructure.
• Experience working in Agile/Scrum environments.
• Ability to manage multiple priorities while maintaining platform reliability and performance.

Preferred Qualifications

• Experience with Kubernetes and containerized platforms.
• Knowledge of CI/CD pipelines and DevOps best practices.
• Experience with monitoring and observability tools.
• Familiarity with enterprise security, governance, and compliance frameworks.
• AWS or Cloudera certifications are a plus.