System Engineer

Job Duties and Responsibilities In this role, you will design, build, and manage robust CI/CD pipelines and infrastructure automation (IaC), with a primary focus on systems tailored for data processing, data pipelines, and Machine Learning (ML) model deployment. You will collaborate closely with data engineers and data scientists to effectively streamline complex data workflows and implement DataOps/MLOps best practices, ensuring the reliability, scalability, and optimal performance of data platforms and systems within cloud environments. Your work, centered around leveraging advanced automation, CI/CD methodologies, and cloud-native data services, is essential to supporting various data-driven initiatives and enhancing the efficiency of ML operations throughout the business. Proficiency and hands-on experience with CI/CD for data, IaC, and cloud data platforms are mandatory for this position.

Primary Skills

• Deep expertise in designing, building, and managing CI/CD pipelines specifically for data workflows and/or ML models, enabling efficient automation of data processing and model deployment lifecycles.
• Strong understanding of Infrastructure as Code (IaC) principles and proficiency with relevant tools (e.g., Terraform, CloudFormation, Ansible), including its application to provisioning and managing data-intensive infrastructure on cloud platforms.
• Proficiency in designing, building, and managing scalable and resilient data infrastructure solutions specifically within cloud environments (AWS, Azure, or GCP), integrating various data services (e.g., S3, Glue, Redshift, EMR, SageMaker, Azure Data Factory, Synapse, GCP Dataflow, BigQuery, AI Platform) and ensuring data platform reliability.
• Advanced skills in containerization (Docker) and orchestration (Kubernetes) applied to data and ML workloads, including deploying and managing containerized data pipelines and ML models.
• Ability to collaborate effectively with cross-functional teams, including data engineers and data scientists, translating MLOps/DataOps requirements into technical CI/CD and infrastructure solutions, and communicating system capabilities to stakeholders.

Preferred Qualifications

• Strong scripting skills, particularly in Python, for automation, integration, and development of custom tooling for data operations and CI/CD pipelines.
• General hands-on experience designing, building, and maintaining monitoring, logging, and alerting systems (e.g., Prometheus, Grafana, ELK Stack, CloudWatch, Datadog) for complex data platforms and ML systems.
• Familiarity with data pipeline orchestration tools (e.g., Apache Airflow, Kubeflow Pipelines, Azure Data Factory) beyond basic pipeline execution.
• Understanding of broader data engineering concepts (ETL/ELT, data warehousing, data lakes) and big data technologies (e.g., Spark, Kafka) to inform DevOps practices for data systems.
• Comprehensive knowledge of MLOps principles including model deployment strategies, versioning, continuous monitoring, and governance frameworks within production environments.
• Broader ability to collaborate effectively with operations teams and contribute to incident response and root cause analysis for data platform issues.
• Experience with data security best practices and their implementation within CI/CD pipelines and cloud infrastructure for data.