Lead Data Engineer

Job Title - Lead Data Engineer

Salary - $160,000 - $180,000

We are currently seeking an experienced Data Engineer to join the Big Data and Advanced Analytics department. The Data Engineer will work closely with business domain experts to create an Enterprise Data Lakehouse to support data analytic use cases for the midstream oil and gas operations, engineering, and measurements business units.

Day to day:

• Design and implement reliable data pipelines to integrate disparate data sources into a single Data Lakehouse.

• Design and implement data quality pipelines to ensure data correctness and building trusted datasets.

• Design and implement a Data Lakehouse solution which accurately reflects business operations.

• Assist with data platform performance tuning and physical data model support including partitioning and compaction.

• Provide guidance in data visualizations and reporting efforts to ensure solutions are aligned to business objectives.

• Automate and optimize the data lifecycle, find insights from raw data, and applying DevOps principle to data pipelines.

• Work with business leaders to deliver custom software solutions meeting data needs.

• Build and support a data platform for data engineering teams to build, deploy and manage applications

Education:

• Bachelor's degree in Computer Science, Information Technology, or a related field.

Must Haves:

• Experience: 5+ years as a Data Engineer focused on designing and maintaining data pipeline architectures.

• Programming: 5+ years of in-depth experience with Python and SQL (specifically T-SQL).

• Cloud & Data Stack: Hands-on experience Snowflake.

• Architecture: Proven track record in high-level data platform design, strategy, and implementation. This includes:

• Data modeling and scalable architectures.

• Data governance, security, and compliance.

Preferred Skills

• Industry Domain: Background in Midstream, Supply Chain Logistics, or Chemical Engineering.

• 5+ years of full SDLC experience (version control, testing, refactoring, and deployment).

• Familiarity with Python packages: pandas, NumPy, PyArrow, Pytest, Scikit-Learn, and Boto3.

• Modern Data Tools:

• ELT/ETL tools: dbt and Airbyte.

• Orchestration & Containers: Apache Airflow and Kubernetes.

• Storage: S3 Object Storage and implementing Data Lakehouses (Apache Iceberg or Delta Lake).

• Commercial Systems: Familiarity with OpenLink or Quorum.

• Infrastructure: Experience with Infrastructure as Code (IaC) for building repeatable cloud environments.

This is a direct hire opportunity. The selected candidate will be employed directly by our client. All compensation and benefits, including but not limited to medical insurance, retirement plans, paid time off, and other perks, will be provided by the client in accordance with their internal policies and subject to applicable laws and eligibility requirements.