Data Engineer - GCP, SQL,BiqQuery, Dataflow, Pub/Sub, Logging, IAM
Indexed description
Job Description
Job Description
We're seeking a talented Data Engineer with hands-on experience in the Google Cloud data ecosystem and a proven track record. You'll be instrumental in designing, building, and maintaining our data infrastructure and pipelines, enabling critical insights and supporting data-driven initiatives across the organization.
Responsibilities
- Data Pipeline Development: Design, build, and optimize robust and scalable data pipelines to ingest, transform, and load data from various sources into our data warehouse and knowledge graphs.
- Cloud Data Stack Expertise: Implement and manage data solutions using Google Cloud Platform (GCP) services such as BigQuery, Dataflow, Pub/Sub, Cloud Storage, Spanner and Dataproc and Azure Cloud Services
- Knowledge Graph Engineering: Develop and maintain data models, ingest data, and create efficient queries within Neo4j and/or Stardog. Leverage your expertise to build and expand our enterprise knowledge graph.
- Data Quality & Governance: Implement best practices for data quality, data validation, and data governance, ensuring data accuracy, consistency, and reliability.
- Performance Optimization: Continuously monitor and optimize the performance of data pipelines and database queries, identifying and resolving bottlenecks.
- Collaboration: Work closely with data scientists, analysts, and other engineering teams to understand data requirements and deliver effective data solutions.
- Documentation: Create and maintain comprehensive documentation for data pipelines, data models, and knowledge graph schemas.
Bachelor’s degree in computer science, Engineering, or a related quantitative field.
Experience
- 5+ years of professional experience as a Data Engineer or in a similar role.
- Strong hands-on experience with Google Cloud Platform (GCP) data services, including BigQuery, Dataflow, Pub/Sub, Cloud Storage, and Composer (Apache Airflow).
- Proficiency in Python for data manipulation and pipeline orchestration.
- Experience with SQL and data warehousing concepts.
- Familiarity with data modeling techniques relational database
- Experience with version control systems (e.g., Git).
- Application Support
- Terraform or any other IaC tools
- Experience with other GCP services
- Knowledge of streaming data technologies (e.g., Kafka, Google Cloud Dataflow streaming).
- Familiarity with data governance tools and principles.
- Certifications in Google Cloud Platform data engineering.
UPS is committed to providing a workplace free of discrimination, harassment, and retaliation.
Create a free Caio profile to unlock more results and save your role and location preferences.
Unlock free search