Back to search
Coditude Linkedin · Posted 4d ago

Senior Data Engineer — Source Integrations & Pipelines

Pune/Pimpri-Chinchwad

Linkedin
Continue to application Add your email once, then Caio opens the original posting.

Indexed description

Phase: Initial phase (foundational)

Required experience: 7+ years in data engineering, building and operating production data pipelines.

Role Summary

Builds the end-to-end data pipelines that bring every source into the platform reliably. Responsible for source integrations, ingestion, transformation, and data processing that feed the lakehouse and, later, the graph and ML layers. This is the core delivery role for the initial pilot use cases.

Key Responsibilities

  • Build and operate end-to-end data pipelines from source systems into the lakehouse / data fabric.
  • Integrate structured, semi-structured, document, and time-series sources into a common platform.
  • Implement data processing, cleansing, and transformation for the priority pilot use cases.
  • Ensure pipeline reliability, monitoring, and data quality across all feeds.
  • Work with the architect to align pipelines to platform standards and modeling conventions.
  • Prepare curated datasets for the visualization and MLOps workstreams.

Must-have Skills And Experience

  • Strong data engineering background building production pipelines end to end.
  • Hands-on experience with Azure data services and readiness to work with Snowflake.
  • Experience integrating diverse sources: Oracle ERP, MongoDB, PostgreSQL, Cassandra, Redis, InfluxDB, and time-series data.
  • Solid data processing skills (batch and streaming) and strong SQL.
  • Experience with object and file storage such as MinIO / NFS.
  • Data quality, testing, and pipeline observability practices.

Nice to have

  • Snowflake production experience.
  • Exposure to OT / IoT data feeds.
  • Familiarity with orchestration frameworks and infrastructure-as-code.

Relevant stack

Azure Data Factory / Synapse, Snowflake, Oracle ERP, MongoDB, PostgreSQL, Cassandra, Redis, InfluxDB, time-series sources, MinIO / NFS, APIs.

General attributes

  • Proactive and self-driven, able to take ownership and move work forward without waiting to be told.
  • AI-enabled in day-to-day work, comfortable using AI tools and copilots to accelerate delivery and quality.
  • Strong self-learner who stays current with evolving tools, platforms, and practices.
  • Good team player who collaborates well across engineering, operations, and stakeholder groups.

Required Skills

Broad source integration Data quality, testing, pipeline observability Azure data services OT/IoT exposure Snowflake production experience Object/file storage — MinIO/NFS

Free. 20 seconds. No password. See every match in this search.

Create a free Caio profile to unlock more results and save your role and location preferences.

Unlock free search
Want help applying to roles like this? Search Caio for free. If repetitive applications get heavy, Managed Job Search adds supervised execution for $99/month.
View Managed Job Search