Data Engineer

Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems.

Must have skills : Python Frameworks

Good to have skills : NA

Minimum 3 Year(s) Of Experience Is Required

Educational Qualification : 15 years full time education

Project Role : Data Engineer

Project Role Description : Seasoned Python Developer with strong hands-on expertise in Python programming, cloud integrations, database development, and data engineering and cloud computing platforms such as Databricks, AWS, Azure & Google. Build and operate very large data warehouses or data lakes, ETL optimization, designing, coding, tuning big data processes using Big Data and Apache Spark. Build data pipelines applications to stream and process datasets at low latencies. Show efficiency in handling data - tracking data lineage, ensuring data quality, and improving discoverability of data.

Technical Experience :

• Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems.
• Detailed experience working on large databases (SQL, NoSQL, query optimization). Working experience in programming languages Python, Java, Scala Experience using airflow or orchestration platforms to build data pipelines in 2-3 projects.
• 2-3 years of experience developing CICD pipelines using GIT, Jenkins, Docker, Kubernetes, Shell Scripting, Terraform.
• Must be able to understand ETL technologies and translate into Cloud (AWS, Azure, Google Cloud) native tools, Python or PySpark.