Data Engineer

IDR is seeking a Data Engineer to join one of our top clients for an opportunity in Malvern, PA. This role involves designing scalable data pipelines, ETL workflows, and cloud-based analytics solutions within the financial services industry. The position offers a chance to work with cutting-edge technologies and contribute to high-impact data projects.

Position Overview for the Data Engineer:

• Develops and maintains large-scale data pipelines and workflows using Python, SQL, PySpark, and Databricks.
• Implements cloud-based data solutions utilizing AWS services.
• Designs data models aligned with Medallion Architecture (Bronze/Silver/Gold) and Data Warehousing best practices.
• Collaborates with cross-functional teams to ensure data quality and accessibility.
• Contributes to the development and visualization of dashboards through Databricks.

Requirements for the Data Engineer:

• Expert knowledge of Python, SQL, PySpark, Databricks, Delta Lake, Apache Spark, Databricks Dashboards, Git, GitHub, CI/CD Pipelines, Data Modeling, Dimensional Modeling, Data Warehousing, Medallion Architecture (Bronze/Silver/Gold).
• Proven experience designing and implementing scalable data pipelines and ETL workflows.
• Strong background working with AWS cloud services.
• Excellent understanding of data modeling and architecture principles.
• At least 3 years of professional data engineering experience, with specific focus in Databricks projects.

What's in it for you?

• Competitive compensation package
• Full Benefits; Medical, Vision, Dental, and more!
• Opportunity to get in with an industry leading organization.

Why IDR?

• 25+ Years of Proven Industry Experience in 4 major markets
• Employee Stock Ownership Program
• Dedicated Engagement Manager who is committed to you and your success.
• Medical, Dental, Vision, and Life Insurance
• ClearlyRated's Best of Staffing® Client and Talent Award winner 12 years in a row.