Data Engineering Manager
Indexed description
Recruiting for this role ends on 10/1/2026
Our experienced technology professionals offer deep technical experience in their area of focus and are committed to delivering efficient, technology-based solutions to our clients. Our professionals are also aligned to industry sectors. By combining our technical capabilities with our industry experience, we create unmatched market offerings to solve our clients' business issues. Additionally, we have long-term partnerships with many of the world's leading technological companies, allowing us to understand solution alternatives and recommend and support the most appropriate solution for our clients. By leveraging these elements, we can help our clients convert leading edge ideas into tangible results.
Work you'll do
A Data Engineering Manager will lead solution architecture for a PySpark-based enterprise data processing framework, improving scalability and standardizing transformation patterns across programs. Migrate major projects from on-premises platforms to AWS-based cloud data lake ecosystems, improving platform flexibility and modernization readiness. Design and implement a metadata-driven processing framework using PySpark, Python, Databricks, and AWS, reducing project go-live timelines. Build data lineage capabilities and conducted POCs in data observability, improving change impact analysis and accelerating enhancement delivery. Develop ingestion frameworks in Talend and later migrated orchestration to Apache Airflow, improving data onboarding efficiency. Performance tuning complex data quality rules, improving processing efficiency and reducing run-cost and maintenance overhead. Work across the full SDLC, including client discussions, requirements gathering, architecture design, development oversight, and production deployment. The successful candidate would possess these skills:
- Ability to work independently and collaborate as part of a team
- Effective written and verbal communication skills
- Meticulous attention to detail and quality of work product
- Ability to build and sustain professional relationships
- Ability to lead projects or workstreams
- Ability to manage and prioritize multiple tasks in a fast-paced and dynamic environment
- Strong interpersonal skills and professional demeanor
- Ability to meet deadlines
- Ability to provide clear guidance to others
Required Qualifications
- 8+ years of AWS cloud services and data lake build & support
- Strong experience in PySpark-based data engineering and ETL development
- SQL (Snowflake) pipeline build, data analysis, and data validation
- Requirements gathering and stakeholder coordination
- Bachelor's degree, preferably in Computer Science, Information Technology, Computer Engineering, or related IT discipline; or equivalent experience
- Limited immigration sponsorship may be available.
- Ability to travel 10%, on average, based on the work you do and the clients and industries/sectors you serve
- Strong analytical, problem-solving, communication, stakeholder-management, offshore coordination, documentation, and knowledge-transfer skills
- Analytical ability to manage multiple projects and prioritize tasks into manageable work product
- Can operate independently or with minimum supervision
- Excellent Written and Communication Skills
- Ability to deliver technical demonstrations
You may also be eligible to participate in a discretionary annual incentive program, subject to the rules governing the program, whereby an award, if any, depends on various factors, including, without limitation, individual and organizational performance.
Create a free Caio profile to unlock more results and save your role and location preferences.
Unlock free search