AI Databricks Engineer - New York

AI Databricks Engineer - Buy-Side Asset Management | Hybrid - New York (4 Days Per Week) - 12-Month - Contract to Hire Opportunity

We're partnering with a leading buy-side asset manager to hire a Data Engineer into their Engineering team. This opportunity offers the chance to become a key technical individual to the firm's growing US business.

The role sits within a small, highly collaborative engineering function and would suit someone who enjoys combining hands-on engineering with direct business engagement.

Key Responsibilities:

• Build and maintain scalable data solutions and pipelines within Databricks and Azure.
• Partner with US business stakeholders to understand requirements and deliver practical data-driven solutions.
• Act as a bridge between the business and engineering teams, translating business needs into technical outcomes.
• Support reporting, analytics and AI-related initiatives across the organisation.
• Contribute to data platform development, governance, optimisation and best practices.
• Work closely with a small, collaborative engineering team to solve complex data challenges and drive continuous improvements.

Required Experience:

• Strong Data Engineering experience with Databricks, Azure, Python/PySpark and SQL, including hands-on ownership of Databricks environments, data pipelines, governance, security and Change Data Capture (CDC).
• Experience building and supporting cloud-based data platforms, integrating data from external APIs, deploying production-ready data assets, and independently diagnosing and resolving pipeline and data quality issues.
• Strong understanding of modern engineering practices, including Git-based workflows, CI/CD (e.g. GitHub Actions), orchestration tooling, monitoring, validation frameworks and performance optimisation.
• Experience designing scalable data models and working with complex datasets. Exposure to financial datasets such as trade, reference, position or time-series data would be highly beneficial.
• Comfortable working in a fast-paced environment with evolving requirements, partnering directly with stakeholders to translate business needs into technical solutions and independently support business users.
• Experience with reporting tools such as Tableau or Sigma, platform engineering, AI-assisted development tools (Copilot, Cursor, Claude Code), or investment management environments would be highly advantageous.
• Strong communication skills with the ability to explain technical concepts to non-technical audiences.

Why Apply?

• High visibility role supporting the US business.
• Direct exposure to stakeholders and decision makers.
• Opportunity to work closely with engineering leadership.
• Mix of technical ownership and business engagement.
• Modern cloud and data technology stack.
• Small team environment with significant impact and autonomy.

If you or someone you know would be interested in learning more, please apply below or contact [email protected].