PySpark Developer

Developer (PySpark + Fabric) - London | Hybrid | 6 Month Contract

Opportunity for a strong PySpark Developer to join a large-scale financial data transformation programme focused on modernising critical data engineering and mastering platforms.

You’ll build and optimise PySpark pipelines handling complex financial instrument data, bi-temporal processing, streaming workloads, and publication pipelines across Microsoft Fabric and Azure ecosystems.

Key skills:

• Python 3.11+, PySpark 3.5, Spark SQL, Structured Streaming
• Microsoft Fabric, Lakehouses, Eventstream, Notebooks
• Azure Cosmos DB, Delta Lake, ADLS/OneLake
• CI/CD with GitLab, ARM Templates, Artifactory
• Great Expectations, pytest, DataDog, KQL
• Strong ETL/data modelling experience including temporal data patterns
• Experience building scalable, high-performance data pipelines

Desirable:

• Financial markets/reference data knowledge
• Entity resolution/matching systems
• Delta Lake CDF
• Power BI exposure
• Strong Azure cloud engineering background

Hybrid working in London with occasional travel between customer offices.

Excellent long-term project pipeline and strong day rates available for the right candidate.

For more information and direct contact with me, please send me an email and call only if required. I will be sure to check responses daily and shortlist the best candidates; if you are shortlisted, you will hear from me.

[email protected]

+44 7908143845