Data Processing Developer

Job Summary

A Data Processing Services Developer would help us build the foundations for building Data Processing pipelines. This means managing SDP (Science Data Processing) resources (such as substantial storage, processing and networking) as well as functions (such as databases, monitoring as well as delivery & preservation) in coordination with overall observation requirements. A good candidate would help us with building and integrating such functionality with a focus on robustness and scientific reproducibility.

Key Responsibilities, Accountabilities and Dutie

sThe main functions of this role will be to

• :Help design, implement and integrate key SDP services

.
Mandatory Knowledge, Skills, and Experien

ceThe successful candidate will bring to the tea

• m:Experience with Python developme
• ntExperience with development of robust distributed systems in container orchestration environments such as Kubernet
• esExperience of Agile development processe
• s,Experience of the Git distributed version control syste
• m,Experience in test automation and continuous integratio
• n,Experience of Test Driven Developmen
• t,Excellent communication skills, ability to work effectively as part of a tea

m.
Other requiremen

• ts:Ability to travel to international planning meetings 2-4 times/y

ear
Desirable Knowledge, Skills, and Experi

enceThe ideal candidate would also have knowledge of some relevant topics such

• as:Management and scheduling workloads in a semi-autonomous fashion subject to external constra
• intsProvisioning and maintenance of large-scale storage, potentially covering all the way from burst buffers to long-term preservat
• ion.Monitoring and profiling of workloads in a cluster environ
• mentDesign and implementation of databases and their interfaces for scientific use cases, suc
• h asprocessing inputs (telescope & sky models, calibration), as wel
• l asprocessing outputs (data product catal
• og).Design and implementation of (operator) user interfaces for such serv