Databricks Platform Engineer
Indexed description
We are seeking a Data Platform Engineer to build scalable data pipelines and transformations in Databricks, with a focus on geospatial data processing. The engineer will integrate ArcGIS and AWS data sources, develop Python and SQL transformations, and support secure, governed data access through Unity Catalog.
Responsibilities
- Build and maintain Databricks data pipelines and transformations.
- Ingest and process spatial data using the ArcGIS Python API.
- Develop scalable data transformations using Python, PySpark, and SQL.
- Integrate Databricks with AWS S3 data sources.
- Create and maintain Delta tables and curated datasets.
- Configure data access and governance using Unity Catalog.
- Implement data-quality checks, monitoring, and performance optimization.
Qualifications
- Strong hands-on experience with Databricks, Python, and SQL.
- Experience building production data pipelines and transformations.
- Experience with spatial data and the ArcGIS Python API.
- Knowledge of AWS S3 and cloud-based data architectures.
- Experience with Delta Lake and Unity Catalog.
- Understanding of spatial formats, coordinate systems, and geospatial transformations.
Create a free Caio profile to unlock more results and save your role and location preferences.
Unlock free search