Data Engineer / Azure Databricks Engineer - Senior
Indexed description
Position SummaryNationwide IT Services, NIS, is seeking a Senior Data Engineer / Azure Databricks Engineer to support Air National Guard Chief Data and Artificial Intelligence Office (ANG CDAO) Support Services. The Senior Data Engineer will design, build, deploy, and optimize enterprise data pipelines and repository workloads in Azure Databricks. This role supports moving and transforming CDAO mission data while strengthening data quality, observability, performance, and repeatable delivery practices.
Key Responsibilities
- Design, develop, test, deploy, maintain, and optimize scalable ETL/ELT pipelines and Databricks workloads.
- Develop SQL and Python transformations, notebooks, jobs, and orchestration routines for scheduled and event-driven processing.
- Implement automated data-quality checks, reconciliation controls, exception handling, quarantine processes, and recovery procedures.
- Create and maintain source-to-target mappings, data lineage, technical documentation, operating procedures, and runbooks.
- Integrate APIs, files, databases, and cloud data services across DAF, ANG, DoD, and CDAO platforms.
- Implement Git-based version control, CI/CD automation, testing, monitoring, alerting, and controlled releases.
- Tune pipeline and repository performance to support availability, execution-success, data-currency, and response-time objectives.
- 8+ years of relevant professional experience.
- Bachelor’s degree in computer science, Data Engineering, Information Systems, Engineering, or a related field.
- Advanced hands-on proficiency with SQL, Python, Azure Databricks, and Azure data engineering services.
- Experience with ETL/ELT orchestration, APIs, data-quality controls, monitoring, troubleshooting, and performance tuning.
- Experience using Git and CI/CD practices to test and promote data-engineering code.
- Ability to document complex data flows, mappings, dependencies, and operating procedures.
- Must satisfy applicable DoD/DAF security, cybersecurity-awareness, CUI, OPSEC, and access-training requirements.
- Microsoft Azure or Databricks certification.
- Experience integrating DoD or DAF source systems.
- Familiarity with defense HR, financial, logistics, readiness, or maintenance data.
Our benefits package includes medical, dental, and vision insurance, life and disability insurance, a 401(k) plan with employer match, paid holidays, PTO (sick/vacation), commuter benefits, employee assistance program (EAP), educational reimbursement, and pet insurance.
Salary range $150,000 to $188,000 annually
The salary range is a general guideline. We consider several factors when determining base salary offers, including the position's scope and responsibilities, the candidate's experience, education, and skills, and current market conditions.
Create a free Caio profile to unlock more results and save your role and location preferences.
Unlock free search