Threat Detection & Automation Engineer

Threat Detection & Automation Engineer

📍 Location: Sussex, WI, US

🏢 Industry: Financial services

💼 Work Setting: Hybrid

Are you a data engineering professional who enjoys building scalable data platforms, security automation solutions, and cloud-native analytics capabilities?

We are seeking a Senior Data Engineer to design, build, and optimize enterprise data pipelines, cybersecurity analytics platforms, and automation solutions that enable data-driven decision-making across Incident Response & Cybersecurity (IRC) functions. This role combines expertise in data engineering, cloud architecture, automation, security integrations, and analytics to deliver scalable, reliable, and governed data products.

The ideal candidate combines strong knowledge of SQL, Python, ETL/ELT architecture, AWS cloud services, data governance, cybersecurity operations, and automation engineering with the ability to collaborate across technical and business teams.

Key Responsibilities

Data Pipeline Engineering

• Design, develop, and maintain scalable data pipelines supporting security, operational, and business analytics.
• Build robust batch and streaming data processing solutions.
• Create reusable and maintainable data engineering frameworks.
• Ensure reliable ingestion, transformation, and delivery of structured and unstructured data.
• Support enterprise reporting and analytics initiatives.

ETL/ELT Development

• Develop and maintain enterprise-grade ETL/ELT processes.
• Design data transformation frameworks that support analytics and reporting needs.
• Implement efficient data integration patterns.
• Optimize extraction, transformation, and loading workflows.
• Support large-scale data processing across multiple systems and platforms.

Cloud Data Platform Engineering

• Design and support cloud-native data architectures.
• Develop:
• Data Lakes
• Analytical Data Pipelines
• Cloud-Based Processing Frameworks
• Scalable Data Repositories
• Leverage AWS services to support modern analytics capabilities.
• Ensure solutions are scalable, secure, and cost-effective.

Data Governance & Quality

• Implement controls supporting:
• Data Quality
• Data Integrity
• Data Lineage
• Data Governance
• Compliance Requirements
• Establish standards and best practices across the data lifecycle.
• Validate data sources and monitor pipeline accuracy.
• Ensure trusted and auditable analytics outputs.

Performance Optimization

• Monitor and improve existing data pipelines.
• Troubleshoot performance bottlenecks and reliability issues.
• Improve pipeline efficiency, scalability, and resiliency.
• Optimize large-scale data processing workloads.
• Drive continuous improvement initiatives across data platforms.

Security Automation & Orchestration

• Identify opportunities to automate cybersecurity workflows and operational processes.
• Develop automation solutions using:
• Python
• PowerShell
• Kubernetes
• Cloud-Native Services
• Build integrations that reduce manual effort and improve security operations efficiency.
• Support security orchestration initiatives and operational automation.

SOAR & Security Tool Integration

• Develop and maintain integrations with:
• Security Platforms
• Detection Tools
• Incident Response Systems
• Third-Party APIs
• Enable automated workflows across cybersecurity operations.
• Support orchestration and response initiatives.

Preferred Technologies

• Splunk SOAR (Phantom)
• Splunk SIEM
• ServiceNow
• Security APIs

Software Development & Automation Engineering

• Design and maintain internal tools and automation utilities.
• Develop custom solutions primarily using:
• Python
• PowerShell
• Bash
• JavaScript
• Support deployment, maintenance, and continuous enhancement of automation platforms.
• Promote software engineering best practices.

Dashboarding & Monitoring

• Support creation and maintenance of:
• Operational Dashboards
• Security Metrics
• Business Intelligence Reporting
• Monitoring Alerts
• Deliver visibility into cybersecurity and operational performance.
• Enable data-driven decision-making across teams.

DevOps & Platform Engineering

• Deploy and support solutions using modern DevOps practices.
• Work with:
• GitLab
• GitLab CI/CD
• Terraform
• Kubernetes
• Ansible
• Automate infrastructure provisioning and application deployment.
• Support Infrastructure as Code (IaC) initiatives.

Stakeholder Collaboration

• Partner with:
• Data Scientists
• Cybersecurity Analysts
• Incident Response Teams
• Engineers
• Business Stakeholders
• Gather requirements and translate business needs into technical solutions.
• Deliver scalable and actionable data products.
• Communicate technical concepts to both technical and non-technical audiences.

Technical Leadership & Mentorship

• Mentor junior engineers and analysts.
• Promote engineering standards and best practices.
• Support knowledge sharing and professional development.
• Encourage innovation and continuous learning.
• Serve as a technical resource across the team.

Documentation & Knowledge Management

• Maintain detailed documentation for:
• Data Pipelines
• Automation Tools
• Workflows
• Integrations
• Operational Processes
• Ensure solutions are maintainable and operationally supported.
• Support auditability and knowledge transfer activities.

Qualifications

Education

Preferred

• Bachelor's Degree in:
• Computer Science
• Data Engineering
• Software Engineering
• Cybersecurity
• Computer Engineering
• Information Systems
• Related Field

Experience

Required

• 5+ years of experience working with:
• SQL
• Python
• Data Engineering
• Data Pipeline Development
• Data Governance
• Experience building large-scale data processing solutions.
• Experience supporting cloud-native data architectures.
• Experience developing automation solutions.

Technical Skills

Data Engineering

• Data Pipeline Development
• ETL/ELT Design
• Data Modeling
• Data Warehousing
• Data Governance
• Data Integration
• Event Streaming
• Data Quality Management

Programming

• Python
• SQL
• PowerShell
• Bash
• JavaScript

Additional Preferred Languages

• Java
• C++
• Go

Cloud & Infrastructure

• AWS
• Cloud-Native Development
• Data Lakes
• Infrastructure as Code
• Kubernetes
• Terraform
• Ansible

DevOps & CI/CD

• GitLab
• GitLab CI/CD
• Infrastructure Automation
• Continuous Integration
• Continuous Deployment

Security Technologies

Preferred

• Splunk SIEM
• Splunk SOAR (Phantom)
• ServiceNow
• Security Operations Platforms
• Identity & Access Management (IAM)
• Firewalls
• IDS/IPS
• EDR Technologies

Analytics & Reporting

• Dashboard Development
• KPI Reporting
• Operational Analytics
• Monitoring Solutions
• Alerting Frameworks

Preferred Certifications

• AWS Certified Data Engineer – Associate
• Databricks Certified Data Engineer Associate
• Databricks Certified Data Engineer Professional
• Relevant Cloud or Data Engineering Certifications

Core Competencies

• Data Engineering
• SQL
• Python
• Data Architecture
• Data Modeling
• ETL/ELT
• Data Governance
• AWS
• Databricks
• Security Automation
• Splunk
• ServiceNow
• Kubernetes
• Terraform
• CI/CD
• API Development
• Cloud Data Platforms
• Cybersecurity Analytics
• Technical Mentorship