Data Engineer

Role: Data Engineering Lead – 2 open demands.

• Play a large role in the implementation of complex features
• Lead collaboration with data scientists to understand the existing modeling pipeline and identify optimization opportunities.
• Oversee the integration and management of data from various sources and storage systems, establishing processes and pipelines to produce cohesive datasets for analysis and modeling.
• Push the boundaries of analytics and powerful, scalable applications
• Design and develop data pipelines to automate repetitive tasks within data science and data engineering.
• Demonstrated experience leading cross-functional teams or working across different teams to solve complex problems.
• Identify patterns and innovative solutions in existing spaces, consistently seeking opportunities to simplify, automate tasks, and build reusable components for multiple use cases and teams.
• Create data products that are well-modeled, thoroughly documented, and easy to understand and maintain.
• Drive innovative improvements to team development processes
• Partner with Product Managers and Customer Experience Designers to develop a deep understanding of users and use cases and apply that knowledge to scoping and building new modules and features
• Collaborate across teams with exceptional peers who are passionate about what they do

Ideal Candidate Qualifications:

• 8+ years of Engineering experience primarily in Data Engineering in an agile production environment
• High proficiency in using Python or Scala, Spark, Hadoop platforms & tools (Hive, Impala, Oozie, Airflow, NiFi, Scoop), SQL to build Big Data products & platforms
• Extensive experience with Databricks, Spark , PySpark or Java + PySprak Processing engine.
• Proficiency in, at least, one modern programming language such as Python, Java or Scala
• Strong Computer Science fundamentals in object-oriented design, data structures, algorithm design, problem solving, and complexity analysis.