Big Data Engineer

Job Title: Big Data Engineer (Hybrid - Rockville, MD or Tysons, VA | 3 days onsite)

About the Role:

We are seeking an experienced Big Data Engineer to join our Market Regulation team, working on high-end engineering challenges. This role requires hands-on expertise with large-scale big data architectures and the ability to design and optimize solutions in a high data volume environment.

Key Responsibilities:

• Design, develop, and maintain big data pipelines and architectures supporting market regulation initiatives.
• Work with large-scale data processing using Spark, optimizing for performance and reliability.
• Develop and optimize complex SQL queries, including window functions and query tuning.
• Build and maintain batch processing workflows on AWS using EC2, EMR, and S3.
• Integrate AI-assisted workflows and automated validation processes to enhance data processing and accuracy.
• Collaborate with cross-functional teams to deliver scalable, efficient data solutions.

Must-Have Qualifications:

• 3-5 years of hands-on experience in big data engineering, especially in environments with high data volume.
• Strong proficiency with Spark and advanced SQL techniques.
• Skilled in Python programming for data processing and automation.
• Experience with AWS services: EC2, EMR, and S3 for batch processing.
• Minimum experience with AI-assisted workflows and automated validations.

Nice to Have:

• Exposure to the financial industry or market regulation domain.

Location: Rockville, MD or Tysons, VA (Hybrid, 3 days onsite)

If you are passionate about big data engineering and eager to solve complex architectural challenges in a dynamic environment, apply now to join our innovative Market Regulation team!

Apply today!