Data Engineer II

The posting, as SpyCloud wrote it

SpyCloud is on a mission to make the internet a safer place by disrupting the criminal underground. SpyCloud’s solutions thwart cyberattacks and protect more than 4 billion accounts worldwide. Cybersecurity is an exciting, evolving space, and being at the forefront of the fight to disrupt cybercrime makes SpyCloud a special place to work. If you’re driven to align your career with a fantastic mission, look no further!

This Data Engineer II role sits on the Engineering team, reporting to the Senior Director of Data Engineering. Your focus will include evolving our data platform using technologies such as Databricks and Spark, building scalable data pipelines, and serving customers through purpose-built data stores including RDS, Elasticsearch, and DynamoDB. This is a hands-on individual contributor role with ownership of complex data engineering initiatives that support the scalability, reliability, and continued evolution of SpyCloud’s data platform.

What You'll Do

• Design, build, and improve scalable data systems with a focus on performance, and reliability.
• Use Databricks and Spark to prepare, transform, and serve high-quality data to customers and downstream systems.
• Build, optimize, and maintain real-time and batch data pipelines using AWS technologies such as Lambda, S3, DynamoDB, ECS, EKS, and Terraform, as well as develop and maintain pipelines within the Databricks ecosystem.
• Identify and implement improvements to the cost and performance of data ingestion, transformation, and storage workflows.
• Be available for on-call support to troubleshoot and remediate data pipeline failures outside of standard business hours.
• Partner with Product Management, SpyCloud Labs, and Engineering teams to understand requirements and develop effective technical solutions.
• Evaluate and implement modern tooling for data processing, ML/AI data workflows, and observability.
• Contribute to architectural and technical design decisions and help implement scalable solutions aligned with engineering standards.
• Promote engineering best practices through collaboration, code reviews, documentation, and knowledge sharing.
• Identify and resolve complex data quality, performance, security, and reliability issues across production systems.
• Independently own complex data engineering initiatives from design and development through deployment and production support.
• Contribute to the continued evolution of SpyCloud’s data platform as product, customer, and AI requirements grow.

Requirements

• 5+ years of professional experience in data engineering, software engineering, or a closely related discipline.
• Experience building and operating Lakehouse, Databricks, or large-scale distributed data architectures in AWS.
• Strong understanding of distributed systems as they relate to data storage, processing, and computing.
• Experience building and managing Elasticsearch/OpenSearch data stores.
• Hands-on experience designing, building,

[…]
This policy applies to all terms and conditions of employment, including recruiting, hiring, placement, promotion, termination, layoff, recall, transfer, leaves of absence, compensation, and training.

SpyCloud expressly prohibits any form of workplace harassment. Improper interference with the ability of SpyCloud's employees to perform their job duties may result in discipline up to and including discharge. SpyCloud shares the right to work and participates in the E-Verify program in all locations.

If you need assistance or accommodation due to a disability, you may contact us.

Our Culture

Our culture is something really special. We’re all driven to disrupt the cybercriminal economy as we keep customer accounts safe from compromise. We support a truly worthy and serious mission, but we have fun doing it together. If you are driven, inventive, and collaborative, you’ll fit right in.

SpyCloud’s Recruitment Policy

We will never ask an applicant for sensitive or personal financial information during the recruitment process. We advise all applicants seeking employment with SpyCloud to review available information on recruitment fraud. Anyone who suspects that they have been contacted by someone falsely representing SpyCloud should email careers@spycloud.com .

Compensation Transparency Policy

At SpyCloud, we believe in transparency and fairness in compensation. We strive to ensure that all employees are fairly compensated for their contributions, and we openly discuss our compensation philosophy and structure. We are committed to providing competitive salaries and benefits packages to attract and retain top talent, and we encourage open dialogue and feedback regarding compensation matters.

Learn more and apply: SpyCloud Careers

SpyCloud is not sponsoring visas at this time.

For applicants residing in California, please click here to read SpyCloud's CCPA Notice.

For applicants residing in the UK, please click here to read SpyCloud's Employee Privacy Notice.