Member of Technical Staff - Engineering Lead, Data Acquisition
Reflection · On-site · San Francisco, CA
Posted by Reflection on their careers site · View the original posting
Draft this job a résumé tailored to it, and the application filled in
draft's reading
Open 17 days
The posting, as Reflection wrote it
OUR MISSION
Reflection is a research lab making intelligence open and accessible for everyone to use, customize, and build on. We build open models that let anyone control their intelligence and help shape the future of AI. Our mission: make intelligence open and accessible to all.
ABOUT THE ROLE
Reflection's Data team builds the training corpora our frontier models learn from. Before a model can learn anything, the data has to be found, fetched, extracted, and delivered reliably, responsibly, and at enormous scale. The ingestion layer is the machinery that turns the open web, licensed corpora, and other large-scale sources into well-structured, versioned, auditable datasets for pre-training.
As Data Acquisition Lead, you'll provide front-line leadership of the team that builds this layer, spanning all three of its pillars web crawl, data ingestion pipelines, and data lakes. You'll build, mentor, and grow a team of data ingestion engineers, guide the technical and architectural decisions across crawling, extraction, and corpus storage/delivery, and work closely with the pre-training research, data quality, and data partnerships teams that depend on what you ship. You'll stay close enough to the stack to make targeted contributions as an individual contributor and to maintain a deep understanding of the team's technical work. Subtle decisions at the ingestion layer what we crawl, how we extract, what we keep ripple through training and directly affect where our models are strong, safe, and where they fail.
WHAT YOU'LL DO
- Build, mentor, and grow a high-performing team of data ingestion engineers. Coach and support your reports in understanding, and pursuing, their professional growth.
- Provide front-line leadership across the full ingestion stack — web crawling and acquisition, extraction and normalization pipelines, and the data lakes that version and deliver training corpora and the campaigns that run across all three.
- Stay hands-on: become familiar with the team's technical stack enough to make targeted contributions as an individual contributor.
- Manage day-to-day execution: prioritize the team's work and run data acquisition and ingestion campaigns in a highly dynamic, fast-paced environment.
- Guide technical and architectural decisions, emphasizing scalability, reliability, auditability, and cost crawler architecture and politeness/scheduling, distributed processing (Ray/Beam/Spark), orchestration (Airflow/Prefect), storage formats and layout (Parquet, JSONL, WARC; object stores and lakehouse formats), deduplication, and dataset versioning and delivery.
- Close the loop with research: partner with pre-training and data quality teams to tie ingestion decisions to measurable downstream model impact, and run experiments to evaluate crawling strategies, extraction methods, and ingestion tradeoffs.
- Work across the wider data effort data partnerships, digitization operations, and external vendors to onboard
[…]
ife & family: 22 weeks paid parental leave for all new birthing and non-birthing parents, including adoptive and surrogate journeys.
- Vacation days: Unlimited paid time off in the U.S. and 30 days in the U.K.
- Sponsorship support: We sponsor visas to help exceptional talent join our team and support long-term immigration pathways where applicable.
- Team building: We have regular off-sites, happy hours, and team celebrations.
Export Control Notice: This position may require access to technology or source code subject to the U.S. Export Administration Regulations. Any offer of employment for this role may be conditioned on the Company's ability to provide the candidate with access to such technology or source code in compliance with applicable U.S. export control laws, which may require the Company to seek government authorization.
Equal Employment Opportunity Statement
Reflection AI is an equal opportunity employer. We do not discriminate on the basis of race, color, religion, sex (including pregnancy), sexual orientation, gender identity or expression, national origin, ancestry, age, marital status, disability, genetic information, veteran or military status, or any other characteristic protected by applicable law. We encourage applicants from all backgrounds to apply.
We provide reasonable accommodations throughout the hiring process for applicants with disabilities, pregnancy-related limitations, or religious needs. If you need an accommodation, contact us at accommodations@reflection.ai.
We will consider for employment qualified applicants with arrest and conviction records in accordance with applicable law, including the San Francisco Fair Chance Ordinance and the California Fair Chance Act.
Candidate and Employee Privacy Notice: We collect and process personal data about applicants for the purposes described in our Candidate and Employee Privacy Notice https://cdn.sanity.io/files/irpbdun9/production/ee08c38bf22bd45a3f3a81f6a7abddb9210726b7.pdf