Research Lead - Pre-training Safety

The posting, as FAR.AI wrote it

ABOUT US

FAR.AI http://FAR.AI is a non-profit AI research institute working to ensure advanced AI is safe and beneficial for everyone. Our mission is to facilitate breakthrough AI safety research, advance global understanding of AI risks and solutions, and foster a coordinated global response.

Since our founding in July 2022, we've grown to 50+ staff https://www.far.ai/about/team, published 40+ academic papers https://scholar.google.com/citations?user=FVJ24k8AAAAJ, and convened leading AI safety events https://far.ai/events/. Our work is recognized globally, with publications at premier venues such as NeurIPS, ICML including a Best Paper Honorable Mention in 2026 https://icml.cc/virtual/2026/oral/71065, and ICLR, and features in the Financial Times https://www.ft.com/content/175e5314-a7f7-4741-a786-273219f433a1, Nature News https://www.nature.com/articles/d41586-024-02218-7, Wired Magazine https://www.wired.com/story/jailbreaking-ai-models-google-anthropic-openai-spacexai/ and MIT Technology Review https://www.technologyreview.com/2020/02/28/905615/reinforcement-learning-adversarial-attack-gaming-ai-deepmind-alphazero-selfdriving-cars/. We conduct pre-deployment testing on behalf of frontier developers such as OpenAI, independent evaluations for governments including the EU AI Office https://www.far.ai/news/far-ai-selected-to-lead-eu-ai-act-cbrn-risk-consortium, and publish the AI Security Leaderboard https://leaderboard.far.ai/ based on our red-teaming expertise. We help steer and grow the AI safety field through developing https://arxiv.org/abs/2405.06624 research https://arxiv.org/abs/2506.20702 roadmaps https://www.researchgate.net/publication/396910034_Open_Technical_Problems_in_Open-Weight_AI_Model_Risk_Management with renowned researchers such as Yoshua Bengio; running FAR.Labs https://www.far.ai/programs/far-labs, an AI safety-focused co-working space in Berkeley housing 40+ members; and supporting the community through targeted grants https://www.far.ai/programs/grantmaking to technical researchers.

FAR.AI http://FAR.AI is hiring a Research Lead to develop and lead our work on pre-training safety, shaping models’ capabilities and internal representations at their source, rather than trying to fix them after the fact.

Our initial focus is capability control: removing harmful capabilities while preserving benign ones. We see this as a promising way to prevent misuse of open-weight models in areas such as CBRN and cyber by removing offensive capabilities, and reducing loss-of-control risks by removing knowledge of oversight mechanisms. We will validate approaches like pre-training data filtering at scale, drive adoption of successful methods, and explore techniques such as gradient routing and unlearning..

We are scaling methods like Deep Ignorance https://deepignorance.ai/ by over an order of magnitude (>100B parameter models with >1T tokens). You will direct this work, partner with our red team to stress-test the resultin
[…]
LG). Outside the USA or Singapore, you will be employed via an EOR organisation on behalf of FAR.AI http://FAR.AI or as a contractor.

• Location: Both remote and in-person (Berkeley, CA or Singapore) are possible. We sponsor visas for in-person employees, and can hire remotely in most countries.
• Hours: Full-time (40 hours/week).
• Compensation: $290,000–$450,000/year depending on experience and location. For exceptional candidates, we will accommodate a higher salary range. We will also pay for work-related travel and equipment expenses. We offer catered lunch and dinner at our offices in Berkeley.
• Compensation: $290,000 to $450,000 per year. Compensation is determined by our salary bands and your level of experience, with adjustments for location. We also pay for work-related travel and equipment expenses, and offer catered lunch and dinner at our office in Berkeley.
• Application materials: Expect ~1–2 hours of preparation. We ask for a CV, a short research direction statement (the form supports both fully-formed agendas and developing ones), 2–3 selected works with a brief note on your personal contribution, and a short note on why FAR.AI http://FAR.AI is a good home for your direction.
• Process: From application: a 45-minute bilateral fit call, 2 technical assessments with FAR.AI http://FAR.AI technical staff, a 3-5-day paid work trial (depending on your availability), and structured reference calls. Typical elapsed time: 4 weeks. Total candidate time end-to-end is ~30-45 hours, with the paid work trial being the bulk. If a 3-5-day block isn't feasible for you, reach out — we can discuss alternatives.

If you have any questions about the role, feel free to contact us at talent@far.ai. Otherwise, if you don't have questions, the best way to ensure a proper review of your skills and qualifications is by applying directly via the application form. Please don't email us to share your resume (it won't have any impact on our decision). Thank you!