ML Researcher - Posttraining

The posting, as Krea wrote it

ABOUT KREA

At Krea, we are building next-generation AI creative tools.

We're dedicated to making AI intuitive and controllable for creatives - our mission is to build tools that empower human creativity, not replace it. We believe AI is a new medium that allows us to express ourselves through various formats - text, images, video, sound, and even 3D. We're building better, smarter, and more controllable tools to harness this medium. We recently took this a step forward with the launch of Krea 2 https://www.krea.ai/blog/krea-2-image-model, our first foundation model, built completely from scratch for aesthetic diversity and stylistic control, and with Krea 3 https://www.krea.ai/blog/introducing-krea-3 - our new image editing model.

We've raised over $83M and are backed by world-class investors such as a16z, Bain Capital, and Abstract. We work full-time and in-person at our waterfront office in San Francisco. We care about creativity: our team includes musicians, designers, visual artists, and engineers.

We're looking for someone with deep experience in large-scale finetuning of diffusion models and posttraining techniques. We have loads of high-quality data and aim to enhance the quality and aesthetics of the models we're building.

OUR CULTURE

• We work full-time and in-person at our North Beach office in San Francisco.
• We believe that demonstrated interest in the creative space is key: our team includes musicians, designers, visual artists and more.
• Fast iteration and execution speed. Bias towards action, agency, and independence.

WHAT YOU'LL DO

• Finetune diffusion models at scale to improve image aesthetics and quality.
• Implement posttraining techniques ranging from supervised finetuning, preference optimization, reinforcement learning, on-policy distillation, and various distillation / acceleration techniques.
• Design comprehensive eval suites and reward designs for the reinforcement learning stage focused on image space.
• Train custom VLM as reward models as part of our reward design.
• Train custom LLMs for prompt expansion through finetuning and reinforcement learning.
• Coordinate with data teams and partners to manage collection of preference data and model evaluation results.
• Work on safety alignment of our models for open source release.
• Collaborate with our AI research and engineering teams to integrate advancements into our products.

WHAT WE'RE LOOKING FOR

• Proven work of posttraining diffusion models for image or video generation.
• Experience with large-scale model training, inference, and optimization.
• Strong understanding of both LLM and diffusion post training pipelines and algorithms such as PPO, GRPO, DPO, OPD, and MOPD.
• Strong proficiency in PyTorch and understanding of its inner workings.
• Strong background in distributed training paradigms such as FSDP, CP, SP, USP, TP, and EP. Knowing how different parallelism strategies work together and their tradeo

[…]
es such as vLLM and sglang as well as existing RL frameworks in LLM space such as slime, miles, tinker, and verl.

• Understanding of various RL infrastructure and optimization techniques such as async RL, fast weight transfer, pipelining rollouts, managing off policy data.
• Ability to monitor model regression and identify weak areas and turn them into concrete evals and reward design.
• Experience training VLM models. Many of our custom reward models use VLM to provide reward signals for our models.
• Keeping up with the developments in related fields such as LLM, VLM, representation learning, and robotics research.
• Being comfortable working in a goal-oriented research environment.
• Having good judgement around when one should explore different training strategies and when it's time to commit to a specific strategy to scale compute and data.
• Comfortable working with underspecified goals. We expect every technical member to take an ambiguous research goal and break it down into concrete requirements, plans, experiment plan, and execution items.
• Good research taste — bias towards simplicity and methods that scale well with compute, data, and minimal human supervision.

WHAT WE OFFER

• Team: Work alongside a world-class team building the future of AI creative tooling
• Impact: Significant scope and company-wide impact
• Competitive compensation: generous salary & equity packages
• Health & wellness: 100% health & 99% dental/vision insurance premiums covered for employees, health FSA accounts, & long-term disability coverage
• Time off: Flexible PTO policy
• Financial planning: 401k with a 4% company-sponsored match
• Meals in the office: breakfast, lunch, dinner - you name it, we'll cover it
• Transit: Ubers covered to & from the office
• Sponsorship: We're open to sponsoring international visas where we can (e.g., STEM OPT, OPT, H-1B, O-1, E-3).
• And more!

Please note the above benefits & perks are for full-time employees