**Exploring Model Graders for Reinforcement Fine-Tuning**

# **Exploring Model Graders for Reinforcement Fine-Tuning** *This guide is for developers and ML practitioners who already know their way around OpenAIʼs APIs, have a basic understanding of reinforcement fine-tuning (RFT), and wish to use their fine-tuned models for research or other appropriate uses. OpenAI’s services are not intended for the personalized treatment or diagnosis of any medical condition and are subject to our [applicable terms](https://openai.com/policies/).* [Reinforcement fine-tuning (RFT)](https://platform.openai.com/docs/guides/reinforcement-fine-tuning) of reasoning models consists of running reinforcement learning on top of the models to improve their reasoning performance by exploring the solution space and reinforcing strategies that result in a higher reward. RFT helps the model make sharper decisions and interpret context more effectively. In this guide, weʼll walk through how to apply RFT to the OpenAI `o4-mini` reasoning model, using a task from the life sciences research domain: predicting outcomes from doctor-patient transcripts and descriptions, which is a necessary assessment in many health research studies. We'll use a subset of the medical-o1-verifiable-problem [dataset](https://huggingface.co/datasets/FreedomIntelligence/medical-o1-verifiable-problem/viewer/default/train?row=0). You will learn key steps to take in order to successfully run RFT jobs for your use cases. Here’s what we’ll cover: - **[1. Setup](#1-setup)** - **[2. Gathering the dataset](#2-gathering-the-dataset)** - **[3. Benchmarking the base model](#3-benchmarking-the-base-model)** - **[4. Defining your grader](#4-defining-your-grader)** - **[5. Training](#5-training)** - **[6. Using your fine-tuned model](#6-using-your-fine-tuned-model)** ---