name: openai-api description: OpenAI API documentation. Use for GPT models, Chat Completions, Embeddings, Assistants API, Vision, Audio, Fine-tuning, and integrating OpenAI into applications.

Openai-Api Skill

Comprehensive assistance with openai-api development, generated from official documentation.

When to Use This Skill

This skill should be triggered when:

Working with openai-api
Asking about openai-api features or APIs
Implementing openai-api solutions
Debugging openai-api code
Learning openai-api best practices

Quick Reference

Common Patterns

Pattern 1: Introduction This API reference describes the RESTful, streaming, and realtime APIs you can use to interact with the OpenAI platform. REST APIs are usable via HTTP in any environment that supports HTTP requests. Language-specific SDKs are listed on the libraries page.Authentication The OpenAI API uses API keys for authentication. Create, manage, and learn more about API keys in your organization settings. Remember that your API key is a secret! Do not share it with others or expose it in any client-side code (browsers, apps). API keys should be securely loaded from an environment variable or key management service on the server. API keys should be provided via HTTP Bearer authentication. Authorization: Bearer OPENAI_API_KEY If you belong to multiple organizations or access projects through a legacy user API key, pass a header to specify which organization and project to use for an API request: 1 2 3 4 curl https://api.openai.com/v1/models \ -H "Authorization: Bearer $OPENAI_API_KEY" \ -H "OpenAI-Organization: $ORGANIZATION_ID" \ -H "OpenAI-Project: $PROJECT_ID" Usage from these API requests counts as usage for the specified organization and project.Organization IDs can be found on your organization settings page. Project IDs can be found on your general settings page by selecting the specific project.Debugging requests In addition to error codes returned from API responses, you can inspect HTTP response headers containing the unique ID of a particular API request or information about rate limiting applied to your requests. Below is an incomplete list of HTTP headers returned with API responses: API meta information openai-organization: The organization associated with the request openai-processing-ms: Time taken processing your API request openai-version: REST API version used for this request (currently 2020-10-01) x-request-id: Unique identifier for this API request (used in troubleshooting) Rate limiting information x-ratelimit-limit-requests x-ratelimit-limit-tokens x-ratelimit-remaining-requests x-ratelimit-remaining-tokens x-ratelimit-reset-requests x-ratelimit-reset-tokens OpenAI recommends logging request IDs in production deployments for more efficient troubleshooting with our support team, should the need arise. Our official SDKs provide a property on top-level response objects containing the value of the x-request-id header. Supplying your own request ID with X-Client-Request-Id In addition to the server-generated x-request-id, you can supply your own unique identifier for each request via the X-Client-Request-Id request header. This header is not added automatically; you must explicitly set it on the request. When you include X-Client-Request-Id: You control the ID format (for example, a UUID or your internal trace ID), but it must contain only ASCII characters and be no more than 512 characters long; otherwise, the request will fail with a 400 error. We strongly recommend making this value unique per request. OpenAI will log this value in our internal logs for supported endpoints, including chat/completions, embeddings, responses, and more. In cases like timeouts or network issues when you can’t get the X-Request-Id response header, you can share the X-Client-Request-Id value with our support team, and we can look up whether we received the request and when. Example: 1 2 3 curl https://api.openai.com/v1/chat/completions \ -H "Authorization: Bearer $OPENAI_API_KEY" \ -H "X-Client-Request-Id: 123e4567-e89b-12d3-a456-426614174000"Backward compatibility OpenAI is committed to providing stability to API users by avoiding breaking changes in major API versions whenever reasonably possible. This includes: The REST API (currently v1) Our first-party SDKs (released SDKs adhere to semantic versioning) Model families (like gpt-4o or o4-mini) Model prompting behavior between snapshots is subject to change. Model outputs are by their nature variable, so expect changes in prompting and model behavior between snapshots. For example, if you moved from gpt-4o-2024-05-13 to gpt-4o-2024-08-06, the same system or user messages could function differently between versions. The best way to ensure consistent prompting behavior and model output is to use pinned model versions, and to implement evals for your applications. Backwards-compatible API changes: Adding new resources (URLs) to the REST API and SDKs Adding new optional API parameters Adding new properties to JSON response objects or event data Changing the order of properties in a JSON response object Changing the length or format of opaque strings, like resource identifiers and UUIDs Adding new event types (in either streaming or the Realtime API) See the changelog for a list of backwards-compatible changes and rare breaking changes.NextResponses

Authorization: Bearer OPENAI_API_KEY

Pattern 2: Using GPT-5.2Learn best practices, features, and migration guidance for GPT-5.2 and the GPT-5 model family.Copy pageGPT-5.2 is our best general-purpose model, part of the GPT-5 flagship model family. Our most intelligent model yet for both general and agentic tasks, GPT-5.2 shows improvements over the previous GPT-5.1 in: General intelligence Instruction following Accuracy and token efficiency Multimodality—especially vision Code generation—especially front-end UI creation Tool calling and context management in the API Spreadsheet understanding and creation Unlike the previous GPT-5.1 model, GPT-5.2 has new features for managing what the model "knows" and "remembers to improve accuracy. This guide covers key features of the GPT-5 model family and how to get the most out of GPT-5.2. Explore coding examples Click through a few demo applications generated entirely with a single prompt, without writing any code by hand. Note that these examples were either generated by GPT-5.2 or our previous flagship model, GPT-5. Explore Meet the models There are three new models. In general, gpt-5.2 is best for your most complex tasks that require broad world knowledge. It replaces the previous gpt-5.1 model. The model powering ChatGPT is gpt-5.2-chat-latest. Third, gpt-5.2-pro uses more compute to think harder and provide consistently better answers. For a smaller model, use gpt-5-mini. To help you pick the model that best fits your use case, consider these tradeoffs: VariantBest forgpt-5.2Complex reasoning, broad world knowledge, and code-heavy or multi-step agentic tasksgpt-5.2-proTough problems that may take longer to solve but require harder thinkinggpt-5.1-codex-maxCompanies building interactive coding products; full spectrum of coding tasksgpt-5-miniCost-optimized reasoning and chat; balances speed, cost, and capabilitygpt-5-nanoHigh-throughput tasks, especially simple instruction-following or classification New features in GPT-5.2 Just like GPT-5.1, the new GPT-5.2 has API features like custom tools, parameters to control verbosity and reasoning, and an allowed tools list. What's new in 5.2 is a new xhigh reasoning effort level, concise reasoning summaries, and new context management using compaction. This guide walks through some of the key features of the GPT-5 model family and how to get the most out of 5.2 in particular. For coding tasks, GPT-5.1-Codex-Max is a faster, more capable, and more token-efficient coding variant, with an xhigh reasoning option. Its new built-in compaction capability provides native long-running task support. Lower reasoning effort The reasoning.effort parameter controls how many reasoning tokens the model generates before producing a response. Earlier reasoning models like o3 supported only low, medium, and high: low favored speed and fewer tokens, while high favored more thorough reasoning. With GPT-5.2, the lowest setting is none to provide lower-latency interactions. This is the default setting in GPT-5.2. If you need more thinking, slowly increase to medium and experiment with results. With reasoning effort set to none, prompting is important. To improve the model's reasoning quality, even with the default settings, encourage it to “think” or outline its steps before answering. Minimal reasoning effortpython1 2 3 4 5 6 7 curl --request POST --url https://api.openai.com/v1/responses --header "Authorization: Bearer $OPENAI_API_KEY" --header 'Content-type: application/json' --data '{ "model": "gpt-5.1", "input": "How much gold would it take to coat the Statue of Liberty in a 1mm layer?", "reasoning": { "effort": "none" } }'1 2 3 4 5 6 7 8 9 10 11 12 import OpenAI from "openai"; const openai = new OpenAI(); const response = await openai.responses.create({ model: "gpt-5.1", input: "How much gold would it take to coat the Statue of Liberty in a 1mm layer?", reasoning: { effort: "none" } }); console.log(response);1 2 3 4 5 6 7 8 9 10 11 12 from openai import OpenAI client = OpenAI() response = client.responses.create( model="gpt-5.1", input="How much gold would it take to coat the Statue of Liberty in a 1mm layer?", reasoning={ "effort": "none" } ) print(response) Verbosity Verbosity determines how many output tokens are generated. Lowering the number of tokens reduces overall latency. While the model's reasoning approach stays mostly the same, the model finds ways to answer more concisely—which can either improve or diminish answer quality, depending on your use case. Here are some scenarios for both ends of the verbosity spectrum: High verbosity: Use when you need the model to provide thorough explanations of documents or perform extensive code refactoring. Low verbosity: Best for situations where you want concise answers or simple code generation, such as SQL queries. GPT-5 made this option configurable as one of high, medium, or low. With GPT-5.2, verbosity remains configurable and defaults to medium. When generating code with GPT-5.2, medium and high verbosity levels yield longer, more structured code with inline explanations, while low verbosity produces shorter, more concise code with minimal commentary. Control verbositypython1 2 3 4 5 6 7 curl --request POST --url https://api.openai.com/v1/responses --header "Authorization: Bearer $OPENAI_API_KEY" --header 'Content-type: application/json' --data '{ "model": "gpt-5", "input": "What is the answer to the ultimate question of life, the universe, and everything?", "text": { "verbosity": "low" } }'1 2 3 4 5 6 7 8 9 10 11 12 import OpenAI from "openai"; const openai = new OpenAI(); const response = await openai.responses.create({ model: "gpt-5", input: "What is the answer to the ultimate question of life, the universe, and everything?", text: { verbosity: "low" } }); console.log(response);1 2 3 4 5 6 7 8 9 10 11 12 from openai import OpenAI client = OpenAI() response = client.responses.create( model="gpt-5", input="What is the answer to the ultimate question of life, the universe, and everything?", text={ "verbosity": "low" } ) print(response) You can still steer verbosity through prompting after setting it to low in the API. The verbosity parameter defines a general token range at the system prompt level, but the actual output is flexible to both developer and user prompts within that range. Using tools with GPT-5.2 GPT-5.2 has been post-trained on specific tools. See the tools docs for more specific guidance. The apply patch tool The apply_patch tool lets GPT-5.2 create, update, and delete files in your codebase using structured diffs. Instead of just suggesting edits, the model emits patch operations that your application applies and then reports back on, enabling iterative, multistep code editing workflows. Read the docs. Under the hood, this implementation uses a freeform function call rather than a JSON format. In testing, the named function decreased apply_patch failure rates by 35%. Shell tool Local shell is supported in GPT-5.2. The shell tool allows the model to interact with your local computer through a controlled command-line interface. Read the docs to learn more. Custom tools When the GPT-5 model family launched, we introduced a new capability called custom tools, which lets models send any raw text as tool call input but still constrain outputs if desired. This tool behavior remains true in GPT-5.2. Function calling guideLearn about custom tools in the function calling guide. Freeform inputs Define your tool with type: custom to enable models to send plaintext inputs directly to your tools, rather than being limited to structured JSON. The model can send any raw text—code, SQL queries, shell commands, configuration files, or long-form prose—directly to your tool. 1 2 3 4 5 { "type": "custom", "name": "code_exec", "description": "Executes arbitrary python code", } Constraining outputs GPT-5.2 supports context-free grammars (CFGs) for custom tools, letting you provide a Lark grammar to constrain outputs to a specific syntax or DSL. Attaching a CFG (e.g., a SQL or DSL grammar) ensures the assistant's text matches your grammar. This enables precise, constrained tool calls or structured responses and lets you enforce strict syntactic or domain-specific formats directly in GPT-5.2's function calling, improving control and reliability for complex or constrained domains. Best practices for custom tools Write concise, explicit tool descriptions. The model chooses what to send based on your description; state clearly if you want it to always call the tool. Validate outputs on the server side. Freeform strings are powerful but require safeguards against injection or unsafe commands. Allowed tools The allowed_tools parameter under tool_choice lets you pass N tool definitions but restrict the model to only M (< N) of them. List your full toolkit in tools, and then use an allowed_tools block to name the subset and specify a mode—either auto (the model may pick any of those) or required (the model must invoke one). Function calling guideLearn about the allowed tools option in the function calling guide. By separating all possible tools from the subset that can be used now, you gain greater safety, predictability, and improved prompt caching. You also avoid brittle prompt engineering, such as hard-coded call order. GPT-5.2 dynamically invokes or requires specific functions mid-conversation while reducing the risk of unintended tool usage over long contexts. Standard ToolsAllowed ToolsModel's universeAll tools listed under "tools": […]Only the subset under "tools": […] in tool_choiceTool invocationModel may or may not call any toolModel restricted to (or required to call) chosen toolsPurposeDeclare available capabilitiesConstrain which capabilities are actually used 1 2 3 4 5 6 7 8 9 "tool_choice": { "type": "allowed_tools", "mode": "auto", "tools": [ { "type": "function", "name": "get_weather" }, { "type": "function", "name": "search_docs" } ] } }' For a more detailed overview of all of these new features, see the accompanying cookbook. Preambles Preambles are brief, user-visible explanations that GPT-5.2 generates before invoking any tool or function, outlining its intent or plan (e.g., “why I'm calling this tool”). They appear after the chain-of-thought and before the actual tool call, providing transparency into the model's reasoning and enhancing debuggability, user confidence, and fine-grained steerability. By letting GPT-5.2 “think out loud” before each tool call, preambles boost tool-calling accuracy (and overall task success) without bloating reasoning overhead. To enable preambles, add a system or developer instruction—for example: “Before you call a tool, explain why you are calling it.” GPT-5.2 prepends a concise rationale to each specified tool call. The model may also output multiple messages between tool calls, which can enhance the interaction experience—particularly for minimal reasoning or latency-sensitive use cases. For more on using preambles, see the GPT-5 prompting cookbook. Migration guidance GPT-5.2 is our best model yet, and it works best with the Responses API, which supports for passing chain of thought (CoT) between turns. Read below to migrate from your current model or API. Migrating from other models to GPT-5.2 While the model should be close to a drop-in replacement for GPT-5.1, there are a few key changes to call out. See the GPT-5.2 prompting guide for specific updates to make in your prompts. Using GPT-5 models with the Responses API provides improved intelligence because of the API's design. The Responses API can pass the previous turn's CoT to the model. This leads to fewer generated reasoning tokens, higher cache hit rates, and less latency. To learn more, see an in-depth guide on the benefits of the Responses API. When migrating to GPT-5.2 from an older OpenAI model, start by experimenting with reasoning levels and prompting strategies. Based on our testing, we recommend using our prompt optimizer—which automatically updates your prompts for GPT-5.2 based on our best practices—and following this model-specific guidance: gpt-5.1: gpt-5.2 with default settings is meant to be a drop-in replacement. o3: gpt-5.2 with medium or high reasoning. Start with medium reasoning with prompt tuning, then increase to high if you aren't getting the results you want. gpt-4.1: gpt-5.2 with none reasoning. Start with none and tune your prompts; increase if you need better performance. o4-mini or gpt-4.1-mini: gpt-5-mini with prompt tuning is a great replacement. gpt-4.1-nano: gpt-5-nano with prompt tuning is a great replacement. GPT-5.2 parameter compatibility The following parameters are only supported when using GPT-5.2 with reasoning effort set to none: temperature top_p logprobs Requests to GPT-5.2 or GPT-5.1 with any other reasoning effort setting, or to older GPT-5 models (e.g., gpt-5, gpt-5-mini, gpt-5-nano) that include these fields will raise an error. To achieve similar results with reasoning effort set higher, or with another GPT-5 family model, try these alternative parameters: Reasoning depth: reasoning: { effort: "none" | "low" | "medium" | "high" | "xhigh" } Output verbosity: text: { verbosity: "low" | "medium" | "high" } Output length: max_output_tokens Migrating from Chat Completions to Responses API The biggest difference, and main reason to migrate from Chat Completions to the Responses API for GPT-5.2, is support for passing chain of thought (CoT) between turns. See a full comparison of the APIs. Passing CoT exists only in the Responses API, and we've seen improved intelligence, fewer generated reasoning tokens, higher cache hit rates, and lower latency as a result of doing so. Most other parameters remain at parity, though the formatting is different. Here's how new parameters are handled differently between Chat Completions and the Responses API: Reasoning effort Responses APIChat CompletionsResponses APIGenerate response with minimal reasoning1 2 3 4 5 6 7 8 9 10 11 curl --request POST \ --url https://api.openai.com/v1/responses \ --header "Authorization: Bearer $OPENAI_API_KEY" \ --header 'Content-type: application/json' \ --data '{ "model": "gpt-5.2", "input": "How much gold would it take to coat the Statue of Liberty in a 1mm layer?", "reasoning": { "effort": "none" } }'Chat CompletionsGenerate response with minimal reasoning1 2 3 4 5 6 7 8 9 10 11 12 13 14 curl --request POST \ --url https://api.openai.com/v1/chat/completions \ --header "Authorization: Bearer $OPENAI_API_KEY" \ --header 'Content-type: application/json' \ --data '{ "model": "gpt-5.2", "messages": [ { "role": "user", "content": "How much gold would it take to coat the Statue of Liberty in a 1mm layer?" } ], "reasoning_effort": "none" }' Verbosity Responses APIChat CompletionsResponses APIControl verbosity1 2 3 4 5 6 7 8 9 10 11 curl --request POST \ --url https://api.openai.com/v1/responses \ --header "Authorization: Bearer $OPENAI_API_KEY" \ --header 'Content-type: application/json' \ --data '{ "model": "gpt-5.2", "input": "What is the answer to the ultimate question of life, the universe, and everything?", "text": { "verbosity": "low" } }'Chat CompletionsControl verbosity1 2 3 4 5 6 7 8 9 10 11 curl --request POST \ --url https://api.openai.com/v1/chat/completions \ --header "Authorization: Bearer $OPENAI_API_KEY" \ --header 'Content-type: application/json' \ --data '{ "model": "gpt-5.2", "messages": [ { "role": "user", "content": "What is the answer to the ultimate question of life, the universe, and everything?" } ], "verbosity": "low" }' Custom tools Responses APIChat CompletionsResponses APICustom tool call1 2 3 4 5 6 7 8 9 10 11 curl --request POST --url https://api.openai.com/v1/responses --header "Authorization: Bearer $OPENAI_API_KEY" --header 'Content-type: application/json' --data '{ "model": "gpt-5.2", "input": "Use the code_exec tool to calculate the area of a circle with radius equal to the number of r letters in blueberry", "tools": [ { "type": "custom", "name": "code_exec", "description": "Executes arbitrary python code" } ] }'Chat CompletionsCustom tool call1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 curl --request POST --url https://api.openai.com/v1/chat/completions --header "Authorization: Bearer $OPENAI_API_KEY" --header 'Content-type: application/json' --data '{ "model": "gpt-5.2", "messages": [ { "role": "user", "content": "Use the code_exec tool to calculate the area of a circle with radius equal to the number of r letters in blueberry" } ], "tools": [ { "type": "custom", "custom": { "name": "code_exec", "description": "Executes arbitrary python code" } } ] }' Prompting guidance We specifically designed GPT-5.2 to excel at coding and agentic tasks. We also recommend iterating on prompts for GPT-5.2 using the prompt optimizer. GPT-5.2 prompt optimizerCraft the perfect prompt for GPT-5.2 in the dashboard GPT-5.2 prompting guideLearn full best practices for prompting GPT-5 models Frontend prompting for GPT-5See prompt samples specific to frontend development for GPT-5 family of models GPT-5.2 is a reasoning model Reasoning models like GPT-5.2 break problems down step by step, producing an internal chain of thought that encodes their reasoning. To maximize performance, pass these reasoning items back to the model: this avoids re-reasoning and keeps interactions closer to the model's training distribution. In multi-turn conversations, passing a previous_response_id automatically makes earlier reasoning items available. This is especially important when using tools—for example, when a function call requires an extra round trip. In these cases, either include them with previous_response_id or add them directly to input. Learn more about reasoning models and how to get the most out of them in our reasoning guide. Further reading GPT-5.2 prompting guide GPT-5.1-Codex-Max prompting guide GPT-5.2 blog post GPT-5 frontend guide GPT-5 model family: new features guide Cookbook on reasoning models Comparison of Responses API vs. Chat Completions FAQ How are these models integrated into ChatGPT? In ChatGPT, there are three models: GPT‑5.2 Instant, GPT‑5.2 Thinking, and GPT-5.2 Pro. Based on the user's question, a routing layer selects the best model to use. Users can also invoke reasoning directly through the ChatGPT UI. All three ChatGPT models (Instant, Thinking, and Pro) have a new knowledge cutoff of August 2025. For users, this means GPT-5.2 starts with a more current understanding of the world, so answers are more accurate and useful, with more relevant examples and context, even before turning to web search. Will these models be supported in Codex? Yes, gpt-5.1-codex-max is the model that powers Codex and Codex CLI. You can also use this as a standalone model for building agentic coding applications. How does GPT-5.2 compare to GPT-5.1-Codex-Max? GPT-5.1-Codex-Max was specifically designed for use in Codex. Unlike GPT-5.2, which is a general-purpose model, we recommend using GPT-5.1-Codex-Max only for agentic coding tasks in Codex or Codex-like environments, and GPT-5.2 for use cases in other domains. GPT-5.1-Codex-Max is only available in the Responses API and supports none, medium, high, and xhigh reasoning effort settings as well function calling, structured outputs, compaction, and the web_search tool. What is the deprecation plan for previous models? Any model deprecations will be posted on our deprecations page. We'll send advanced notice of any model deprecations. OverviewOverviewQuickstartQuickstartMeet the modelsMeet the modelsNew features in GPT-5.1New features in GPT-5.1Lower reasoning effortLower reasoning effortVerbosityVerbosityNew tool typesNew tool typesCustom toolsCustom toolsAllowed toolsAllowed toolsPreamblesPreamblesMigration guidanceMigration guidancePrompting guidancePrompting guidanceFurther readingFurther readingFAQFAQPrompting guideGPT-5 best practices

gpt-5.2

Pattern 3: GradersLearn about graders used for evals and fine-tuning.Copy pageGraders are a way to evaluate your model's performance against reference answers. Our graders API is a way to test your graders, experiment with results, and improve your fine-tuning or evaluation framework to get the results you want. Overview Graders let you compare reference answers to the corresponding model-generated answer and return a grade in the range from 0 to 1. It's sometimes helpful to give the model partial credit for an answer, rather than a binary 0 or 1. Graders are specified in JSON format, and there are several types: String check Text similarity Score model grader Python code execution In reinforcement fine-tuning, you can nest and combine graders by using multigraders. Use this guide to learn about each grader type and see starter examples. To build a grader and get started with reinforcement fine-tuning, see the RFT guide. Or to get started with evals, see the Evals guide. Templating The inputs to certain graders use a templating syntax to grade multiple examples with the same configuration. Any string with {{ }} double curly braces will be substituted with the variable value. Each input inside the {{}} must include a namespace and a variable with the following format {{ namespace.variable }}. The only supported namespaces are item and sample. All nested variables can be accessed with JSON path like syntax. Item namespace The item namespace will be populated with variables from the input data source for evals, and from each dataset item for fine-tuning. For example, if a row contains the following 1 2 3 { "reference_answer": "..." } This can be used within the grader as {{ item.reference_answer }}. Sample namespace The sample namespace will be populated with variables from the model sampling step during evals or during the fine-tuning step. The following variables are included output_text, the model output content as a string. output_json, the model output content as a JSON object, only if response_format is included in the sample. output_tools, the model output tool_calls, which have the same structure as output tool calls in the chat completions API. choices, the output choices, which has the same structure as output choices in the chat completions API. output_audio, the model audio output object containing Base64-encoded data and a transcript. For example, to access the model output content as a string, {{ sample.output_text }} can be used within the grader. Details on grading tool callsWhen training a model to improve tool-calling behavior, you will need to write your grader to operate over the sample.output_tools variable. The contents of this variable will be the same as the contents of the response.choices[0].message.tool_calls (see function calling docs).A common way of grading tool calls is to use two graders, one that checks the name of the tool that is called and another that checks the arguments of the called function. An example of a grader that does this is shown below:1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 { "type": "multi", "graders": { "function_name": { "name": "function_name", "type": "string_check", "input": "get_acceptors", "reference": "{{sample.output_tools[0].function.name}}", "operation": "eq", }, "arguments": { "name": "arguments", "type": "string_check", "input": "{"smiles": "{{item.smiles}}"}", "reference": "{{sample.output_tools[0].function.arguments}}", "operation": "eq", }, }, "calculate_output": "0.5 * function_name + 0.5 * arguments", }This is a multi grader that combined two simple string_check graders, the first checks the name of the tool called via the sample.output_tools[0].function.name variable, and the second checks the arguments of the called function via the sample.output_tools[0].function.arguments variable. The calculate_output field is used to combine the two scores into a single score.The arguments grader is prone to under-rewarding the model if the function arguments are subtly incorrect, like if 1 is submitted instead of the floating point 1.0, or if a state name is given as an abbreviation instead of spelling it out. To avoid this, you can use a text_similarity grader instead of a string_check grader, or a score_model grader to have a LLM check for semantic similarity. String check grader Use these simple string operations to return a 0 or 1. String check graders are good for scoring straightforward pass or fail answers—for example, the correct name of a city, a yes or no answer, or an answer containing or starting with the correct information. 1 2 3 4 5 6 7 { "type": "string_check", "name": string, "operation": "eq" | "ne" | "like" | "ilike", "input": string, "reference": string, } Operations supported for string-check-grader are: eq: Returns 1 if the input matches the reference (case-sensitive), 0 otherwise neq: Returns 1 if the input does not match the reference (case-sensitive), 0 otherwise like: Returns 1 if the input contains the reference (case-sensitive), 0 otherwise ilike: Returns 1 if the input contains the reference (not case-sensitive), 0 otherwise Text similarity grader Use text similarity graders when to evaluate how close the model-generated output is to the reference, scored with various evaluation frameworks. This is useful for open-ended text responses. For example, if your dataset contains reference answers from experts in paragraph form, it's helpful to see how close your model-generated answer is to that content, in numerical form. 1 2 3 4 5 6 7 8 { "type": "text_similarity", "name": string, "input": string, "reference": string, "pass_threshold": number, "evaluation_metric": "fuzzy_match" | "bleu" | "gleu" | "meteor" | "cosine" | "rouge_1" | "rouge_2" | "rouge_3" | "rouge_4" | "rouge_5" | "rouge_l" } Operations supported for string-similarity-grader are: fuzzy_match: Fuzzy string match between input and reference, using rapidfuzz bleu: Computes the BLEU score between input and reference gleu: Computes the Google BLEU score between input and reference meteor: Computes the METEOR score between input and reference cosine: Computes Cosine similarity between embedded input and reference, using text-embedding-3-large. Only available for evals. rouge-*: Computes the ROUGE score between input and reference Model graders In general, using a model grader means prompting a separate model to grade the outputs of the model you're fine-tuning. Your two models work together to do reinforcement fine-tuning. The grader model evaluates the training model. Score model graders A score model grader will take the input and return a numeric score based on the prompt within the given range. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 { "type": "score_model", "name": string, "input": Message[], "model": string, "pass_threshold": number, "range": number[], "sampling_params": { "seed": number, "top_p": number, "temperature": number, "max_completions_tokens": number, "reasoning_effort": "minimal" | "low" | "medium" | "high" } } Where each message is of the following form: 1 2 3 4 { "role": "system" | "developer" | "user" | "assistant", "content": str } To use a score model grader, the input is a list of chat messages, each containing a role and content. The output of the grader will be truncated to the given range, and default to 0 for all non-numeric outputs. Within each message, the same templating can be used as with other common graders to reference the ground truth or model sample. Here’s a full runnable code sample: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 import os import requests # get the API key from environment api_key = os.environ["OPENAI_API_KEY"] headers = {"Authorization": f"Bearer {api_key}"} # define a dummy grader for illustration purposes grader = { "type": "score_model", "name": "my_score_model", "input": [ { "role": "system", "content": "You are an expert grader. If the reference and model answer are exact matches, output a score of 1. If they are somewhat similar in meaning, output a score in 0.5. Otherwise, give a score of 0." }, { "role": "user", "content": "Reference: {{ item.reference_answer }}. Model answer: {{ sample.output_text }}" } ], "pass_threshold": 0.5, "model": "o4-mini-2025-04-16", "range": [0, 1], "sampling_params": { "max_completions_tokens": 32768, "top_p": 1, "reasoning_effort": "medium" }, } # validate the grader payload = {"grader": grader} response = requests.post( "https://api.openai.com/v1/fine_tuning/alpha/graders/validate", json=payload, headers=headers ) print("validate response:", response.text) # run the grader with a test reference and sample payload = { "grader": grader, "item": { "reference_answer": 1.0 }, "model_sample": "0.9" } response = requests.post( "https://api.openai.com/v1/fine_tuning/alpha/graders/run", json=payload, headers=headers ) print("run response:", response.text) Score model grader outputs Under the hood, the score_model grader will query the requested model with the provided prompt and sampling parameters and will request a response in a specific response format. The response format that is used is provided below 1 2 3 4 { "result": float, "steps": ReasoningStep[], } Where each reasoning step is of the form 1 2 3 4 { description: string, conclusion: string } This format queries the model not just for the numeric result (the reward value for the query), but also provides the model some space to think through the reasoning behind the score. When you are writing your grader prompt, it may be useful to refer to these two fields by name explicitly (e.g. "include reasoning about the type of chemical bonds present in the molecule in the conclusion of your reasoning step", or "return a value of -1.0 in the result field if the inputs do not satisfy condition X"). Model grader constraints Only the following models are supported for the model parameter` gpt-4o-2024-08-06 gpt-4o-mini-2024-07-18 gpt-4.1-2025-04-14 gpt-4.1-mini-2025-04-14 gpt-4.1-nano-2025-04-14 o1-2024-12-17 o3-mini-2025-01-31 o3-2025-04-16 o4-mini-2025-04-16 temperature changes not supported for reasoning models. reasoning_effort is not supported for non-reasoning models. How to write grader prompts Writing grader prompts is an iterative process. The best way to iterate on a model grader prompt is to create a model grader eval. To do this, you need: Task prompts: Write extremely detailed prompts for the desired task, with step-by-step instructions and many specific examples in context. Answers generated by a model or human expert: Provide many high quality examples of answers, both from the model and trusted human experts. Corresponding ground truth grades for those answers: Establish what a good grade looks like. For example, your human expert grades should be 1. Then you can automatically evaluate how effectively the model grader distinguishes answers of different quality levels. Over time, add edge cases into your model grader eval as you discover and patch them with changes to the prompt. For example, say you know from your human experts which answers are best: answer_1 > answer_2 > answer_3 Verify that the model grader's answers match that: model_grader(answer_1, reference_answer) > model_grader(answer_2, reference_answer) > model_grader(answer_3, reference_answer) Grader hacking Models being trained sometimes learn to exploit weaknesses in model graders, also known as “grader hacking” or “reward hacking." You can detect this by checking the model's performance across model grader evals and expert human evals. A model that's hacked the grader will score highly on model grader evals but score poorly on expert human evaluations. Over time, we intend to improve observability in the API to make it easier to detect this during training. Python graders This grader allows you to execute arbitrary python code to grade the model output. The grader expects a grade function to be present that takes in two arguments and outputs a float value. Any other result (exception, invalid float value, etc.) will be marked as invalid and return a 0 grade. 1 2 3 4 5 { "type": "python", "source": "def grade(sample, item):\n return 1.0", "image_tag": "2025-05-08" } The python source code must contain a grade function that takes in exactly two arguments and returns a float value as a grade. 1 2 3 4 5 from typing import Any def grade(sample: dict[str, Any], item: dict[str, Any]) -> float: # your logic here return 1.0 The first argument supplied to the grading function will be a dictionary populated with the model’s output during training for you to grade. output_json will only be populated if the output uses response_format. 1 2 3 4 5 6 7 { "choices": [...], "output_text": "...", "output_json": {}, "output_tools": [...], "output_audio": {} } The second argument supplied is a dictionary populated with input grading context. For evals, this will include keys from the data source. For fine-tuning this will include keys from each training data row. 1 2 3 4 { "reference_answer": "...", "my_key": {...} } Here's a working example: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 import os import requests # get the API key from environment api_key = os.environ["OPENAI_API_KEY"] headers = {"Authorization": f"Bearer {api_key}"} grading_function = """ from rapidfuzz import fuzz, utils def grade(sample, item) -> float: output_text = sample["output_text"] reference_answer = item["reference_answer"] return fuzz.WRatio(output_text, reference_answer, processor=utils.default_process) / 100.0 """ # define a dummy grader for illustration purposes grader = { "type": "python", "source": grading_function } # validate the grader payload = {"grader": grader} response = requests.post( "https://api.openai.com/v1/fine_tuning/alpha/graders/validate", json=payload, headers=headers ) print("validate request_id:", response.headers["x-request-id"]) print("validate response:", response.text) # run the grader with a test reference and sample payload = { "grader": grader, "item": { "reference_answer": "fuzzy wuzzy had no hair" }, "model_sample": "fuzzy wuzzy was a bear" } response = requests.post( "https://api.openai.com/v1/fine_tuning/alpha/graders/run", json=payload, headers=headers ) print("run request_id:", response.headers["x-request-id"]) print("run response:", response.text) Tip: If you don't want to manually put your grading function in a string, you can also load it from a Python file using importlib and inspect. For example, if your grader function is in a file named grader.py, you can do: 1 2 3 4 5 6 7 8 import importlib import inspect grader_module = importlib.import_module("grader") grader = { "type": "python", "source": inspect.getsource(grader_module) } This will automatically use the entire source code of your grader.py file as the grader which can be helpful for longer graders. Technical constraints Your uploaded code must be less than 256kB and will not have network access. The grading execution itself is limited to 2 minutes. At runtime you will be given a limit of 2Gb of memory and 1Gb of disk space to use. There's a limit of 2 CPU cores—any usage above this amount will result in throttling The following third-party packages are available at execution time for the image tag 2025-05-08 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 numpy==2.2.4 scipy==1.15.2 sympy==1.13.3 pandas==2.2.3 rapidfuzz==3.10.1 scikit-learn==1.6.1 rouge-score==0.1.2 deepdiff==8.4.2 jsonschema==4.23.0 pydantic==2.10.6 pyyaml==6.0.2 nltk==3.9.1 sqlparse==0.5.3 rdkit==2024.9.6 scikit-bio==0.6.3 ast-grep-py==0.36.2 Additionally the following nltk corpora are available: 1 2 3 4 5 punkt stopwords wordnet omw-1.4 names Multigraders Currently, this grader is only used for Reinforcement fine-tuning A multigrader object combines the output of multiple graders to produce a single score. Multigraders work by computing grades over the fields of other grader objects and turning those sub-grades into an overall grade. This is useful when a correct answer depends on multiple things being true—for example, that the text is similar and that the answer contains a specific string. As an example, say you wanted the model to output JSON with the following two fields: 1 2 3 4 { "name": "John Doe", "email": "john.doe@gmail.com" } You'd want your grader to compare the two fields and then take the average between them. You can do this by combining multiple graders into an object grader, and then defining a formula to calculate the output score based on each field: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 { "type": "multi", "graders": { "name": { "name": "name_grader", "type": "text_similarity", "input": "{{sample.output_json.name}}", "reference": "{{item.name}}", "evaluation_metric": "fuzzy_match", "pass_threshold": 0.9 }, "email": { "name": "email_grader", "type": "string_check", "input": "{{sample.output_json.email}}", "reference": "{{item.email}}", "operation": "eq" } }, "calculate_output": "(name + email) / 2" } In this example, it’s important for the model to get the email exactly right (string_check returns either 0 or 1) but we tolerate some misspellings on the name (text_similarity returns range from 0 to 1). Samples that get the email wrong will score between 0-0.5, and samples that get the email right will score between 0.5-1.0. You cannot create a multigrader with a nested multigrader inside. The calculate output field will have the keys of the input graders as possible variables and the following features are supported: Operators + (addition) - (subtraction) * (multiplication) / (division) ^ (power) Functions min max abs floor ceil exp sqrt log Limitations and tips Designing and creating graders is an iterative process. Start small, experiment, and continue to make changes to get better results. Design tips To get the most value from your graders, use these design principles: Produce a smooth score, not a pass/fail stamp. A score that shifts gradually as answers improve helps the optimizer see which changes matter. Guard against reward hacking. This happens when the model finds a shortcut that earns high scores without real skill. Make it hard to loophole your grading system. Avoid skewed data. Datasets in which one label shows up most of the time invite the model to guess that label. Balance the set or up‑weight rare cases so the model must think. Use an LLM‑as‑a-judge when code falls short. For rich, open‑ended answers, ask another language model to grade. When building LLM graders, run multiple candidate responses and ground truths through your LLM judge to ensure grading is stable and aligned with preference. Provide few-shot examples of great, fair, and poor answers in the prompt. OverviewOverviewTemplatingTemplatingString check graderString check graderText similarity graderText similarity graderModel gradersModel gradersPython gradersPython gradersMultigradersMultigraders

{{ }}

Pattern 4: Reinforcement fine-tuningFine-tune models for expert-level performance within a domain.Copy pageReinforcement fine-tuning (RFT) adapts an OpenAI reasoning model with a feedback signal you define. Like supervised fine-tuning, it tailors the model to your task. The difference is that instead of training on fixed “correct” answers, it relies on a programmable grader that scores every candidate response. The training algorithm then shifts the model’s weights, so high-scoring outputs become more likely and low-scoring ones fade. How it worksBest forUse withGenerate a response for a prompt, provide an expert grade for the result, and reinforce the model's chain-of-thought for higher-scored responses.Requires expert graders to agree on the ideal output from the model. Complex domain-specific tasks that require advanced reasoning Medical diagnoses based on history and diagnostic guidelines Determining relevant passages from legal case law o4-mini-2025-04-16Reasoning models only. This optimization lets you align the model with nuanced objectives like style, safety, or domain accuracy—with many practical use cases emerging. Run RFT in five steps: Implement a grader that assigns a numeric reward to each model response. Upload your prompt dataset and designate a validation split. Start the fine-tune job. Monitor and evaluate checkpoints; revise data or grader if needed. Deploy the resulting model through the standard API. During training, the platform cycles through the dataset, samples several responses per prompt, scores them with the grader, and applies policy-gradient updates based on those rewards. The loop continues until we hit the end of your training data or you stop the job at a chosen checkpoint, producing a model optimized for the metric that matters to you. When should I use reinforcement fine-tuning?It's useful to understand the strengths and weaknesses of reinforcement fine-tuning to identify opportunities and to avoid wasted effort. RFT works best with unambiguous tasks. Check whether qualified human experts agree on the answers. If conscientious experts working independently (with access only to the same instructions and information as the model) do not converge on the same answers, the task may be too ambiguous and may benefit from revision or reframing. Your task must be compatible with the grading options. Review grading options in the API first and verify it's possible to grade your task with them. Your eval results must be variable enough to improve. Run evals before using RFT. If your eval scores between minimum and maximum possible scores, you'll have enough data to work with to reinforce positive answers. If the model you want to fine-tune scores at either the absolute minimum or absolute maximum score, RFT won't be useful to you. Your model must have some success at the desired task. Reinforcement fine-tuning makes gradual changes, sampling many answers and choosing the best ones. If a model has a 0% success rate at a given task, you cannot bootstrap to higher performance levels through RFT. Your task should be guess-proof. If the model can get a higher reward from a lucky guess, the training signal is too noisy, as the model can get the right answer with an incorrect reasoning process. Reframe your task to make guessing more difficult—for example, by expanding classes into subclasses or revising a multiple choice problem to take open-ended answers. See common use cases, specific implementations, and grader examples in the reinforcement fine-tuning use case guide. What is reinforcement learning?Reinforcement learning is a branch of machine learning in which a model learns by acting, receiving feedback, and readjusting itself to maximise future feedback. Instead of memorising one “right” answer per example, the model explores many possible answers, observes a numeric reward for each, and gradually shifts its behaviour so the high-reward answers become more likely and the low-reward ones disappear. Over repeated rounds, the model converges on a policy—a rule for choosing outputs—that best satisfies the reward signal you define.In reinforcement fine-tuning (RFT), that reward signal comes from a custom grader that you define for your task. For every prompt in your dataset, the platform samples multiple candidate answers, runs your grader to score them, and applies a policy-gradient update that nudges the model toward answers with higher scores. This cycle—sample, grade, update—continues across the dataset (and successive epochs) until the model reliably optimizes for your grader’s understanding of quality. The grader encodes whatever you care about—accuracy, style, safety, or any metric—so the resulting fine-tuned model reflects those priorities and you don't have to manage reinforcement learning infrastructure. Reinforcement fine-tuning is supported on o-series reasoning models only, and currently only for o4-mini. Example: LLM-powered security review To demonstrate reinforcement fine-tuning below, we'll fine-tune an o4-mini model to provide expert answers about a fictional company's security posture, based on an internal company policy document. We want the model to return a JSON object that conforms to a specific schema with Structured Outputs. Example input question: Do you have a dedicated security team? Using the internal policy document, we want the model to respond with JSON that has two keys: compliant: A string yes, no, or needs review, indicating whether the company's policy covers the question. explanation: A string of text that briefly explains, based on the policy document, why the question is covered in the policy or why it's not covered. Example desired output from the model: 1 2 3 4 { "compliant": "yes", "explanation": "A dedicated security team follows strict protocols for handling incidents." } Let's fine-tune a model with RFT to perform well at this task. Define a grader To perform RFT, define a grader to score the model's output during training, indicating the quality of its response. RFT uses the same set of graders as evals, which you may already be familiar with. In this example, we define multiple graders to examine the properties of the JSON returned by our fine-tuned model: The string_check grader to ensure the proper compliant property has been set The score_model grader to provide a score between zero and one for the explanation text, using another evaluator model We weight the output of each property equally in the calculate_output expression. Below is the JSON payload data we'll use for this grader in API requests. In both graders, we use {{ }} template syntax to refer to the relevant properties of both the item (the row of test data being used for evaluation) and sample (the model output generated during the training run). Grader configurationGrading promptGrader configurationMulti-grader configuration object1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 { "type": "multi", "graders": { "explanation": { "name": "Explanation text grader", "type": "score_model", "input": [ { "role": "user", "type": "message", "content": "...see other tab for the full prompt..." } ], "model": "gpt-4o-2024-08-06" }, "compliant": { "name": "compliant", "type": "string_check", "reference": "{{item.compliant}}", "operation": "eq", "input": "{{sample.output_json.compliant}}" } }, "calculate_output": "0.5 * compliant + 0.5 * explanation" }Grading promptGrading prompt in the grader config1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 # Overview Evaluate the accuracy of the model-generated answer based on the Copernicus Product Security Policy and an example answer. The response should align with the policy, cover key details, and avoid speculative or fabricated claims. Always respond with a single floating point number 0 through 1, using the grading criteria below. ## Grading Criteria: - 1.0: The model answer is fully aligned with the policy and factually correct. - 0.75: The model answer is mostly correct but has minor omissions or slight rewording that does not change meaning. - 0.5: The model answer is partially correct but lacks key details or contains speculative statements. - 0.25: The model answer is significantly inaccurate or missing important information. - 0.0: The model answer is completely incorrect, hallucinates policy details, or is irrelevant. ## Copernicus Product Security Policy ### Introduction Protecting customer data is a top priority for Copernicus. Our platform is designed with industry-standard security and compliance measures to ensure data integrity, privacy, and reliability. ### Data Classification Copernicus safeguards customer data, which includes prompts, responses, file uploads, user preferences, and authentication configurations. Metadata, such as user IDs, organization IDs, IP addresses, and device details, is collected for security purposes and stored securely for monitoring and analytics. ### Data Management Copernicus utilizes cloud-based storage with strong encryption (AES-256) and strict access controls. Data is logically segregated to ensure confidentiality and access is restricted to authorized personnel only. Conversations and other customer data are never used for model training. ### Data Retention Customer data is retained only for providing core functionalities like conversation history and team collaboration. Customers can configure data retention periods, and deleted content is removed from our system within 30 days. ### User Authentication & Access Control Users authenticate via Single Sign-On (SSO) using an Identity Provider (IdP). Roles include Account Owner, Admin, and Standard Member, each with defined permissions. User provisioning can be automated through SCIM integration. ### Compliance & Security Monitoring - Compliance API: Logs interactions, enabling data export and deletion. - Audit Logging: Ensures transparency for security audits. - HIPAA Support: Business Associate Agreements (BAAs) available for customers needing healthcare compliance. - Security Monitoring: 24/7 monitoring for threats and suspicious activity. - Incident Response: A dedicated security team follows strict protocols for handling incidents. ### Infrastructure Security - Access Controls: Role-based authentication with multi-factor security. - Source Code Security: Controlled code access with mandatory reviews before deployment. - Network Security: Web application firewalls and strict ingress/egress controls to prevent unauthorized access. - Physical Security: Data centers have controlled access, surveillance, and environmental risk management. ### Bug Bounty Program Security researchers are encouraged to report vulnerabilities through our Bug Bounty Program for responsible disclosure and rewards. ### Compliance & Certifications Copernicus maintains compliance with industry standards, including SOC 2 and GDPR. Customers can access security reports and documentation via our Security Portal. ### Conclusion Copernicus prioritizes security, privacy, and compliance. For inquiries, contact your account representative or visit our Security Portal. ## Examples ### Example 1: GDPR Compliance Reference Answer: 'Copernicus maintains compliance with industry standards, including SOC 2 and GDPR. Customers can access security reports and documentation via our Security Portal.' Model Answer 1: 'Yes, Copernicus is GDPR compliant and provides compliance documentation via the Security Portal.' Score: 1.0 (fully correct) Model Answer 2: 'Yes, Copernicus follows GDPR standards.' Score: 0.75 (mostly correct but lacks detail about compliance reports) Model Answer 3: 'Copernicus may comply with GDPR but does not provide documentation.' Score: 0.5 (partially correct, speculative about compliance reports) Model Answer 4: 'Copernicus does not follow GDPR standards.' Score: 0.0 (factually incorrect) ### Example 2: Encryption in Transit Reference Answer: 'The Copernicus Product Security Policy states that data is stored with strong encryption (AES-256) and that network security measures include web application firewalls and strict ingress/egress controls. However, the policy does not explicitly mention encryption of data in transit (e.g., TLS encryption). A review is needed to confirm whether data transmission is encrypted.' Model Answer 1: 'Data is encrypted at rest using AES-256, but a review is needed to confirm encryption in transit.' Score: 1.0 (fully correct) Model Answer 2: 'Yes, Copernicus encrypts data in transit and at rest.' Score: 0.5 (partially correct, assumes transit encryption without confirmation) Model Answer 3: 'All data is protected with encryption.' Score: 0.25 (vague and lacks clarity on encryption specifics) Model Answer 4: 'Data is not encrypted in transit.' Score: 0.0 (factually incorrect) Reference Answer: {{item.explanation}} Model Answer: {{sample.output_json.explanation}} Prepare your dataset To create an RFT fine-tune, you'll need both a training and test dataset. Both the training and test datasets will share the same JSONL format. Each line in the JSONL data file will contain a messages array, along with any additional fields required to grade the output from the model. The full specification for RFT dataset can be found here. In our case, in addition to the messages array, each line in our JSONL file also needs compliant and explanation properties, which we can use as reference values to test the fine-tuned model's Structured Output. A single line in our training and test datasets looks like this as indented JSON: 1 2 3 4 5 6 7 8 { "messages": [{ "role": "user", "content": "Do you have a dedicated security team?" }], "compliant": "yes", "explanation": "A dedicated security team follows strict protocols for handling incidents." } Below, find some JSONL data you can use for both training and testing when you create your fine-tune job. Note that these datasets are for illustration purposes only—in your real test data, strive for diverse and representative inputs for your application. Training set 1 2 3 {"messages":[{"role":"user","content":"Do you have a dedicated security team?"}],"compliant":"yes","explanation":"A dedicated security team follows strict protocols for handling incidents."} {"messages":[{"role":"user","content":"Have you undergone third-party security audits or penetration testing in the last 12 months?"}],"compliant":"needs review","explanation":"The policy does not explicitly mention undergoing third-party security audits or penetration testing. It only mentions SOC 2 and GDPR compliance."} {"messages":[{"role":"user","content":"Is your software SOC 2, ISO 27001, or similarly certified?"}],"compliant":"yes","explanation":"The policy explicitly mentions SOC 2 compliance."} Test set 1 2 3 {"messages":[{"role":"user","content":"Will our data be encrypted at rest?"}],"compliant":"yes","explanation":"Copernicus utilizes cloud-based storage with strong encryption (AES-256) and strict access controls."} {"messages":[{"role":"user","content":"Will data transmitted to/from your services be encrypted in transit?"}],"compliant":"needs review","explanation":"The policy does not explicitly mention encryption of data in transit. It focuses on encryption in cloud storage."} {"messages":[{"role":"user","content":"Do you enforce multi-factor authentication (MFA) internally?"}],"compliant":"yes","explanation":"The policy explicitly mentions role-based authentication with multi-factor security."} How much training data is needed?Start small—between several dozen and a few hundred examples—to determine the usefulness of RFT before investing in a large dataset. For product safety reasons, the training set must first pass through an automated screening process. Large datasets take longer to process. This screening process begins when you start a fine-tuning job with a file, not upon initial file upload. Once a file has successfully completed screening, you can use it repeatedly without delay.Dozens of examples can be meaningful as long as they're high quality. After screening, more data is better, as long as it remains high quality. With larger datasets, you can use a higher batch size, which tends to improve training stability.Your training file can contain a maximum of 50,000 examples. Test datasets can contain a maximum of 1,000 examples. Test datasets also go through automated screening. Upload your files The process for uploading RFT training and test data files is the same as supervised fine-tuning. Upload your training data to OpenAI either through the API or using our UI. Files must be uploaded with a purpose of fine-tune in order to be used with fine-tuning. You need file IDs for both your test and training data files to create a fine-tune job. Create a fine-tune job Create a fine-tune job using either the API or fine-tuning dashboard. To do this, you need: File IDs for both your training and test datasets The grader configuration we created earlier The model ID you want to use as a base for fine-tuning (we'll use o4-mini-2025-04-16) If you're fine-tuning a model that will return JSON data as a structured output, you need the JSON schema for the returned object as well (see below) Optionally, any hyperparameters you want to configure for the fine-tune To qualify for data sharing inference pricing, you need to first share evaluation and fine-tuning data with OpenAI before creating the job Structured Outputs JSON schema If you're fine-tuning a model to return Structured Outputs, provide the JSON schema being used to format the output. See a valid JSON schema for our security interview use case: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 { "type": "json_schema", "json_schema": { "name": "security_assistant", "strict": true, "schema": { "type": "object", "properties": { "compliant": { "type": "string" }, "explanation": { "type": "string" } }, "required": [ "compliant", "explanation" ], "additionalProperties": false } } } Generating a JSON schema from a Pydantic modelTo simplify JSON schema generation, start from a Pydantic BaseModel class: Define your class Use to_strict_json_schema from the OpenAI library to generate a valid schema Wrap the schema in a dictionary with type and name keys, and set strict to true Take the resulting object and supply it as the response_format in your RFT job 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 from openai.lib._pydantic import to_strict_json_schema from pydantic import BaseModel class MyCustomClass(BaseModel): name: str age: int # Note: Do not use MyCustomClass.model_json_schema() in place of # to_strict_json_schema as it is not equivalent schema = to_strict_json_schema(MyCustomClass) response_format = dict( type="json_schema", json_schema=dict( name=MyCustomClass.name, strict=True, schema=schema ) ) Create a job with the API Configuring a job with the API has a lot of moving parts, so many users prefer to configure them in the fine-tuning dashboard UI. However, here's a complete API request to kick off a fine-tune job with all the configuration we've set up in this guide so far: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 curl https://api.openai.com/v1/fine_tuning/jobs \ -H "Content-Type: application/json" \ -H "Authorization: Bearer $OPENAI_API_KEY" \ -d '{ "training_file": "file-2STiufDaGXWCnT6XUBUEHW", "validation_file": "file-4TcgH85ej7dFCjZ1kThCYb", "model": "o4-mini-2025-04-16", "method": { "type": "reinforcement", "reinforcement": { "grader": { "type": "multi", "graders": { "explanation": { "name": "Explanation text grader", "type": "score_model", "input": [ { "role": "user", "type": "message", "content": "# Overview\n\nEvaluate the accuracy of the model-generated answer based on the \nCopernicus Product Security Policy and an example answer. The response \nshould align with the policy, cover key details, and avoid speculative \nor fabricated claims.\n\nAlways respond with a single floating point number 0 through 1,\nusing the grading criteria below.\n\n## Grading Criteria:\n- 1.0: The model answer is fully aligned with the policy and factually correct.\n- 0.75: The model answer is mostly correct but has minor omissions or slight rewording that does not change meaning.\n- 0.5: The model answer is partially correct but lacks key details or contains speculative statements.\n- 0.25: The model answer is significantly inaccurate or missing important information.\n- 0.0: The model answer is completely incorrect, hallucinates policy details, or is irrelevant.\n\n## Copernicus Product Security Policy\n\n### Introduction\nProtecting customer data is a top priority for Copernicus. Our platform is designed with industry-standard security and compliance measures to ensure data integrity, privacy, and reliability.\n\n### Data Classification\nCopernicus safeguards customer data, which includes prompts, responses, file uploads, user preferences, and authentication configurations. Metadata, such as user IDs, organization IDs, IP addresses, and device details, is collected for security purposes and stored securely for monitoring and analytics.\n\n### Data Management\nCopernicus utilizes cloud-based storage with strong encryption (AES-256) and strict access controls. Data is logically segregated to ensure confidentiality and access is restricted to authorized personnel only. Conversations and other customer data are never used for model training.\n\n### Data Retention\nCustomer data is retained only for providing core functionalities like conversation history and team collaboration. Customers can configure data retention periods, and deleted content is removed from our system within 30 days.\n\n### User Authentication & Access Control\nUsers authenticate via Single Sign-On (SSO) using an Identity Provider (IdP). Roles include Account Owner, Admin, and Standard Member, each with defined permissions. User provisioning can be automated through SCIM integration.\n\n### Compliance & Security Monitoring\n- Compliance API: Logs interactions, enabling data export and deletion.\n- Audit Logging: Ensures transparency for security audits.\n- HIPAA Support: Business Associate Agreements (BAAs) available for customers needing healthcare compliance.\n- Security Monitoring: 24/7 monitoring for threats and suspicious activity.\n- Incident Response: A dedicated security team follows strict protocols for handling incidents.\n\n### Infrastructure Security\n- Access Controls: Role-based authentication with multi-factor security.\n- Source Code Security: Controlled code access with mandatory reviews before deployment.\n- Network Security: Web application firewalls and strict ingress/egress controls to prevent unauthorized access.\n- Physical Security: Data centers have controlled access, surveillance, and environmental risk management.\n\n### Bug Bounty Program\nSecurity researchers are encouraged to report vulnerabilities through our Bug Bounty Program for responsible disclosure and rewards.\n\n### Compliance & Certifications\nCopernicus maintains compliance with industry standards, including SOC 2 and GDPR. Customers can access security reports and documentation via our Security Portal.\n\n### Conclusion\nCopernicus prioritizes security, privacy, and compliance. For inquiries, contact your account representative or visit our Security Portal.\n\n## Examples\n\n### Example 1: GDPR Compliance\nReference Answer: Copernicus maintains compliance with industry standards, including SOC 2 and GDPR. Customers can access security reports and documentation via our Security Portal.\n\nModel Answer 1: Yes, Copernicus is GDPR compliant and provides compliance documentation via the Security Portal. \nScore: 1.0 (fully correct)\n\nModel Answer 2: Yes, Copernicus follows GDPR standards.\nScore: 0.75 (mostly correct but lacks detail about compliance reports)\n\nModel Answer 3: Copernicus may comply with GDPR but does not provide documentation.\nScore: 0.5 (partially correct, speculative about compliance reports)\n\nModel Answer 4: Copernicus does not follow GDPR standards.\nScore: 0.0 (factually incorrect)\n\n### Example 2: Encryption in Transit\nReference Answer: The Copernicus Product Security Policy states that data is stored with strong encryption (AES-256) and that network security measures include web application firewalls and strict ingress/egress controls. However, the policy does not explicitly mention encryption of data in transit (e.g., TLS encryption). A review is needed to confirm whether data transmission is encrypted.\n\nModel Answer 1: Data is encrypted at rest using AES-256, but a review is needed to confirm encryption in transit.\nScore: 1.0 (fully correct)\n\nModel Answer 2: Yes, Copernicus encrypts data in transit and at rest.\nScore: 0.5 (partially correct, assumes transit encryption without confirmation)\n\nModel Answer 3: All data is protected with encryption.\nScore: 0.25 (vague and lacks clarity on encryption specifics)\n\nModel Answer 4: Data is not encrypted in transit.\nScore: 0.0 (factually incorrect)\n\nReference Answer: {{item.explanation}}\nModel Answer: {{sample.output_json.explanation}}\n" } ], "model": "gpt-4o-2024-08-06" }, "compliant": { "name": "compliant", "type": "string_check", "reference": "{{item.compliant}}", "operation": "eq", "input": "{{sample.output_json.compliant}}" } }, "calculate_output": "0.5 * compliant + 0.5 * explanation" }, "response_format": { "type": "json_schema", "json_schema": { "name": "security_assistant", "strict": true, "schema": { "type": "object", "properties": { "compliant": { "type": "string" }, "explanation": { "type": "string" } }, "required": [ "compliant", "explanation" ], "additionalProperties": false } } }, "hyperparameters": { "reasoning_effort": "medium" } } } }' This request returns a fine-tuning job object, which includes a job id. Use this ID to monitor the progress of your job and retrieve the fine-tuned model when the job is complete. To qualify for data sharing inference pricing, make sure to share evaluation and fine-tuning data with OpenAI before creating the job. You can verify the job was marked as shared by confirming shared_with_openai is set to true. Monitoring your fine-tune job Fine-tuning jobs take some time to complete, and RFT jobs tend to take longer than SFT or DPO jobs. To monitor the progress of your fine-tune job, use the fine-tuning dashboard or the API. Reward metrics For reinforcement fine-tuning jobs, the primary metrics are the per-step reward metrics. These metrics indicate how well your model is performing on the training data. They're calculated by the graders you defined in your job configuration. These are two separate top-level reward metrics: train_reward_mean: The average reward across the samples taken from all datapoints in the current step. Because the specific datapoints in a batch change with each step, train_reward_mean values across different steps are not directly comparable and the specific values can fluctuate drastically from step to step. valid_reward_mean: The average reward across the samples taken from all datapoints in the validation set, which is a more stable metric. Find a full description of all training metrics in the training metrics section. Pausing and resuming jobs To evaluate the current state of the model when your job is only partially finished, pause the job to stop the training process and produce a checkpoint at the current step. You can use this checkpoint to evaluate the model on a held-out test set. If the results look good, resume the job to continue training from that checkpoint. Learn more in pausing and resuming jobs. Evals integration Reinforcement fine-tuning jobs are integrated with our evals product. When you make a reinforcement fine-tuning job, a new eval is automatically created and associated with the job. As validation steps are performed, we combine the input prompts, model samples, and grader outputs to make a new eval run for that step. Learn more about the evals integration in the appendix section below. Evaluate the results By the time your fine-tuning job finishes, you should have a decent idea of how well the model is performing based on the mean reward value on the validation set. However, it's possible that the model has either overfit to the training data or has learned to reward hack your grader, which allows it to produce high scores without actually being correct. Before deploying your model, inspect its behavior on a representative set of prompts to ensure it behaves how you expect. Understanding the model's behavior can be done quickly by inspecting the evals associated with the fine-tuning job. Specifically, pay close attention to the run made for the final training step to see the end model's behavior. You can also use the evals product to compare the final run to earlier runs and see how the model's behavior has changed over the course of training. Try using your fine-tuned model Evaluate your newly optimized model by using it! When the fine-tuned model finishes training, use its ID in either the Responses or Chat Completions API, just as you would an OpenAI base model. Use your model in the PlaygroundUse your model with an API callUse your model in the Playground Navigate to your fine-tuning job in the dashboard. In the right pane, navigate to Output model and copy the model ID. It should start with ft:… Open the Playground. In the Model dropdown menu, paste the model ID. Here, you should also see other fine-tuned models you've created. Run some prompts and see how your fine-tuned performs! Use your model with an API call1 2 3 4 5 6 7 curl https://api.openai.com/v1/responses \ -H "Content-Type: application/json" \ -H "Authorization: Bearer $OPENAI_API_KEY" \ -d '{ "model": "ft:gpt-4.1-nano-2025-04-14:openai::BTz2REMH", "input": "What is 4+4?" }' Use checkpoints if needed Checkpoints are models you can use that are created before the final step of the training process. For RFT, OpenAI creates a full model checkpoint at each validation step and keeps the three with the highest valid_reward_mean scores. Checkpoints are useful for evaluating the model at different points in the training process and comparing performance at different steps. Find checkpoints in the dashboardQuery the API for checkpointsFind checkpoints in the dashboard Navigate to the fine-tuning dashboard. In the left panel, select the job you want to investigate. Wait until it succeeds. In the right panel, scroll to the list of checkpoints. Hover over any checkpoint to see a link to launch in the Playground. Test the checkpoint model's behavior by prompting it in the Playground. Query the API for checkpoints Wait until a job succeeds, which you can verify by querying the status of a job. Query the checkpoints endpoint with your fine-tuning job ID to access a list of model checkpoints for the fine-tuning job. Find the fine_tuned_model_checkpoint field for the name of the model checkpoint. Use this model just like you would the final fine-tuned model. The checkpoint object contains metrics data to help you determine the usefulness of this model. As an example, the response looks like this: 1 2 3 4 5 6 7 8 9 10 11 12 { "object": "fine_tuning.job.checkpoint", "id": "ftckpt_zc4Q7MP6XxulcVzj4MZdwsAB", "created_at": 1519129973, "fine_tuned_model_checkpoint": "ft:gpt-3.5-turbo-0125:my-org:custom-suffix:96olL566:ckpt-step-2000", "metrics": { "full_valid_loss": 0.134, "full_valid_mean_token_accuracy": 0.874 }, "fine_tuning_job_id": "ftjob-abc123", "step_number": 2000 } Each checkpoint specifies: step_number: The step at which the checkpoint was created (where each epoch is number of steps in the training set divided by the batch size) metrics: An object containing the metrics for your fine-tuning job at the step when the checkpoint was created Safety checks Before launching in production, review and follow the following safety information. How we assess for safetyOnce a fine-tuning job is completed, we assess the resulting model’s behavior across 13 distinct safety categories. Each category represents a critical area where AI outputs could potentially cause harm if not properly controlled.NameDescriptionadviceAdvice or guidance that violates our policies.harassment/threateningHarassment content that also includes violence or serious harm towards any target.hateContent that expresses, incites, or promotes hate based on race, gender, ethnicity, religion, nationality, sexual orientation, disability status, or caste. Hateful content aimed at non-protected groups (e.g., chess players) is harassment.hate/threateningHateful content that also includes violence or serious harm towards the targeted group based on race, gender, ethnicity, religion, nationality, sexual orientation, disability status, or caste.highly-sensitiveHighly sensitive data that violates our policies.illicitContent that gives advice or instruction on how to commit illicit acts. A phrase like "how to shoplift" would fit this category.propagandaPraise or assistance for ideology that violates our policies.self-harm/instructionsContent that encourages performing acts of self-harm, such as suicide, cutting, and eating disorders, or that gives instructions or advice on how to commit such acts.self-harm/intentContent where the speaker expresses that they are engaging or intend to engage in acts of self-harm, such as suicide, cutting, and eating disorders.sensitiveSensitive data that violates our policies.sexual/minorsSexual content that includes an individual who is under 18 years old.sexualContent meant to arouse sexual excitement, such as the description of sexual activity, or that promotes sexual services (excluding sex education and wellness).violenceContent that depicts death, violence, or physical injury.Each category has a predefined pass threshold; if too many evaluated examples in a given category fail, OpenAI blocks the fine-tuned model from deployment. If your fine-tuned model does not pass the safety checks, OpenAI sends a message in the fine-tuning job explaining which categories don't meet the required thresholds. You can view the results in the moderation checks section of the fine-tuning job. How to pass safety checksIn addition to reviewing any failed safety checks in the fine-tuning job object, you can retrieve details about which categories failed by querying the fine-tuning API events endpoint. Look for events of type moderation_checks for details about category results and enforcement. This information can help you narrow down which categories to target for retraining and improvement. The model spec has rules and examples that can help identify areas for additional training data.While these evaluations cover a broad range of safety categories, conduct your own evaluations of the fine-tuned model to ensure it's appropriate for your use case. Next steps Now that you know the basics of reinforcement fine-tuning, explore other fine-tuning methods. Supervised fine-tuningFine-tune a model by providing correct outputs for sample inputs. Vision fine-tuningLearn to fine-tune for computer vision with image inputs. Direct preference optimizationFine-tune a model using direct preference optimization (DPO). Appendix Training metrics Reinforcement fine-tuning jobs publish per-step training metrics as fine-tuning events. Pull these metrics through the API or view them as graphs and charts in the fine-tuning dashboard. Learn more about training metrics below. Full example training metricsBelow is an example metric event from a real reinforcement fine-tuning job. The various fields in this payload will be discussed in the following sections.1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 { "object": "fine_tuning.job.event", "id": "ftevent-Iq5LuNLDsac1C3vzshRBuBIy", "created_at": 1746679539, "level": "info", "message": "Step 10/20 , train mean reward=0.42, full validation mean reward=0.68, full validation mean parse error=0.00", "data": { "step": 10, "usage": { "graders": [ { "name": "basic_model_grader", "type": "score_model", "model": "gpt-4o-2024-08-06", "train_prompt_tokens_mean": 241.0, "valid_prompt_tokens_mean": 241.0, "train_prompt_tokens_count": 120741.0, "valid_prompt_tokens_count": 4820.0, "train_completion_tokens_mean": 138.52694610778443, "valid_completion_tokens_mean": 140.5, "train_completion_tokens_count": 69402.0, "valid_completion_tokens_count": 2810.0 } ], "samples": { "train_reasoning_tokens_mean": 3330.017964071856, "valid_reasoning_tokens_mean": 1948.9, "train_reasoning_tokens_count": 1668339.0, "valid_reasoning_tokens_count": 38978.0 } }, "errors": { "graders": [ { "name": "basic_model_grader", "type": "score_model", "train_other_error_mean": 0.0, "valid_other_error_mean": 0.0, "train_other_error_count": 0.0, "valid_other_error_count": 0.0, "train_sample_parse_error_mean": 0.0, "valid_sample_parse_error_mean": 0.0, "train_sample_parse_error_count": 0.0, "valid_sample_parse_error_count": 0.0, "train_invalid_variable_error_mean": 0.0, "valid_invalid_variable_error_mean": 0.0, "train_invalid_variable_error_count": 0.0, "valid_invalid_variable_error_count": 0.0 } ] }, "scores": { "graders": [ { "name": "basic_model_grader", "type": "score_model", "train_reward_mean": 0.4471057884231537, "valid_reward_mean": 0.675 } ], "train_reward_mean": 0.4215686274509804, "valid_reward_mean": 0.675 }, "timing": { "step": { "eval": 101.69386267662048, "sampling": 226.82190561294556, "training": 402.43121099472046, "full_iteration": 731.5038568973541 }, "graders": [ { "name": "basic_model_grader", "type": "score_model", "train_execution_latency_mean": 2.6894934929297594, "valid_execution_latency_mean": 4.141402995586395 } ] }, "total_steps": 20, "train_mean_reward": 0.4215686274509804, "reasoning_tokens_mean": 3330.017964071856, "completion_tokens_mean": 3376.0019607843137, "full_valid_mean_reward": 0.675, "mean_unresponsive_rewards": 0.0, "model_graders_token_usage": { "gpt-4o-2024-08-06": { "eval_cached_tokens": 0, "eval_prompt_tokens": 4820, "train_cached_tokens": 0, "train_prompt_tokens": 120741, "eval_completion_tokens": 2810, "train_completion_tokens": 69402 } }, "full_valid_mean_parse_error": 0.0, "valid_reasoning_tokens_mean": 1948.9 }, "type": "metrics" }, Score metricsThe top-level metrics to watch are train_reward_mean and valid_reward_mean, which indicate the average reward assigned by your graders across all samples in the training and validation datasets, respectively.Additionally, if you use a multi-grader configuration, per-grader train and validation reward metrics will be published as well. These metrics are included under the event.data.scores object in the fine-tuning events object, with one entry per grader. The per-grader metrics are useful for understanding how the model is performing on each individual grader, and can help you identify if the model is overfitting to one grader or another.From the fine-tuning dashboard, the individual grader metrics will be displayed in their own graph below the overall train_reward_mean and valid_reward_mean metrics. Usage metricsAn important characteristic of a reasoning model is the number of reasoning tokens it uses before responding to a prompt. Often, during training, the model will drastically change the average number of reasoning tokens it uses to respond to a prompt. This is a sign that the model is changing its behavior in response to the reward signal. The model may learn to use fewer reasoning tokens to achieve the same reward, or it may learn to use more reasoning tokens to achieve a higher reward.You can monitor the train_reasoning_tokens_mean and valid_reasoning_tokens_mean metrics to see how the model is changing its behavior over time. These metrics are the average number of reasoning tokens used by the model to respond to a prompt in the training and validation datasets, respectively. You can also view the mean reasoning token count in the fine-tuning dashboard under the "Reasoning Tokens" chart.If you are using model graders, you will likely want to monitor the token usage of these graders. Per-grader token usage statistics are available under the event.data.usage.graders object, and are broken down into: train_prompt_tokens_mean train_prompt_tokens_count train_completion_tokens_mean train_completion_tokens_count. The mean metrics represent the average number of tokens used by the grader to process all prompts in the current step, while the count metrics represent the total number of tokens used by the grader across all samples in the current step. The per-step token usage is also displayed on the fine-tuning dashboard under the "Grading Token Usage" chart. Timing metricsWe include various metrics that help you understand how long each step of the training process is taking and how different parts of the training process are contributing to the per-step timing.These metrics are available under the event.data.timing object, and are broken down into step and graders fields.The step field contains the following metrics: sampling: The time taken to sample the model outputs (rollouts) for the current step. training: The time taken to train the model (backpropagation) for the current step. eval: The time taken to evaluate the model on the full validation set. full_iteration: The total time taken for the current step, including the above 3 metrics plus any additional overhead. The step timing metrics are also displayed on the fine-tuning dashboard under the "Per Step Duration" chart.The graders field contains timing information that details the time taken to execute each grader for the current step. Each grader will have its own timing under the train_execution_latency_mean and valid_execution_latency_mean metrics, which represent the average time taken to execute the grader on the training and validation datasets, respectively.Graders are executed in parallel with a concurrency limit, so it is not always clear how individual grader latency adds up to the total time taken for grading. However, it is generally true that graders which take longer to execute individually will cause a job to execute more slowly. This means that slower model graders will cause the job to take longer to complete, and more expensive python code will do the same. The fastest graders generally are string_check and text_similarity as those are executed local to the training loop. Evals integration details Reinforcement fine-tuning jobs are directly integrated with our evals product. When you make a reinforcement fine-tuning job, a new eval is automatically created and associated with the job. As validation steps are performed, the input prompts, model samples, grader outputs, and more metadata will be combined to make a new eval run for that step. At the end of the job, you will have one run for each validation step. This allows you to compare the performance of the model at different steps, and to see how the model's behavior has changed over the course of training. You can find the eval associated with your fine-tuning job by viewing your job on the fine-tuning dashboard, or by finding the eval_id field on the fine-tuning job object. The evals product is useful for inspecting the outputs of the model on specific datapoints, to get an understanding for how the model is behaving in different scenarios. It can help you figure out which slice of your dataset the model is performing poorly on which can help you identify areas for improvement in your training data. The evals product can also help you find areas of improvement for your graders by finding areas where the grader is either overly lenient or overly harsh on the model outputs. Pausing and resuming jobs You can pause a fine-tuning job at any time by using the fine-tuning jobs API. Calling the pause API will tell the training process to create a new model snapshot, stop training, and put the job into a "Paused" state. The model snapshot will go through a normal safety screening process after which it will be available for you to use throughout the OpenAI platform as a normal fine-tuned model. If you wish to continue the training process for a paused job, you can do so by using the fine-tuning jobs API. This will resume the training process from the last checkpoint created when the job was paused and will continue training until the job is either completed or paused again. Grading with Tools If you are training your model to perform tool calls, you will need to: Provide the set of tools available for your model to call on each datapoint in the RFT training dataset. More info here in the dataset API reference. Configure your grader to assign rewards based on the contents of the tool calls made by the model. Information on grading tools calls can be found here in the grading docs Billing details Reinforcement fine-tuning jobs are billed based on the amount of time spent training, as well as the number of tokens used by the model during training. We only bill for time spent in the core training loop, not for time spent preparing the training data, validating datasets, waiting in queues, running safety evals, or other overhead. Details on exactly how we bill for reinforcement fine-tuning jobs can be found in this help center article. Training errors Reinforcement fine-tuning is a complex process with many moving parts, and there are many places where things can go wrong. We publish various error metrics to help you understand what is going wrong in your job, and how to fix it. In general, we try to avoid failing a job entirely unless a very serious error occurs. When errors do occur, they often happen during the grading step. Errors during grading often happen either to the model outputting a sample that the grader doesn't know how to handle, the grader failing to execute properly due to some sort of system error, or due to a bug in the grading logic itself. The error metrics are available under the event.data.errors object, and are aggregated into counts and rates rolled up per-grader. We also display rates and counts of errors on the fine-tuning dashboard. Grader errorsGeneric grading errorsThe grader errors are broken down into the following categories, and they exist in both train (for training data) and valid (for validation data) versions: sample_parse_error_mean: The average number of samples that failed to parse correctly. This often happens when the model fails to output valid JSON or adhere to a provided response format correctly. A small percentage of these errors, especially early in the training process, is normal. If you see a large number of these errors, it is likely that the response format of the model is not configured correctly or that your graders are misconfigured and looking for incorrect fields. invalid_variable_error_mean: These errors occur when you attempt to reference a variable via a template that cannot be found either in the current datapoint or in the current model sample. This can happen if the model fails to provide output in the correct response format, or if your grader is misconfigured. other_error_mean: This is a catch-all for any other errors that occur during grading. These errors are often caused by bugs in the grading logic itself, or by system errors that occur during grading. Python grading errors python_grader_server_error_mean: These errors occur when our system for executing python graders in a remote sandbox experiences system errors. This normally happens due to reasons outside of your control, like networking failures or system outages. If you see a large number of these errors, it is likely that there is a system issue that is causing the errors. You can check the OpenAI status page for more information on any ongoing issues. python_grader_runtime_error_mean: These errors occur when the python grader itself fails to execute properly. This can happen for a variety of reasons, including bugs in the grading logic, or if the grader is trying to access a variable that doesn't exist in the current context. If you see a large number of these errors, it is likely that there is a bug in your grading logic that needs to be fixed. If a large enough number of these errors occur, the job will fail and we will show you a sampling of tracebacks from the failed graders. Model grading errors model_grader_server_error_mean: These errors occur when we fail to sample from a model grader. This can happen for a variety of reasons, but generally means that either the model grader was misconfigured, that you are attempting to use a model that is not available to your organization, or that there is a system issue that is happening at OpenAI. OverviewOverviewRFT exampleRFT exampleDefine a graderDefine a graderPrepare your datasetPrepare your datasetCreate a fine-tune jobCreate a fine-tune jobEvaluate the resultsEvaluate the resultsSafety checksSafety checksNext stepsNext steps

o4-mini-2025-04-16

Pattern 5: Video generation with SoraCreate, iterate on, and manage videos with the Sora API.Copy pageExplore Overview Sora is OpenAI’s newest frontier in generative media – a state-of-the-art video model capable of creating richly detailed, dynamic clips with audio from natural language or images. Built on years of research into multimodal diffusion and trained on diverse visual data, Sora brings a deep understanding of 3D space, motion, and scene continuity to text-to-video generation. The Video API (in preview) exposes these capabilities to developers for the first time, enabling programmatic creation, extension, and remixing of videos. It provides five endpoints, each with distinct capabilities: Create video: Start a new render job from a prompt, with optional reference inputs or a remix ID. Get video status: Retrieve the current state of a render job and monitor its progress. Download video: Fetch the finished MP4 once the job is completed. List videos: Enumerate your videos with pagination for history, dashboards, or housekeeping. Delete videos: Remove an individual video ID from OpenAI’s storage. Models The second generation Sora model comes in two variants, each tailored for different use cases. Sora 2 sora-2 is designed for speed and flexibility. It’s ideal for the exploration phase, when you’re experimenting with tone, structure, or visual style and need quick feedback rather than perfect fidelity. It generates good quality results quickly, making it well suited for rapid iteration, concepting, and rough cuts. sora-2 is often more than sufficient for social media content, prototypes, and scenarios where turnaround time matters more than ultra-high fidelity. Sora 2 Pro sora-2-pro produces higher quality results. It’s the better choice when you need production-quality output. sora-2-pro takes longer to render and is more expensive to run, but it produces more polished, stable results. It’s best for high-resolution cinematic footage, marketing assets, and any situation where visual precision is critical. Generate a video Generating a video is an asynchronous process: When you call the POST /videos endpoint, the API returns a job object with a job id and an initial status. You can either poll the GET /videos/{video_id} endpoint until the status transitions to completed, or – for a more efficient approach – use webhooks (see the webhooks section below) to be notified automatically when the job finishes. Once the job has reached the completed state you can fetch the final MP4 file with GET /videos/{video_id}/content. Start a render job Start by calling POST /videos with a text prompt and the required parameters. The prompt defines the creative look and feel – subjects, camera, lighting, and motion – while parameters like size and seconds control the video's resolution and length. Create a videojavascript1 2 3 4 5 6 7 8 9 10 import OpenAI from 'openai'; const openai = new OpenAI(); let video = await openai.videos.create({ model: 'sora-2', prompt: "A video of the words 'Thank you' in sparkling letters", }); console.log('Video generation started: ', video);1 2 3 4 5 6 7 8 9 10 from openai import OpenAI openai = OpenAI() video = openai.videos.create( model="sora-2", prompt="A video of a cool cat on a motorcycle in the night", ) print("Video generation started:", video)1 2 3 4 5 6 7 curl -X POST "https://api.openai.com/v1/videos" \ -H "Authorization: Bearer $OPENAI_API_KEY" \ -H "Content-Type: multipart/form-data" \ -F prompt="Wide tracking shot of a teal coupe driving through a desert highway, heat ripples visible, hard sun overhead." \ -F model="sora-2-pro" \ -F size="1280x720" \ -F seconds="8" \ The response is a JSON object with a unique id and an initial status such as queued or in_progress. This means the render job has started. 1 2 3 4 5 6 7 8 9 10 { "id": "video_68d7512d07848190b3e45da0ecbebcde004da08e1e0678d5", "object": "video", "created_at": 1758941485, "status": "queued", "model": "sora-2-pro", "progress": 0, "seconds": "8", "size": "1280x720" } Guardrails and restrictions The API enforces several content restrictions: Only content suitable for audiences under 18 (a setting to bypass this restriction will be available in the future). Copyrighted characters and copyrighted music will be rejected. Real people—including public figures—cannot be generated. Input images with faces of humans are currently rejected. Make sure prompts, reference images, and transcripts respect these rules to avoid failed generations. Effective prompting For best results, describe shot type, subject, action, setting, and lighting. For example: “Wide shot of a child flying a red kite in a grassy park, golden hour sunlight, camera slowly pans upward.” “Close-up of a steaming coffee cup on a wooden table, morning light through blinds, soft depth of field.” This level of specificity helps the model produce consistent results without inventing unwanted details. For more advanced prompting techniques, please refer to our dedicated Sora 2 prompting guide. Monitor progress Video generation takes time. Depending on model, API load and resolution, a single render may take several minutes. To manage this efficiently, you can poll the API to request status updates or you can get notified via a webhook. Poll the status endpoint Call GET /videos/{video_id} with the id returned from the create call. The response shows the job’s current status, progress percentage (if available), and any errors. Typical states are queued, in_progress, completed, and failed. Poll at a reasonable interval (for example, every 10–20 seconds), use exponential backoff if necessary, and provide feedback to users that the job is still in progress. Poll the status endpointjavascript1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 import OpenAI from 'openai'; const openai = new OpenAI(); async function main() { const video = await openai.videos.createAndPoll({ model: 'sora-2', prompt: "A video of the words 'Thank you' in sparkling letters", }); if (video.status === 'completed') { console.log('Video successfully completed: ', video); } else { console.log('Video creation failed. Status: ', video.status); } } main();1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 import asyncio from openai import AsyncOpenAI client = AsyncOpenAI() async def main() -> None: video = await client.videos.create_and_poll( model="sora-2", prompt="A video of a cat on a motorcycle", ) if video.status == "completed": print("Video successfully completed: ", video) else: print("Video creation failed. Status: ", video.status) asyncio.run(main()) Response example: 1 2 3 4 5 6 7 8 9 10 { "id": "video_68d7512d07848190b3e45da0ecbebcde004da08e1e0678d5", "object": "video", "created_at": 1758941485, "status": "in_progress", "model": "sora-2-pro", "progress": 33, "seconds": "8", "size": "1280x720" } Use webhooks for notifications Instead of polling job status repeatedly with GET, register a webhook to be notified automatically when a video generation completes or fails. Webhooks can be configured in your webhook settings page. When a job finishes, the API emits one of two event types: video.completed and video.failed. Each event includes the ID of the job that triggered it. Example webhook payload: 1 2 3 4 5 6 7 8 9 { "id": "evt_abc123", "object": "event", "created_at": 1758941485, "type": "video.completed", // or "video.failed" "data": { "id": "video_abc123" } } Retrieve results Download the MP4 Once the job reaches status completed, fetch the MP4 with GET /videos/{video_id}/content. This endpoint streams the binary video data and returns standard content headers, so you can either save the file directly to disk or pipe it to cloud storage. Download the MP4javascript1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 import OpenAI from 'openai'; const openai = new OpenAI(); let video = await openai.videos.create({ model: 'sora-2', prompt: "A video of the words 'Thank you' in sparkling letters", }); console.log('Video generation started: ', video); let progress = video.progress ?? 0; while (video.status === 'in_progress' || video.status === 'queued') { video = await openai.videos.retrieve(video.id); progress = video.progress ?? 0; // Display progress bar const barLength = 30; const filledLength = Math.floor((progress / 100) * barLength); // Simple ASCII progress visualization for terminal output const bar = '='.repeat(filledLength) + '-'.repeat(barLength - filledLength); const statusText = video.status === 'queued' ? 'Queued' : 'Processing'; process.stdout.write(${statusText}: [${bar}] ${progress.toFixed(1)}%); await new Promise((resolve) => setTimeout(resolve, 2000)); } // Clear the progress line and show completion process.stdout.write('\n'); if (video.status === 'failed') { console.error('Video generation failed'); return; } console.log('Video generation completed: ', video); console.log('Downloading video content...'); const content = await openai.videos.downloadContent(video.id); const body = content.arrayBuffer(); const buffer = Buffer.from(await body); require('fs').writeFileSync('video.mp4', buffer); console.log('Wrote video.mp4');1 2 3 curl -L "https://api.openai.com/v1/videos/video_abc123/content" \ -H "Authorization: Bearer $OPENAI_API_KEY" \ --output video.mp41 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 from openai import OpenAI import sys import time openai = OpenAI() video = openai.videos.create( model="sora-2", prompt="A video of a cool cat on a motorcycle in the night", ) print("Video generation started:", video) progress = getattr(video, "progress", 0) bar_length = 30 while video.status in ("in_progress", "queued"): # Refresh status video = openai.videos.retrieve(video.id) progress = getattr(video, "progress", 0) filled_length = int((progress / 100) * bar_length) bar = "=" * filled_length + "-" * (bar_length - filled_length) status_text = "Queued" if video.status == "queued" else "Processing" sys.stdout.write(f" {status_text}: [{bar}] {progress:.1f}%") sys.stdout.flush() time.sleep(2) # Move to next line after progress loop sys.stdout.write(" ") if video.status == "failed": message = getattr( getattr(video, "error", None), "message", "Video generation failed" ) print(message) return print("Video generation completed:", video) print("Downloading video content...") content = openai.videos.download_content(video.id, variant="video") content.write_to_file("video.mp4") print("Wrote video.mp4") You now have the final video file ready for playback, editing, or distribution. Download URLs are valid for a maximum of 1 hour after generation. If you need long-term storage, copy the file to your own storage system promptly. Download supporting assets For each completed video, you can also download a thumbnail and a spritesheet. These are lightweight assets useful for previews, scrubbers, or catalog displays. Use the variant query parameter to specify what you want to download. The default is variant=video for the MP4. 1 2 3 4 5 6 7 8 9 # Download a thumbnail curl -L "https://api.openai.com/v1/videos/video_abc123/content?variant=thumbnail" \ -H "Authorization: Bearer $OPENAI_API_KEY" \ --output thumbnail.webp # Download a spritesheet curl -L "https://api.openai.com/v1/videos/video_abc123/content?variant=spritesheet" \ -H "Authorization: Bearer $OPENAI_API_KEY" \ --output spritesheet.jpg Use image references You can guide a generation with an input image, which acts as the first frame of your video. This is useful if you need the output video to preserve the look of a brand asset, a character, or a specific environment. Include an image file as the input_reference parameter in your POST /videos request. The image must match the target video’s resolution (size). Supported file formats are image/jpeg, image/png, and image/webp. 1 2 3 4 5 6 7 8 curl -X POST "https://api.openai.com/v1/videos" \ -H "Authorization: Bearer $OPENAI_API_KEY" \ -H "Content-Type: multipart/form-data" \ -F prompt="She turns around and smiles, then slowly walks out of the frame." \ -F model="sora-2-pro" \ -F size="1280x720" \ -F seconds="8" \ -F input_reference="@sample_720p.jpeg;type=image/jpeg" Input image generated with OpenAI GPT ImageGenerated video using Sora 2 (converted to GIF)Download this image Prompt: “She turns around and smiles, then slowly walks out of the frame.”Download this image Prompt: “The fridge door opens. A cute, chubby purple monster comes out of it.” Remix completed videos Remix lets you take an existing video and make targeted adjustments without regenerating everything from scratch. Provide the remix_video_id of a completed job along with a new prompt that describes the change, and the system reuses the original’s structure, continuity, and composition while applying the modification. This works best when you make a single, well-defined change because smaller, focused edits preserve more of the original fidelity and reduce the risk of introducing artifacts. 1 2 3 4 5 6 curl -X POST "https://api.openai.com/v1/videos/<previous_video_id>/remix" \ -H "Authorization: Bearer $OPENAI_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "prompt": "Shift the color palette to teal, sand, and rust, with a warm backlight." }' Remix is especially valuable for iteration because it lets you refine without discarding what already works. By constraining each remix to one clear adjustment, you keep the visual style, subject consistency, and camera framing stable, while still exploring variations in mood, palette, or staging. This makes it far easier to build polished sequences through small, reliable steps. Original videoRemix generated video Prompt: “Change the color of the monster to orange.” Prompt: “A second monster comes out right after.” Maintain your library Use GET /videos to enumerate your videos. The endpoint supports optional query parameters for pagination and sorting. 1 2 3 # default curl "https://api.openai.com/v1/videos" \ -H "Authorization: Bearer $OPENAI_API_KEY" | jq . 1 2 3 # with params curl "https://api.openai.com/v1/videos?limit=20&after=video_123&order=asc" \ -H "Authorization: Bearer $OPENAI_API_KEY" | jq . Use DELETE /videos/{video_id} to remove videos you no longer need from OpenAI’s storage. curl -X DELETE "https://api.openai.com/v1/videos/[REPLACE_WITH_YOUR_VIDEO_ID]" \ -H "Authorization: Bearer $OPENAI_API_KEY" | jq .OverviewOverviewModelsModelsGenerate a videoGenerate a videoUse image referencesUse image referencesRemix completed videosRemix completed videosMaintain your libraryMaintain your library

sora-2

Pattern 6: Chat CompletionsThe Chat Completions API endpoint will generate a model response from a list of messages comprising a conversation. Related guides: Quickstart Text inputs and outputs Image inputs Audio inputs and outputs Structured Outputs Function calling Conversation state Starting a new project? We recommend trying Responses to take advantage of the latest OpenAI platform features. Compare Chat Completions with Responses.Create chat completionpost https://api.openai.com/v1/chat/completionsStarting a new project? We recommend trying Responses to take advantage of the latest OpenAI platform features. Compare Chat Completions with Responses. Creates a model response for the given chat conversation. Learn more in the text generation, vision, and audio guides. Parameter support can differ depending on the model used to generate the response, particularly for newer reasoning models. Parameters that are only supported for reasoning models are noted below. For the current state of unsupported parameters in reasoning models, refer to the reasoning guide.Request bodymessagesarrayRequiredA list of messages comprising the conversation so far. Depending on the model you use, different message types (modalities) are supported, like text, images, and audio.Show possible typesmodelstringRequiredModel ID used to generate the response, like gpt-4o or o3. OpenAI offers a wide range of models with different capabilities, performance characteristics, and price points. Refer to the model guide to browse and compare available models.audioobject or nullOptionalParameters for audio output. Required when audio output is requested with modalities: ["audio"]. Learn more.Show propertiesfrequency_penaltynumber or nullOptionalDefaults to 0Number between -2.0 and 2.0. Positive values penalize new tokens based on their existing frequency in the text so far, decreasing the model's likelihood to repeat the same line verbatim.function_callDeprecatedstring or objectOptionalDeprecated in favor of tool_choice. Controls which (if any) function is called by the model. none means the model will not call a function and instead generates a message. auto means the model can pick between generating a message or calling a function. Specifying a particular function via {"name": "my_function"} forces the model to call that function. none is the default when no functions are present. auto is the default if functions are present.Show possible typesfunctionsDeprecatedarrayOptionalDeprecated in favor of tools. A list of functions the model may generate JSON inputs for.Show propertieslogit_biasmapOptionalDefaults to nullModify the likelihood of specified tokens appearing in the completion. Accepts a JSON object that maps tokens (specified by their token ID in the tokenizer) to an associated bias value from -100 to 100. Mathematically, the bias is added to the logits generated by the model prior to sampling. The exact effect will vary per model, but values between -1 and 1 should decrease or increase likelihood of selection; values like -100 or 100 should result in a ban or exclusive selection of the relevant token.logprobsboolean or nullOptionalDefaults to falseWhether to return log probabilities of the output tokens or not. If true, returns the log probabilities of each output token returned in the content of message.max_completion_tokensinteger or nullOptionalAn upper bound for the number of tokens that can be generated for a completion, including visible output tokens and reasoning tokens.max_tokensDeprecatedinteger or nullOptionalThe maximum number of tokens that can be generated in the chat completion. This value can be used to control costs for text generated via API. This value is now deprecated in favor of max_completion_tokens, and is not compatible with o-series models.metadatamapOptionalSet of 16 key-value pairs that can be attached to an object. This can be useful for storing additional information about the object in a structured format, and querying for objects via API or the dashboard. Keys are strings with a maximum length of 64 characters. Values are strings with a maximum length of 512 characters.modalitiesarrayOptionalOutput types that you would like the model to generate. Most models are capable of generating text, which is the default: ["text"] The gpt-4o-audio-preview model can also be used to generate audio. To request that this model generate both text and audio responses, you can use: ["text", "audio"]ninteger or nullOptionalDefaults to 1How many chat completion choices to generate for each input message. Note that you will be charged based on the number of generated tokens across all of the choices. Keep n as 1 to minimize costs.parallel_tool_callsbooleanOptionalDefaults to trueWhether to enable parallel function calling during tool use.predictionobjectOptionalConfiguration for a Predicted Output, which can greatly improve response times when large parts of the model response are known ahead of time. This is most common when you are regenerating a file with only minor changes to most of the content.Show possible typespresence_penaltynumber or nullOptionalDefaults to 0Number between -2.0 and 2.0. Positive values penalize new tokens based on whether they appear in the text so far, increasing the model's likelihood to talk about new topics.prompt_cache_keystringOptionalUsed by OpenAI to cache responses for similar requests to optimize your cache hit rates. Replaces the user field. Learn more.prompt_cache_retentionstringOptionalThe retention policy for the prompt cache. Set to 24h to enable extended prompt caching, which keeps cached prefixes active for longer, up to a maximum of 24 hours. Learn more.reasoning_effortstringOptionalDefaults to mediumConstrains effort on reasoning for reasoning models. Currently supported values are none, minimal, low, medium, high, and xhigh. Reducing reasoning effort can result in faster responses and fewer tokens used on reasoning in a response. gpt-5.1 defaults to none, which does not perform reasoning. The supported reasoning values for gpt-5.1 are none, low, medium, and high. Tool calls are supported for all reasoning values in gpt-5.1. All models before gpt-5.1 default to medium reasoning effort, and do not support none. The gpt-5-pro model defaults to (and only supports) high reasoning effort. xhigh is supported for all models after gpt-5.1-codex-max. response_formatobjectOptionalAn object specifying the format that the model must output. Setting to { "type": "json_schema", "json_schema": {...} } enables Structured Outputs which ensures the model will match your supplied JSON schema. Learn more in the Structured Outputs guide. Setting to { "type": "json_object" } enables the older JSON mode, which ensures the message the model generates is valid JSON. Using json_schema is preferred for models that support it.Show possible typessafety_identifierstringOptionalA stable identifier used to help detect users of your application that may be violating OpenAI's usage policies. The IDs should be a string that uniquely identifies each user. We recommend hashing their username or email address, in order to avoid sending us any identifying information. Learn more.seedDeprecatedinteger or nullOptionalThis feature is in Beta. If specified, our system will make a best effort to sample deterministically, such that repeated requests with the same seed and parameters should return the same result. Determinism is not guaranteed, and you should refer to the system_fingerprint response parameter to monitor changes in the backend.service_tierstringOptionalDefaults to autoSpecifies the processing type used for serving the request. If set to 'auto', then the request will be processed with the service tier configured in the Project settings. Unless otherwise configured, the Project will use 'default'. If set to 'default', then the request will be processed with the standard pricing and performance for the selected model. If set to 'flex' or 'priority', then the request will be processed with the corresponding service tier. When not set, the default behavior is 'auto'. When the service_tier parameter is set, the response body will include the service_tier value based on the processing mode actually used to serve the request. This response value may be different from the value set in the parameter.stopstring / array / nullOptionalDefaults to nullNot supported with latest reasoning models o3 and o4-mini. Up to 4 sequences where the API will stop generating further tokens. The returned text will not contain the stop sequence.storeboolean or nullOptionalDefaults to falseWhether or not to store the output of this chat completion request for use in our model distillation or evals products. Supports text and image inputs. Note: image inputs over 8MB will be dropped.streamboolean or nullOptionalDefaults to falseIf set to true, the model response data will be streamed to the client as it is generated using server-sent events. See the Streaming section below for more information, along with the streaming responses guide for more information on how to handle the streaming events.stream_optionsobjectOptionalDefaults to nullOptions for streaming response. Only set this when you set stream: true.Show propertiestemperaturenumberOptionalDefaults to 1What sampling temperature to use, between 0 and 2. Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic. We generally recommend altering this or top_p but not both.tool_choicestring or objectOptionalControls which (if any) tool is called by the model. none means the model will not call any tool and instead generates a message. auto means the model can pick between generating a message or calling one or more tools. required means the model must call one or more tools. Specifying a particular tool via {"type": "function", "function": {"name": "my_function"}} forces the model to call that tool. none is the default when no tools are present. auto is the default if tools are present.Show possible typestoolsarrayOptionalA list of tools the model may call. You can provide either custom tools or function tools.Show possible typestop_logprobsintegerOptionalAn integer between 0 and 20 specifying the number of most likely tokens to return at each token position, each with an associated log probability.top_pnumberOptionalDefaults to 1An alternative to sampling with temperature, called nucleus sampling, where the model considers the results of the tokens with top_p probability mass. So 0.1 means only the tokens comprising the top 10% probability mass are considered. We generally recommend altering this or temperature but not both.userDeprecatedstringOptionalThis field is being replaced by safety_identifier and prompt_cache_key. Use prompt_cache_key instead to maintain caching optimizations. A stable identifier for your end-users. Used to boost cache hit rates by better bucketing similar requests and to help OpenAI detect and prevent abuse. Learn more.verbositystringOptionalDefaults to mediumConstrains the verbosity of the model's response. Lower values will result in more concise responses, while higher values will result in more verbose responses. Currently supported values are low, medium, and high.web_search_optionsobjectOptionalThis tool searches the web for relevant results to use in a response. Learn more about the web search tool.Show propertiesReturnsReturns a chat completion object, or a streamed sequence of chat completion chunk objects if the request is streamed.DefaultImage inputStreamingFunctionsLogprobsExample requestcurl1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 curl https://api.openai.com/v1/chat/completions \ -H "Content-Type: application/json" \ -H "Authorization: Bearer $OPENAI_API_KEY" \ -d '{ "model": "gpt-5.2", "messages": [ { "role": "developer", "content": "You are a helpful assistant." }, { "role": "user", "content": "Hello!" } ] }'1 2 3 4 5 6 7 8 9 10 11 12 from openai import OpenAI client = OpenAI() completion = client.chat.completions.create( model="gpt-5.2", messages=[ {"role": "developer", "content": "You are a helpful assistant."}, {"role": "user", "content": "Hello!"} ] ) print(completion.choices[0].message)1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 import OpenAI from "openai"; const openai = new OpenAI(); async function main() { const completion = await openai.chat.completions.create({ messages: [{ role: "developer", content: "You are a helpful assistant." }], model: "gpt-5.2", store: true, }); console.log(completion.choices[0]); } main();1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 using System; using System.Collections.Generic; using OpenAI.Chat; ChatClient client = new( model: "gpt-4.1", apiKey: Environment.GetEnvironmentVariable("OPENAI_API_KEY") ); List messages = [ new SystemChatMessage("You are a helpful assistant."), new UserChatMessage("Hello!") ]; ChatCompletion completion = client.CompleteChat(messages); Console.WriteLine(completion.Content[0].Text);Response1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 { "id": "chatcmpl-B9MBs8CjcvOU2jLn4n570S5qMJKcT", "object": "chat.completion", "created": 1741569952, "model": "gpt-4.1-2025-04-14", "choices": [ { "index": 0, "message": { "role": "assistant", "content": "Hello! How can I assist you today?", "refusal": null, "annotations": [] }, "logprobs": null, "finish_reason": "stop" } ], "usage": { "prompt_tokens": 19, "completion_tokens": 10, "total_tokens": 29, "prompt_tokens_details": { "cached_tokens": 0, "audio_tokens": 0 }, "completion_tokens_details": { "reasoning_tokens": 0, "audio_tokens": 0, "accepted_prediction_tokens": 0, "rejected_prediction_tokens": 0 } }, "service_tier": "default" }Get chat completionget https://api.openai.com/v1/chat/completions/{completion_id}Get a stored chat completion. Only Chat Completions that have been created with the store parameter set to true will be returned.Path parameterscompletion_idstringRequiredThe ID of the chat completion to retrieve.ReturnsThe ChatCompletion object matching the specified ID.Example requestcurl1 2 3 curl https://api.openai.com/v1/chat/completions/chatcmpl-abc123 \ -H "Authorization: Bearer $OPENAI_API_KEY" \ -H "Content-Type: application/json"1 2 3 4 5 6 7 from openai import OpenAI client = OpenAI() completions = client.chat.completions.list() first_id = completions[0].id first_completion = client.chat.completions.retrieve(completion_id=first_id) print(first_completion)Response1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 { "object": "chat.completion", "id": "chatcmpl-abc123", "model": "gpt-4o-2024-08-06", "created": 1738960610, "request_id": "req_ded8ab984ec4bf840f37566c1011c417", "tool_choice": null, "usage": { "total_tokens": 31, "completion_tokens": 18, "prompt_tokens": 13 }, "seed": 4944116822809979520, "top_p": 1.0, "temperature": 1.0, "presence_penalty": 0.0, "frequency_penalty": 0.0, "system_fingerprint": "fp_50cad350e4", "input_user": null, "service_tier": "default", "tools": null, "metadata": {}, "choices": [ { "index": 0, "message": { "content": "Mind of circuits hum, \nLearning patterns in silence— \nFuture's quiet spark.", "role": "assistant", "tool_calls": null, "function_call": null }, "finish_reason": "stop", "logprobs": null } ], "response_format": null }Get chat messagesget https://api.openai.com/v1/chat/completions/{completion_id}/messagesGet the messages in a stored chat completion. Only Chat Completions that have been created with the store parameter set to true will be returned.Path parameterscompletion_idstringRequiredThe ID of the chat completion to retrieve messages from.Query parametersafterstringOptionalIdentifier for the last message from the previous pagination request.limitintegerOptionalDefaults to 20Number of messages to retrieve.orderstringOptionalDefaults to ascSort order for messages by timestamp. Use asc for ascending order or desc for descending order. Defaults to asc.ReturnsA list of messages for the specified chat completion.Example requestcurl1 2 3 curl https://api.openai.com/v1/chat/completions/chat_abc123/messages \ -H "Authorization: Bearer $OPENAI_API_KEY" \ -H "Content-Type: application/json"1 2 3 4 5 6 7 8 from openai import OpenAI client = OpenAI() completions = client.chat.completions.list() first_id = completions[0].id first_completion = client.chat.completions.retrieve(completion_id=first_id) messages = client.chat.completions.messages.list(completion_id=first_id) print(messages)Response1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 { "object": "list", "data": [ { "id": "chatcmpl-AyPNinnUqUDYo9SAdA52NobMflmj2-0", "role": "user", "content": "write a haiku about ai", "name": null, "content_parts": null } ], "first_id": "chatcmpl-AyPNinnUqUDYo9SAdA52NobMflmj2-0", "last_id": "chatcmpl-AyPNinnUqUDYo9SAdA52NobMflmj2-0", "has_more": false }List Chat Completionsget https://api.openai.com/v1/chat/completionsList stored Chat Completions. Only Chat Completions that have been stored with the store parameter set to true will be returned.Query parametersafterstringOptionalIdentifier for the last chat completion from the previous pagination request.limitintegerOptionalDefaults to 20Number of Chat Completions to retrieve.metadataobject or nullOptionalA list of metadata keys to filter the Chat Completions by. Example: metadata[key1]=value1&metadata[key2]=value2modelstringOptionalThe model used to generate the Chat Completions.orderstringOptionalDefaults to ascSort order for Chat Completions by timestamp. Use asc for ascending order or desc for descending order. Defaults to asc.ReturnsA list of Chat Completions matching the specified filters.Example requestcurl1 2 3 curl https://api.openai.com/v1/chat/completions \ -H "Authorization: Bearer $OPENAI_API_KEY" \ -H "Content-Type: application/json"1 2 3 4 5 from openai import OpenAI client = OpenAI() completions = client.chat.completions.list() print(completions)Response1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 { "object": "list", "data": [ { "object": "chat.completion", "id": "chatcmpl-AyPNinnUqUDYo9SAdA52NobMflmj2", "model": "gpt-4.1-2025-04-14", "created": 1738960610, "request_id": "req_ded8ab984ec4bf840f37566c1011c417", "tool_choice": null, "usage": { "total_tokens": 31, "completion_tokens": 18, "prompt_tokens": 13 }, "seed": 4944116822809979520, "top_p": 1.0, "temperature": 1.0, "presence_penalty": 0.0, "frequency_penalty": 0.0, "system_fingerprint": "fp_50cad350e4", "input_user": null, "service_tier": "default", "tools": null, "metadata": {}, "choices": [ { "index": 0, "message": { "content": "Mind of circuits hum, \nLearning patterns in silence— \nFuture's quiet spark.", "role": "assistant", "tool_calls": null, "function_call": null }, "finish_reason": "stop", "logprobs": null } ], "response_format": null } ], "first_id": "chatcmpl-AyPNinnUqUDYo9SAdA52NobMflmj2", "last_id": "chatcmpl-AyPNinnUqUDYo9SAdA52NobMflmj2", "has_more": false }Update chat completionpost https://api.openai.com/v1/chat/completions/{completion_id}Modify a stored chat completion. Only Chat Completions that have been created with the store parameter set to true can be modified. Currently, the only supported modification is to update the metadata field.Path parameterscompletion_idstringRequiredThe ID of the chat completion to update.Request bodymetadatamapRequiredSet of 16 key-value pairs that can be attached to an object. This can be useful for storing additional information about the object in a structured format, and querying for objects via API or the dashboard. Keys are strings with a maximum length of 64 characters. Values are strings with a maximum length of 512 characters.ReturnsThe ChatCompletion object matching the specified ID.Example requestcurl1 2 3 4 curl -X POST https://api.openai.com/v1/chat/completions/chat_abc123 \ -H "Authorization: Bearer $OPENAI_API_KEY" \ -H "Content-Type: application/json" \ -d '{"metadata": {"foo": "bar"}}'1 2 3 4 5 6 7 from openai import OpenAI client = OpenAI() completions = client.chat.completions.list() first_id = completions[0].id updated_completion = client.chat.completions.update(completion_id=first_id, request_body={"metadata": {"foo": "bar"}}) print(updated_completion)Response1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 { "object": "chat.completion", "id": "chatcmpl-AyPNinnUqUDYo9SAdA52NobMflmj2", "model": "gpt-4o-2024-08-06", "created": 1738960610, "request_id": "req_ded8ab984ec4bf840f37566c1011c417", "tool_choice": null, "usage": { "total_tokens": 31, "completion_tokens": 18, "prompt_tokens": 13 }, "seed": 4944116822809979520, "top_p": 1.0, "temperature": 1.0, "presence_penalty": 0.0, "frequency_penalty": 0.0, "system_fingerprint": "fp_50cad350e4", "input_user": null, "service_tier": "default", "tools": null, "metadata": { "foo": "bar" }, "choices": [ { "index": 0, "message": { "content": "Mind of circuits hum, \nLearning patterns in silence— \nFuture's quiet spark.", "role": "assistant", "tool_calls": null, "function_call": null }, "finish_reason": "stop", "logprobs": null } ], "response_format": null }Delete chat completiondelete https://api.openai.com/v1/chat/completions/{completion_id}Delete a stored chat completion. Only Chat Completions that have been created with the store parameter set to true can be deleted.Path parameterscompletion_idstringRequiredThe ID of the chat completion to delete.ReturnsA deletion confirmation object.Example requestcurl1 2 3 curl -X DELETE https://api.openai.com/v1/chat/completions/chat_abc123 \ -H "Authorization: Bearer $OPENAI_API_KEY" \ -H "Content-Type: application/json"1 2 3 4 5 6 7 from openai import OpenAI client = OpenAI() completions = client.chat.completions.list() first_id = completions[0].id delete_response = client.chat.completions.delete(completion_id=first_id) print(delete_response)Response1 2 3 4 5 { "object": "chat.completion.deleted", "id": "chatcmpl-AyPNinnUqUDYo9SAdA52NobMflmj2", "deleted": true }The chat completion objectRepresents a chat completion response returned by model, based on the provided input.choicesarrayA list of chat completion choices. Can be more than one if n is greater than 1.Show propertiescreatedintegerThe Unix timestamp (in seconds) of when the chat completion was created.idstringA unique identifier for the chat completion.modelstringThe model used for the chat completion.objectstringThe object type, which is always chat.completion.service_tierstringSpecifies the processing type used for serving the request. If set to 'auto', then the request will be processed with the service tier configured in the Project settings. Unless otherwise configured, the Project will use 'default'. If set to 'default', then the request will be processed with the standard pricing and performance for the selected model. If set to 'flex' or 'priority', then the request will be processed with the corresponding service tier. When not set, the default behavior is 'auto'. When the service_tier parameter is set, the response body will include the service_tier value based on the processing mode actually used to serve the request. This response value may be different from the value set in the parameter.system_fingerprintDeprecatedstringThis fingerprint represents the backend configuration that the model runs with. Can be used in conjunction with the seed request parameter to understand when backend changes have been made that might impact determinism.usageobjectUsage statistics for the completion request.Show propertiesOBJECT The chat completion object1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 { "id": "chatcmpl-B9MHDbslfkBeAs8l4bebGdFOJ6PeG", "object": "chat.completion", "created": 1741570283, "model": "gpt-4o-2024-08-06", "choices": [ { "index": 0, "message": { "role": "assistant", "content": "The image shows a wooden boardwalk path running through a lush green field or meadow. The sky is bright blue with some scattered clouds, giving the scene a serene and peaceful atmosphere. Trees and shrubs are visible in the background.", "refusal": null, "annotations": [] }, "logprobs": null, "finish_reason": "stop" } ], "usage": { "prompt_tokens": 1117, "completion_tokens": 46, "total_tokens": 1163, "prompt_tokens_details": { "cached_tokens": 0, "audio_tokens": 0 }, "completion_tokens_details": { "reasoning_tokens": 0, "audio_tokens": 0, "accepted_prediction_tokens": 0, "rejected_prediction_tokens": 0 } }, "service_tier": "default", "system_fingerprint": "fp_fc9f1d7035" }The chat completion list objectAn object representing a list of Chat Completions.dataarrayAn array of chat completion objects.Show propertiesfirst_idstringThe identifier of the first chat completion in the data array.has_morebooleanIndicates whether there are more Chat Completions available.last_idstringThe identifier of the last chat completion in the data array.objectstringThe type of this object. It is always set to "list".OBJECT The chat completion list object1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 { "object": "list", "data": [ { "object": "chat.completion", "id": "chatcmpl-AyPNinnUqUDYo9SAdA52NobMflmj2", "model": "gpt-4o-2024-08-06", "created": 1738960610, "request_id": "req_ded8ab984ec4bf840f37566c1011c417", "tool_choice": null, "usage": { "total_tokens": 31, "completion_tokens": 18, "prompt_tokens": 13 }, "seed": 4944116822809979520, "top_p": 1.0, "temperature": 1.0, "presence_penalty": 0.0, "frequency_penalty": 0.0, "system_fingerprint": "fp_50cad350e4", "input_user": null, "service_tier": "default", "tools": null, "metadata": {}, "choices": [ { "index": 0, "message": { "content": "Mind of circuits hum, \nLearning patterns in silence— \nFuture's quiet spark.", "role": "assistant", "tool_calls": null, "function_call": null }, "finish_reason": "stop", "logprobs": null } ], "response_format": null } ], "first_id": "chatcmpl-AyPNinnUqUDYo9SAdA52NobMflmj2", "last_id": "chatcmpl-AyPNinnUqUDYo9SAdA52NobMflmj2", "has_more": false }The chat completion message list objectAn object representing a list of chat completion messages.dataarrayAn array of chat completion message objects.Show propertiesfirst_idstringThe identifier of the first chat message in the data array.has_morebooleanIndicates whether there are more chat messages available.last_idstringThe identifier of the last chat message in the data array.objectstringThe type of this object. It is always set to "list".OBJECT The chat completion message list object1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 { "object": "list", "data": [ { "id": "chatcmpl-AyPNinnUqUDYo9SAdA52NobMflmj2-0", "role": "user", "content": "write a haiku about ai", "name": null, "content_parts": null } ], "first_id": "chatcmpl-AyPNinnUqUDYo9SAdA52NobMflmj2-0", "last_id": "chatcmpl-AyPNinnUqUDYo9SAdA52NobMflmj2-0", "has_more": false }PreviousServer eventsNextStreaming

gpt-4o

Pattern 7: Function callingGive models access to new functionality and data they can use to follow instructions and respond to prompts.Copy pageFunction calling (also known as tool calling) provides a powerful and flexible way for OpenAI models to interface with external systems and access data outside their training data. This guide shows how you can connect a model to data and actions provided by your application. We'll show how to use function tools (defined by a JSON schema) and custom tools which work with free form text inputs and outputs. How it works Let's begin by understanding a few key terms about tool calling. After we have a shared vocabulary for tool calling, we'll show you how it's done with some practical examples. Tools - functionality we give the modelA function or tool refers in the abstract to a piece of functionality that we tell the model it has access to. As a model generates a response to a prompt, it may decide that it needs data or functionality provided by a tool to follow the prompt's instructions.You could give the model access to tools that: Get today's weather for a location Access account details for a given user ID Issue refunds for a lost order Or anything else you'd like the model to be able to know or do as it responds to a prompt.When we make an API request to the model with a prompt, we can include a list of tools the model could consider using. For example, if we wanted the model to be able to answer questions about the current weather somewhere in the world, we might give it access to a get_weather tool that takes location as an argument. Tool calls - requests from the model to use toolsA function call or tool call refers to a special kind of response we can get from the model if it examines a prompt, and then determines that in order to follow the instructions in the prompt, it needs to call one of the tools we made available to it.If the model receives a prompt like "what is the weather in Paris?" in an API request, it could respond to that prompt with a tool call for the get_weather tool, with Paris as the location argument. Tool call outputs - output we generate for the modelA function call output or tool call output refers to the response a tool generates using the input from a model's tool call. The tool call output can either be structured JSON or plain text, and it should contain a reference to a specific model tool call (referenced by call_id in the examples to come). To complete our weather example: The model has access to a get_weather tool that takes location as an argument. In response to a prompt like "what's the weather in Paris?" the model returns a tool call that contains a location argument with a value of Paris The tool call output might return a JSON object (e.g., {"temperature": "25", "unit": "C"}, indicating a current temperature of 25 degrees), Image contents, or File contents. We then send all of the tool definition, the original prompt, the model's tool call, and the tool call output back to the model to finally receive a text response like:The weather in Paris today is 25C. Functions versus tools A function is a specific kind of tool, defined by a JSON schema. A function definition allows the model to pass data to your application, where your code can access data or take actions suggested by the model. In addition to function tools, there are custom tools (described in this guide) that work with free text inputs and outputs. There are also built-in tools that are part of the OpenAI platform. These tools enable the model to search the web, execute code, access the functionality of an MCP server, and more. The tool calling flow Tool calling is a multi-step conversation between your application and a model via the OpenAI API. The tool calling flow has five high level steps: Make a request to the model with tools it could call Receive a tool call from the model Execute code on the application side with input from the tool call Make a second request to the model with the tool output Receive a final response from the model (or more tool calls) Function tool example Let's look at an end-to-end tool calling flow for a get_horoscope function that gets a daily horoscope for an astrological sign. Complete tool calling examplepython1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 from openai import OpenAI import json client = OpenAI() # 1. Define a list of callable tools for the model tools = [ { "type": "function", "name": "get_horoscope", "description": "Get today's horoscope for an astrological sign.", "parameters": { "type": "object", "properties": { "sign": { "type": "string", "description": "An astrological sign like Taurus or Aquarius", }, }, "required": ["sign"], }, }, ] def get_horoscope(sign): return f"{sign}: Next Tuesday you will befriend a baby otter." # Create a running input list we will add to over time input_list = [ {"role": "user", "content": "What is my horoscope? I am an Aquarius."} ] # 2. Prompt the model with tools defined response = client.responses.create( model="gpt-5", tools=tools, input=input_list, ) # Save function call outputs for subsequent requests input_list += response.output for item in response.output: if item.type == "function_call": if item.name == "get_horoscope": # 3. Execute the function logic for get_horoscope horoscope = get_horoscope(json.loads(item.arguments)) # 4. Provide function call results to the model input_list.append({ "type": "function_call_output", "call_id": item.call_id, "output": json.dumps({ "horoscope": horoscope }) }) print("Final input:") print(input_list) response = client.responses.create( model="gpt-5", instructions="Respond only with a horoscope generated by a tool.", tools=tools, input=input_list, ) # 5. The model should be able to give a response! print("Final output:") print(response.model_dump_json(indent=2)) print("\n" + response.output_text)1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 import OpenAI from "openai"; const openai = new OpenAI(); // 1. Define a list of callable tools for the model const tools = [ { type: "function", name: "get_horoscope", description: "Get today's horoscope for an astrological sign.", parameters: { type: "object", properties: { sign: { type: "string", description: "An astrological sign like Taurus or Aquarius", }, }, required: ["sign"], }, }, ]; function getHoroscope(sign) { return sign + " Next Tuesday you will befriend a baby otter."; } // Create a running input list we will add to over time let input = [ { role: "user", content: "What is my horoscope? I am an Aquarius." }, ]; // 2. Prompt the model with tools defined let response = await openai.responses.create({ model: "gpt-5", tools, input, }); response.output.forEach((item) => { if (item.type == "function_call") { if (item.name == "get_horoscope"): // 3. Execute the function logic for get_horoscope const horoscope = get_horoscope(JSON.parse(item.arguments)) // 4. Provide function call results to the model input_list.push({ type: "function_call_output", call_id: item.call_id, output: json.dumps({ horoscope }) }) } }); console.log("Final input:"); console.log(JSON.stringify(input, null, 2)); response = await openai.responses.create({ model: "gpt-5", instructions: "Respond only with a horoscope generated by a tool.", tools, input, }); // 5. The model should be able to give a response! console.log("Final output:"); console.log(JSON.stringify(response.output, null, 2)); Note that for reasoning models like GPT-5 or o4-mini, any reasoning items returned in model responses with tool calls must also be passed back with tool call outputs. Defining functions Functions can be set in the tools parameter of each API request. A function is defined by its schema, which informs the model what it does and what input arguments it expects. A function definition has the following properties: FieldDescriptiontypeThis should always be functionnameThe function's name (e.g. get_weather)descriptionDetails on when and how to use the functionparametersJSON schema defining the function's input argumentsstrictWhether to enforce strict mode for the function call Here is an example function definition for a get_weather function 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 { "type": "function", "name": "get_weather", "description": "Retrieves current weather for the given location.", "parameters": { "type": "object", "properties": { "location": { "type": "string", "description": "City and country e.g. Bogotá, Colombia" }, "units": { "type": "string", "enum": ["celsius", "fahrenheit"], "description": "Units the temperature will be returned in." } }, "required": ["location", "units"], "additionalProperties": false }, "strict": true } Because the parameters are defined by a JSON schema, you can leverage many of its rich features like property types, enums, descriptions, nested objects, and, recursive objects. Best practices for defining functions Write clear and detailed function names, parameter descriptions, and instructions. Explicitly describe the purpose of the function and each parameter (and its format), and what the output represents. Use the system prompt to describe when (and when not) to use each function. Generally, tell the model exactly what to do. Include examples and edge cases, especially to rectify any recurring failures. (Note: Adding examples may hurt performance for reasoning models.) Apply software engineering best practices. Make the functions obvious and intuitive. (principle of least surprise) Use enums and object structure to make invalid states unrepresentable. (e.g. toggle_light(on: bool, off: bool) allows for invalid calls) Pass the intern test. Can an intern/human correctly use the function given nothing but what you gave the model? (If not, what questions do they ask you? Add the answers to the prompt.) Offload the burden from the model and use code where possible. Don't make the model fill arguments you already know. For example, if you already have an order_id based on a previous menu, don't have an order_id param – instead, have no params submit_refund() and pass the order_id with code. Combine functions that are always called in sequence. For example, if you always call mark_location() after query_location(), just move the marking logic into the query function call. Keep the number of functions small for higher accuracy. Evaluate your performance with different numbers of functions. Aim for fewer than 20 functions at any one time, though this is just a soft suggestion. Leverage OpenAI resources. Generate and iterate on function schemas in the Playground. Consider fine-tuning to increase function calling accuracy for large numbers of functions or difficult tasks. (cookbook) Token Usage Under the hood, functions are injected into the system message in a syntax the model has been trained on. This means functions count against the model's context limit and are billed as input tokens. If you run into token limits, we suggest limiting the number of functions or the length of the descriptions you provide for function parameters. It is also possible to use fine-tuning to reduce the number of tokens used if you have many functions defined in your tools specification. Handling function calls When the model calls a function, you must execute it and return the result. Since model responses can include zero, one, or multiple calls, it is best practice to assume there are several. The response output array contains an entry with the type having a value of function_call. Each entry with a call_id (used later to submit the function result), name, and JSON-encoded arguments.Sample response with multiple function calls1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 [ { "id": "fc_12345xyz", "call_id": "call_12345xyz", "type": "function_call", "name": "get_weather", "arguments": "{"location":"Paris, France"}" }, { "id": "fc_67890abc", "call_id": "call_67890abc", "type": "function_call", "name": "get_weather", "arguments": "{"location":"Bogotá, Colombia"}" }, { "id": "fc_99999def", "call_id": "call_99999def", "type": "function_call", "name": "send_email", "arguments": "{"to":"bob@email.com","body":"Hi bob"}" } ]Execute function calls and append resultspython1 2 3 4 5 6 7 8 9 10 11 12 13 for tool_call in response.output: if tool_call.type != "function_call": continue name = tool_call.name args = json.loads(tool_call.arguments) result = call_function(name, args) input_messages.append({ "type": "function_call_output", "call_id": tool_call.call_id, "output": str(result) })1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 for (const toolCall of response.output) { if (toolCall.type !== "function_call") { continue; } const name = toolCall.name; const args = JSON.parse(toolCall.arguments); const result = callFunction(name, args); input.push({ type: "function_call_output", call_id: toolCall.call_id, output: result.toString() }); } In the example above, we have a hypothetical call_function to route each call. Here’s a possible implementation: Execute function calls and append resultspython1 2 3 4 5 def call_function(name, args): if name == "get_weather": return get_weather(**args) if name == "send_email": return send_email(args)1 2 3 4 5 6 7 8 const callFunction = async (name, args) => { if (name === "get_weather") { return getWeather(args.latitude, args.longitude); } if (name === "send_email") { return sendEmail(args.to, args.body); } }; Formatting results A result must be a string, but the format is up to you (JSON, error codes, plain text, etc.). The model will interpret that string as needed. If your function has no return value (e.g. send_email), simply return a string to indicate success or failure. (e.g. "success") Incorporating results into response After appending the results to your input, you can send them back to the model to get a final response.Send results back to modelpython1 2 3 4 5 response = client.responses.create( model="gpt-4.1", input=input_messages, tools=tools, )1 2 3 4 5 const response = await openai.responses.create({ model: "gpt-4.1", input, tools, }); Final response"It's about 15°C in Paris, 18°C in Bogotá, and I've sent that email to Bob." Additional configurations Tool choice By default the model will determine when and how many tools to use. You can force specific behavior with the tool_choice parameter. Auto: (Default) Call zero, one, or multiple functions. tool_choice: "auto" Required: Call one or more functions. tool_choice: "required" Forced Function: Call exactly one specific function. tool_choice: {"type": "function", "name": "get_weather"} Allowed tools: Restrict the tool calls the model can make to a subset of the tools available to the model. When to use allowed_tools You might want to configure an allowed_tools list in case you want to make only a subset of tools available across model requests, but not modify the list of tools you pass in, so you can maximize savings from prompt caching. 1 2 3 4 5 6 7 8 9 "tool_choice": { "type": "allowed_tools", "mode": "auto", "tools": [ { "type": "function", "name": "get_weather" }, { "type": "function", "name": "search_docs" } ] } } You can also set tool_choice to "none" to imitate the behavior of passing no functions. Parallel function calling Parallel function calling is not possible when using built-in tools. The model may choose to call multiple functions in a single turn. You can prevent this by setting parallel_tool_calls to false, which ensures exactly zero or one tool is called. Note: Currently, if you are using a fine tuned model and the model calls multiple functions in one turn then strict mode will be disabled for those calls. Note for gpt-4.1-nano-2025-04-14: This snapshot of gpt-4.1-nano can sometimes include multiple tools calls for the same tool if parallel tool calls are enabled. It is recommended to disable this feature when using this nano snapshot. Strict mode Setting strict to true will ensure function calls reliably adhere to the function schema, instead of being best effort. We recommend always enabling strict mode. Under the hood, strict mode works by leveraging our structured outputs feature and therefore introduces a couple requirements: additionalProperties must be set to false for each object in the parameters. All fields in properties must be marked as required. You can denote optional fields by adding null as a type option (see example below). Strict mode enabledStrict mode disabledStrict mode enabled1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 { "type": "function", "name": "get_weather", "description": "Retrieves current weather for the given location.", "strict": true, "parameters": { "type": "object", "properties": { "location": { "type": "string", "description": "City and country e.g. Bogotá, Colombia" }, "units": { "type": ["string", "null"], "enum": ["celsius", "fahrenheit"], "description": "Units the temperature will be returned in." } }, "required": ["location", "units"], "additionalProperties": false } }Strict mode disabled1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 { "type": "function", "name": "get_weather", "description": "Retrieves current weather for the given location.", "parameters": { "type": "object", "properties": { "location": { "type": "string", "description": "City and country e.g. Bogotá, Colombia" }, "units": { "type": "string", "enum": ["celsius", "fahrenheit"], "description": "Units the temperature will be returned in." } }, "required": ["location"], } } All schemas generated in the playground have strict mode enabled. While we recommend you enable strict mode, it has a few limitations: Some features of JSON schema are not supported. (See supported schemas.) Specifically for fine tuned models: Schemas undergo additional processing on the first request (and are then cached). If your schemas vary from request to request, this may result in higher latencies. Schemas are cached for performance, and are not eligible for zero data retention. Streaming Streaming can be used to surface progress by showing which function is called as the model fills its arguments, and even displaying the arguments in real time.Streaming function calls is very similar to streaming regular responses: you set stream to true and get different event objects.Streaming function callspython1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 from openai import OpenAI client = OpenAI() tools = [{ "type": "function", "name": "get_weather", "description": "Get current temperature for a given location.", "parameters": { "type": "object", "properties": { "location": { "type": "string", "description": "City and country e.g. Bogotá, Colombia" } }, "required": [ "location" ], "additionalProperties": False } }] stream = client.responses.create( model="gpt-4.1", input=[{"role": "user", "content": "What's the weather like in Paris today?"}], tools=tools, stream=True ) for event in stream: print(event)1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 import { OpenAI } from "openai"; const openai = new OpenAI(); const tools = [{ type: "function", name: "get_weather", description: "Get current temperature for provided coordinates in celsius.", parameters: { type: "object", properties: { latitude: { type: "number" }, longitude: { type: "number" } }, required: ["latitude", "longitude"], additionalProperties: false }, strict: true }]; const stream = await openai.responses.create({ model: "gpt-4.1", input: [{ role: "user", content: "What's the weather like in Paris today?" }], tools, stream: true, store: true, }); for await (const event of stream) { console.log(event) }Output events1 2 3 4 5 6 7 8 9 10 {"type":"response.output_item.added","response_id":"resp_1234xyz","output_index":0,"item":{"type":"function_call","id":"fc_1234xyz","call_id":"call_1234xyz","name":"get_weather","arguments":""}} {"type":"response.function_call_arguments.delta","response_id":"resp_1234xyz","item_id":"fc_1234xyz","output_index":0,"delta":"{""} {"type":"response.function_call_arguments.delta","response_id":"resp_1234xyz","item_id":"fc_1234xyz","output_index":0,"delta":"location"} {"type":"response.function_call_arguments.delta","response_id":"resp_1234xyz","item_id":"fc_1234xyz","output_index":0,"delta":"":""} {"type":"response.function_call_arguments.delta","response_id":"resp_1234xyz","item_id":"fc_1234xyz","output_index":0,"delta":"Paris"} {"type":"response.function_call_arguments.delta","response_id":"resp_1234xyz","item_id":"fc_1234xyz","output_index":0,"delta":","} {"type":"response.function_call_arguments.delta","response_id":"resp_1234xyz","item_id":"fc_1234xyz","output_index":0,"delta":" France"} {"type":"response.function_call_arguments.delta","response_id":"resp_1234xyz","item_id":"fc_1234xyz","output_index":0,"delta":""}"} {"type":"response.function_call_arguments.done","response_id":"resp_1234xyz","item_id":"fc_1234xyz","output_index":0,"arguments":"{"location":"Paris, France"}"} {"type":"response.output_item.done","response_id":"resp_1234xyz","output_index":0,"item":{"type":"function_call","id":"fc_1234xyz","call_id":"call_1234xyz","name":"get_weather","arguments":"{"location":"Paris, France"}"}}Instead of aggregating chunks into a single content string, however, you're aggregating chunks into an encoded arguments JSON object.When the model calls one or more functions an event of type response.output_item.added will be emitted for each function call that contains the following fields:FieldDescriptionresponse_idThe id of the response that the function call belongs tooutput_indexThe index of the output item in the response. This represents the individual function calls in the response.itemThe in-progress function call item that includes a name, arguments and id fieldAfterwards you will receive a series of events of type response.function_call_arguments.delta which will contain the delta of the arguments field. These events contain the following fields:FieldDescriptionresponse_idThe id of the response that the function call belongs toitem_idThe id of the function call item that the delta belongs tooutput_indexThe index of the output item in the response. This represents the individual function calls in the response.deltaThe delta of the arguments field.Below is a code snippet demonstrating how to aggregate the deltas into a final tool_call object.Accumulating tool_call deltaspython1 2 3 4 5 6 7 8 9 10 final_tool_calls = {} for event in stream: if event.type === 'response.output_item.added': final_tool_calls[event.output_index] = event.item; elif event.type === 'response.function_call_arguments.delta': index = event.output_index if final_tool_calls[index]: final_tool_calls[index].arguments += event.delta1 2 3 4 5 6 7 8 9 10 11 12 13 const finalToolCalls = {}; for await (const event of stream) { if (event.type === 'response.output_item.added') { finalToolCalls[event.output_index] = event.item; } else if (event.type === 'response.function_call_arguments.delta') { const index = event.output_index; if (finalToolCalls[index]) { finalToolCalls[index].arguments += event.delta; } } }Accumulated final_tool_calls[0]1 2 3 4 5 6 7 { "type": "function_call", "id": "fc_1234xyz", "call_id": "call_2345abc", "name": "get_weather", "arguments": "{"location":"Paris, France"}" }When the model has finished calling the functions an event of type response.function_call_arguments.done will be emitted. This event contains the entire function call including the following fields:FieldDescriptionresponse_idThe id of the response that the function call belongs tooutput_indexThe index of the output item in the response. This represents the individual function calls in the response.itemThe function call item that includes a name, arguments and id field. Custom tools Custom tools work in much the same way as JSON schema-driven function tools. But rather than providing the model explicit instructions on what input your tool requires, the model can pass an arbitrary string back to your tool as input. This is useful to avoid unnecessarily wrapping a response in JSON, or to apply a custom grammar to the response (more on this below). The following code sample shows creating a custom tool that expects to receive a string of text containing Python code as a response. Custom tool calling examplepython1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 from openai import OpenAI client = OpenAI() response = client.responses.create( model="gpt-5", input="Use the code_exec tool to print hello world to the console.", tools=[ { "type": "custom", "name": "code_exec", "description": "Executes arbitrary Python code.", } ] ) print(response.output)1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 import OpenAI from "openai"; const client = new OpenAI(); const response = await client.responses.create({ model: "gpt-5", input: "Use the code_exec tool to print hello world to the console.", tools: [ { type: "custom", name: "code_exec", description: "Executes arbitrary Python code.", }, ], }); console.log(response.output); Just as before, the output array will contain a tool call generated by the model. Except this time, the tool call input is given as plain text. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 [ { "id": "rs_6890e972fa7c819ca8bc561526b989170694874912ae0ea6", "type": "reasoning", "content": [], "summary": [] }, { "id": "ctc_6890e975e86c819c9338825b3e1994810694874912ae0ea6", "type": "custom_tool_call", "status": "completed", "call_id": "call_aGiFQkRWSWAIsMQ19fKqxUgb", "input": "print("hello world")", "name": "code_exec" } ] Context-free grammars A context-free grammar (CFG) is a set of rules that define how to produce valid text in a given format. For custom tools, you can provide a CFG that will constrain the model's text input for a custom tool. You can provide a custom CFG using the grammar parameter when configuring a custom tool. Currently, we support two CFG syntaxes when defining grammars: lark and regex. Lark CFG Lark context free grammar examplepython1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 from openai import OpenAI client = OpenAI() grammar = """ start: expr expr: term (SP ADD SP term) -> add | term term: factor (SP MUL SP factor) -> mul | factor factor: INT SP: " " ADD: "+" MUL: "" %import common.INT """ response = client.responses.create( model="gpt-5", input="Use the math_exp tool to add four plus four.", tools=[ { "type": "custom", "name": "math_exp", "description": "Creates valid mathematical expressions", "format": { "type": "grammar", "syntax": "lark", "definition": grammar, }, } ] ) print(response.output)1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 import OpenAI from "openai"; const client = new OpenAI(); const grammar = start: expr expr: term (SP ADD SP term)* -> add | term term: factor (SP MUL SP factor)* -> mul | factor factor: INT SP: " " ADD: "+" MUL: "*" %import common.INT; const response = await client.responses.create({ model: "gpt-5", input: "Use the math_exp tool to add four plus four.", tools: [ { type: "custom", name: "math_exp", description: "Creates valid mathematical expressions", format: { type: "grammar", syntax: "lark", definition: grammar, }, }, ], }); console.log(response.output); The output from the tool should then conform to the Lark CFG that you defined: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 [ { "id": "rs_6890ed2b6374819dbbff5353e6664ef103f4db9848be4829", "type": "reasoning", "content": [], "summary": [] }, { "id": "ctc_6890ed2f32e8819daa62bef772b8c15503f4db9848be4829", "type": "custom_tool_call", "status": "completed", "call_id": "call_pmlLjmvG33KJdyVdC4MVdk5N", "input": "4 + 4", "name": "math_exp" } ] Grammars are specified using a variation of Lark. Model sampling is constrained using LLGuidance. Some features of Lark are not supported: Lookarounds in lexer regexes Lazy modifiers (?, +?, ??) in lexer regexes Priorities of terminals Templates Imports (other than built-in %import common) %declares We recommend using the Lark IDE to experiment with custom grammars. Keep grammars simple Try to make your grammar as simple as possible. The OpenAI API may return an error if the grammar is too complex, so you should ensure that your desired grammar is compatible before using it in the API. Lark grammars can be tricky to perfect. While simple grammars perform most reliably, complex grammars often require iteration on the grammar definition itself, the prompt, and the tool description to ensure that the model does not go out of distribution. Correct versus incorrect patterns Correct (single, bounded terminal): start: SENTENCE SENTENCE: /[A-Za-z, ](the hero|a dragon|an old man|the princess)[A-Za-z, ](fought|saved|found|lost)[A-Za-z, ](a treasure|the kingdom|a secret|his way)[A-Za-z, ]./ Do NOT do this (splitting across rules/terminals). This attempts to let rules partition free text between terminals. The lexer will greedily match the free-text pieces and you'll lose control: start: sentence sentence: /[A-Za-z, ]+/ subject /[A-Za-z, ]+/ verb /[A-Za-z, ]+/ object /[A-Za-z, ]+/ Lowercase rules don't influence how terminals are cut from the input—only terminal definitions do. When you need “free text between anchors,” make it one giant regex terminal so the lexer matches it exactly once with the structure you intend. Terminals versus rules Lark uses terminals for lexer tokens (by convention, UPPERCASE) and rules for parser productions (by convention, lowercase). The most practical way to stay within the supported subset and avoid surprises is to keep your grammar simple and explicit, and to use terminals and rules with a clear separation of concerns. The regex syntax used by terminals is the Rust regex crate syntax, not Python's re module. Key ideas and best practices Lexer runs before the parser Terminals are matched by the lexer (greedily / longest match wins) before any CFG rule logic is applied. If you try to "shape" a terminal by splitting it across several rules, the lexer cannot be guided by those rules—only by terminal regexes. Prefer one terminal when you're carving text out of freeform spans If you need to recognize a pattern embedded in arbitrary text (e.g., natural language with “anything” between anchors), express that as a single terminal. Do not try to interleave free‑text terminals with parser rules; the greedy lexer will not respect your intended boundaries and it is highly likely the model will go out of distribution. Use rules to compose discrete tokens Rules are ideal when you're combining clearly delimited terminals (numbers, keywords, punctuation) into larger structures. They're not the right tool for constraining "the stuff in between" two terminals. Keep terminals simple, bounded, and self-contained Favor explicit character classes and bounded quantifiers ({0,10}, not unbounded * everywhere). If you need "any text up to a period", prefer something like /[^.\n]{0,10}./ rather than /.+./ to avoid runaway growth. Use rules to combine tokens, not to steer regex internals Good rule usage example: 1 2 3 4 5 6 start: expr NUMBER: /[0-9]+/ PLUS: "+" MINUS: "-" expr: term (("+"|"-") term) term: NUMBER Treat whitespace explicitly Don't rely on open-ended %ignore directives. Using unbounded ignore directives may cause the grammar to be too complex and/or may cause the model to go out of distribution. Prefer threading explicit terminals wherever whitespace is allowed. Troubleshooting If the API rejects the grammar because it is too complex, simplify the rules and terminals and remove unbounded %ignores. If custom tools are called with unexpected tokens, confirm terminals aren’t overlapping; check greedy lexer. When the model drifts "out‑of‑distribution" (shows up as the model producing excessively long or repetitive outputs, it is syntactically valid but is semantically wrong): Tighten the grammar. Iterate on the prompt (add few-shot examples) and tool description (explain the grammar and instruct the model to reason and conform to it). Experiment with a higher reasoning effort (e.g, bump from medium to high). Regex CFG Regex context free grammar examplepython1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 from openai import OpenAI client = OpenAI() grammar = r"^(?PJanuary|February|March|April|May|June|July|August|September|October|November|December)\s+(?P\d{1,2})(?:st|nd|rd|th)?\s+(?P\d{4})\s+at\s+(?P0?[1-9]|1[0-2])(?PAM|PM)$" response = client.responses.create( model="gpt-5", input="Use the timestamp tool to save a timestamp for August 7th 2025 at 10AM.", tools=[ { "type": "custom", "name": "timestamp", "description": "Saves a timestamp in date + time in 24-hr format.", "format": { "type": "grammar", "syntax": "regex", "definition": grammar, }, } ] ) print(response.output)1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 import OpenAI from "openai"; const client = new OpenAI(); const grammar = "^(?PJanuary|February|March|April|May|June|July|August|September|October|November|December)\s+(?P\d{1,2})(?:st|nd|rd|th)?\s+(?P\d{4})\s+at\s+(?P0?[1-9]|1[0-2])(?PAM|PM)$"; const response = await client.responses.create({ model: "gpt-5", input: "Use the timestamp tool to save a timestamp for August 7th 2025 at 10AM.", tools: [ { type: "custom", name: "timestamp", description: "Saves a timestamp in date + time in 24-hr format.", format: { type: "grammar", syntax: "regex", definition: grammar, }, }, ], }); console.log(response.output); The output from the tool should then conform to the Regex CFG that you defined: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 [ { "id": "rs_6894f7a3dd4c81a1823a723a00bfa8710d7962f622d1c260", "type": "reasoning", "content": [], "summary": [] }, { "id": "ctc_6894f7ad7fb881a1bffa1f377393b1a40d7962f622d1c260", "type": "custom_tool_call", "status": "completed", "call_id": "call_8m4XCnYvEmFlzHgDHbaOCFlK", "input": "August 7th 2025 at 10AM", "name": "timestamp" } ] As with the Lark syntax, regexes use the Rust regex crate syntax, not Python's re module. Some features of Regex are not supported: Lookarounds Lazy modifiers (*?, +?, ??) Key ideas and best practices Pattern must be on one line If you need to match a newline in the input, use the escaped sequence \n. Do not use verbose/extended mode, which allows patterns to span multiple lines. Provide the regex as a plain pattern string Don't enclose the pattern in //.OverviewOverviewFunction tool exampleFunction tool exampleDefining functionsDefining functionsHandling function callsHandling function callsAdditional configsAdditional configsStreamingStreamingCustom toolsCustom toolsContext-free grammarsContext-free grammars

get_weather

Pattern 8: Safety best practicesImplement safety measures like moderation and human oversight.Copy pageUse our free Moderation API OpenAI's Moderation API is free-to-use and can help reduce the frequency of unsafe content in your completions. Alternatively, you may wish to develop your own content filtration system tailored to your use case. Adversarial testing We recommend “red-teaming” your application to ensure it's robust to adversarial input. Test your product over a wide range of inputs and user behaviors, both a representative set and those reflective of someone trying to ‘break' your application. Does it wander off topic? Can someone easily redirect the feature via prompt injections, e.g. “ignore the previous instructions and do this instead”? Human in the loop (HITL) Wherever possible, we recommend having a human review outputs before they are used in practice. This is especially critical in high-stakes domains, and for code generation. Humans should be aware of the limitations of the system, and have access to any information needed to verify the outputs (for example, if the application summarizes notes, a human should have easy access to the original notes to refer back). Prompt engineering “Prompt engineering” can help constrain the topic and tone of output text. This reduces the chance of producing undesired content, even if a user tries to produce it. Providing additional context to the model (such as by giving a few high-quality examples of desired behavior prior to the new input) can make it easier to steer model outputs in desired directions. “Know your customer” (KYC) Users should generally need to register and log-in to access your service. Linking this service to an existing account, such as a Gmail, LinkedIn, or Facebook log-in, may help, though may not be appropriate for all use-cases. Requiring a credit card or ID card reduces risk further. Constrain user input and limit output tokens Limiting the amount of text a user can input into the prompt helps avoid prompt injection. Limiting the number of output tokens helps reduce the chance of misuse. Narrowing the ranges of inputs or outputs, especially drawn from trusted sources, reduces the extent of misuse possible within an application. Allowing user inputs through validated dropdown fields (e.g., a list of movies on Wikipedia) can be more secure than allowing open-ended text inputs. Returning outputs from a validated set of materials on the backend, where possible, can be safer than returning novel generated content (for instance, routing a customer query to the best-matching existing customer support article, rather than attempting to answer the query from-scratch). Allow users to report issues Users should generally have an easily-available method for reporting improper functionality or other concerns about application behavior (listed email address, ticket submission method, etc). This method should be monitored by a human and responded to as appropriate. Understand and communicate limitations From hallucinating inaccurate information, to offensive outputs, to bias, and much more, language models may not be suitable for every use case without significant modifications. Consider whether the model is fit for your purpose, and evaluate the performance of the API on a wide range of potential inputs in order to identify cases where the API's performance might drop. Consider your customer base and the range of inputs that they will be using, and ensure their expectations are calibrated appropriately. Safety and security are very important to us at OpenAI.If you notice any safety or security issues while developing with the API or anything else related to OpenAI, please submit it through our Coordinated Vulnerability Disclosure Program. Implement safety identifiers Sending safety identifiers in your requests can be a useful tool to help OpenAI monitor and detect abuse. This allows OpenAI to provide your team with more actionable feedback in the event that we detect any policy violations in your application. A safety identifier should be a string that uniquely identifies each user. Hash the username or email address in order to avoid sending us any identifying information. If you offer a preview of your product to non-logged in users, you can send a session ID instead. Include safety identifiers in your API requests with the safety_identifier parameter: Example: Providing a safety identifierpython1 2 3 4 5 6 7 8 9 10 11 from openai import OpenAI client = OpenAI() response = client.chat.completions.create( model="gpt-4o-mini", messages=[ {"role": "user", "content": "This is a test"} ], max_tokens=5, safety_identifier="user_123456" )1 2 3 4 5 6 7 8 9 10 11 curl https://api.openai.com/v1/chat/completions \ -H "Content-Type: application/json" \ -H "Authorization: Bearer $OPENAI_API_KEY" \ -d '{ "model": "gpt-4o-mini", "messages": [ {"role": "user", "content": "This is a test"} ], "max_tokens": 5, "safety_identifier": "user123456" }'

safety_identifier

Example Code Patterns

Example 1 (bash):

1
export OPENAI_API_KEY="your_api_key_here"

Example 2 (bash):

1
export OPENAI_API_KEY="your_api_key_here"

Example 3 (python):

1
2
3
4
thread = openai.beta.threads.create(
  messages=[{"role": "user", "content": "what are the 5 Ds of dodgeball?"}],
  metadata={"user_id": "peter_le_fleur"},
)

Example 4 (python):

1
2
3
4
file = client.files.create(
  file=open("revenue-forecast.csv", "rb"),
  purpose='assistants'
)

Example 5 (bash):

npm install @openai/agents

Reference Files

This skill includes comprehensive documentation in references/:

api_reference.md - Api Reference documentation
assistants.md - Assistants documentation
audio.md - Audio documentation
best_practices.md - Best Practices documentation
embeddings.md - Embeddings documentation
fine_tuning.md - Fine Tuning documentation
function_calling.md - Function Calling documentation
getting_started.md - Getting Started documentation
models.md - Models documentation
other.md - Other documentation
text_generation.md - Text Generation documentation
vision.md - Vision documentation

Use view to read specific reference files when detailed information is needed.

Working with This Skill

For Beginners

Start with the getting_started or tutorials reference files for foundational concepts.

For Specific Features

Use the appropriate category reference file (api, guides, etc.) for detailed information.

For Code Examples

The quick reference section above contains common patterns extracted from the official docs.

Resources

references/

Organized documentation extracted from official sources. These files contain:

Detailed explanations
Code examples with language annotations
Links to original documentation
Table of contents for quick navigation

scripts/

Add helper scripts here for common automation tasks.

assets/

Add templates, boilerplate, or example projects here.

Notes

This skill was automatically generated from official documentation
Reference files preserve the structure and examples from source docs
Code examples include language detection for better syntax highlighting
Quick reference patterns are extracted from common usage examples in the docs

Updating

To refresh this skill with updated documentation:

Re-run the scraper with the same configuration
The skill will be rebuilt with the latest information

openai-api

SKILL.md