--- description: Run machine learning models, powered by serverless GPUs, on Cloudflare's global network. title: Cloudflare Workers AI image: https://developers.cloudflare.com/workers-ai/og.png?v=28f3ca6015fd2e9e --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/workers-ai/llms.txt > Use this file to discover all available pages before exploring further. # Cloudflare Workers AI Last updated Apr 21, 2026|Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Run machine learning models, powered by serverless GPUs, on Cloudflare's global network. Available on Free and Paid plans Workers AI allows you to run AI models in a serverless way, without having to worry about scaling, maintaining, or paying for unused infrastructure. You can invoke models running on GPUs on Cloudflare's network from your own code — from [Workers](https://developers.cloudflare.com/workers/), [Pages](https://developers.cloudflare.com/pages/), or anywhere via [the Cloudflare API](https://developers.cloudflare.com/api/resources/ai/methods/run/). Workers AI gives you access to: - **50+ [open-source models](https://developers.cloudflare.com/workers-ai/models/)**, available as a part of our model catalog - Serverless, **pay-for-what-you-use** [pricing model](https://developers.cloudflare.com/workers-ai/platform/pricing/) - All as part of a **fully-featured developer platform**, including [AI Gateway](https://developers.cloudflare.com/ai-gateway/), [Vectorize](https://developers.cloudflare.com/vectorize/), [Workers](https://developers.cloudflare.com/workers/) and more... [Get started](https://developers.cloudflare.com/workers-ai/get-started) [Watch a Workers AI demo](https://youtu.be/cK_leoJsBWY?si=4u6BIy_uBOZf9Ve8) Custom requirements If you have custom requirements like private custom models or higher limits, complete the [Custom Requirements Form ↗︎](https://forms.gle/axnnpGDb6xrmR31T6). Cloudflare will contact you with next steps. Workers AI is now Generally Available To report bugs or give feedback, go to the [#workers-ai Discord channel ↗︎](https://discord.cloudflare.com). If you are having issues with Wrangler, report issues in the [Wrangler GitHub repository ↗︎](https://github.com/cloudflare/workers-sdk/issues/new/choose). --- ## Features [Models](https://developers.cloudflare.com/workers-ai/models/) Workers AI comes with a curated set of popular open-source models that enable you to do tasks such as image classification, text generation, object detection and more. Browse models --- ## Related products [AI Gateway](https://developers.cloudflare.com/ai-gateway/) Observe and control your AI applications with caching, rate limiting, request retries, model fallback, and more. [Vectorize](https://developers.cloudflare.com/vectorize/) Build full-stack AI applications with Vectorize, Cloudflare’s vector database. Adding Vectorize enables you to perform tasks such as semantic search, recommendations, anomaly detection or can be used to provide context and memory to an LLM. [Workers](https://developers.cloudflare.com/workers/) Build serverless applications and deploy instantly across the globe for exceptional performance, reliability, and scale. [Pages](https://developers.cloudflare.com/pages/) Create full-stack applications that are instantly deployed to the Cloudflare global network. [R2](https://developers.cloudflare.com/r2/) Store large amounts of unstructured data without the costly egress bandwidth fees associated with typical cloud storage services. [D1](https://developers.cloudflare.com/d1/) Create new serverless SQL databases to query from your Workers and Pages projects. [Durable Objects](https://developers.cloudflare.com/durable-objects/) A globally distributed coordination API with strongly consistent storage. [KV](https://developers.cloudflare.com/kv/) Create a global, low-latency, key-value data storage. --- ## More resources ### [Get started](https://developers.cloudflare.com/workers-ai/get-started/workers-wrangler/) Build and deploy your first Workers AI application. ### [Plans](https://developers.cloudflare.com/workers-ai/platform/pricing/) Learn about Free and Paid plans. ### [Limits](https://developers.cloudflare.com/workers-ai/platform/limits/) Learn about Workers AI limits. ### [Use cases](https://developers.cloudflare.com/use-cases/ai/) Learn how you can build and deploy ambitious AI applications to Cloudflare's global network. ### [Storage options](https://developers.cloudflare.com/workers/platform/storage-options/) Learn which storage option is best for your project. ### [Developer Discord](https://discord.cloudflare.com) Connect with the Workers community on Discord to ask questions, share what you are building, and discuss the platform with other developers. ### [@CloudflareDev](https://x.com/cloudflaredev) Follow @CloudflareDev on Twitter to learn about product announcements, and what is new in Cloudflare Workers. Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"WebPage","@id":"https://developers.cloudflare.com/workers-ai/#page","headline":"Cloudflare Workers AI","description":"Run machine learning models, powered by serverless GPUs, on Cloudflare's global network.","url":"https://developers.cloudflare.com/workers-ai/","inLanguage":"en","image":"https://developers.cloudflare.com/workers-ai/og.png?v=28f3ca6015fd2e9e","dateModified":"2026-04-21","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"},"keywords":["AI"]} ``` --- --- description: Set up your first Workers AI project using the dashboard, CLI, or REST API. title: Getting started image: https://developers.cloudflare.com/workers-ai/get-started/og.png?v=7e846a5a03484ca2 --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/workers-ai/llms.txt > Use this file to discover all available pages before exploring further. # Getting started Last updated Apr 21, 2026|Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/get-started/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) There are several options to build your Workers AI projects on Cloudflare. To get started, choose your preferred method: - [Workers Bindings](https://developers.cloudflare.com/workers-ai/get-started/workers-wrangler/) - [REST API](https://developers.cloudflare.com/workers-ai/get-started/rest-api/) - [Dashboard](https://developers.cloudflare.com/workers-ai/get-started/dashboard/) Note These examples are geared towards creating new Workers AI projects. For help adding Workers AI to an existing Worker, refer to [Workers Bindings](https://developers.cloudflare.com/workers-ai/configuration/bindings/). Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"WebPage","@id":"https://developers.cloudflare.com/workers-ai/get-started/#page","headline":"Getting started","description":"Set up your first Workers AI project using the dashboard, CLI, or REST API.","url":"https://developers.cloudflare.com/workers-ai/get-started/","inLanguage":"en","image":"https://developers.cloudflare.com/workers-ai/get-started/og.png?v=7e846a5a03484ca2","dateModified":"2026-04-21","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Create and deploy a Workers AI application using the Cloudflare dashboard. title: Dashboard image: https://developers.cloudflare.com/workers-ai/get-started/dashboard/og.png?v=eaaf4cda315f2202 --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/workers-ai/llms.txt > Use this file to discover all available pages before exploring further. # Dashboard Last updated Apr 21, 2026|Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/get-started/dashboard/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Follow this guide to create a Workers AI application using the Cloudflare dashboard. ## Prerequisites Sign up for a [Cloudflare account ↗︎](https://dash.cloudflare.com/sign-up/workers-and-pages) if you have not already. ## Setup To create a Workers AI application: 1. In the Cloudflare dashboard, go to the **Workers & Pages** page. [Go to **Workers & Pages** ↗](https://dash.cloudflare.com/?to=/:account/workers-and-pages) 2. Select **Create application**. 3. Under **Select a template**, select **LLM Chat App**. 4. Select **Deploy**. 5. Name your Worker, then select **Create and deploy**. 6. Preview your Worker at its provided [`workers.dev`](https://developers.cloudflare.com/workers/configuration/routing/workers-dev/) subdomain. ## Development ### Dashboard Editing in the dashboard is helpful for simpler use cases. Once you have created your Worker script, you can edit and deploy your Worker using the Cloudflare dashboard: 1. In the Cloudflare dashboard, go to the **Workers & Pages** page. [Go to **Workers & Pages** ↗](https://dash.cloudflare.com/?to=/:account/workers-and-pages) 2. Select your application. 3. Select **Edit Code**. ![Edit code directly within the Cloudflare dashboard](https://developers.cloudflare.com/cdn-cgi/image/onerror=redirect,width=566,height=386,format=webp/_astro/workers-edit-code.CKxxvQSe.png) ### Wrangler CLI To develop more advanced applications or [implement tests](https://developers.cloudflare.com/workers/testing/), start working in the Wrangler CLI. 1. Install [`npm` ↗︎](https://docs.npmjs.com/getting-started). 2. Install [`Node.js` ↗︎](https://nodejs.org/en/). Node.js version manager Use [Vite+ (`vp env`) ↗︎](https://viteplus.dev/guide/env) to install and manage Node.js versions. For supported Node.js versions, refer to the [Wrangler system requirements](https://developers.cloudflare.com/workers/wrangler/install-and-update/#install-wrangler). 3. Run the following command, replacing the value of `[]` which the location you want to put your Worker Script. npmyarnpnpm ``` npm create cloudflare@latest -- [] --type=pre-existing ``` ``` yarn create cloudflare [] --type=pre-existing ``` ``` pnpm create cloudflare@latest [] --type=pre-existing ``` After you run this command - and work through the prompts - your local changes will not automatically sync with dashboard. So, once you download your script, continue using the CLI. Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/get-started/dashboard/#page","headline":"Dashboard","description":"Create and deploy a Workers AI application using the Cloudflare dashboard.","url":"https://developers.cloudflare.com/workers-ai/get-started/dashboard/","inLanguage":"en","image":"https://developers.cloudflare.com/workers-ai/get-started/dashboard/og.png?v=eaaf4cda315f2202","dateModified":"2026-04-21","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Use the Cloudflare Workers AI REST API to deploy a large language model (LLM). title: REST API image: https://developers.cloudflare.com/workers-ai/get-started/rest-api/og.png?v=5b1fb19ca805e26f --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/workers-ai/llms.txt > Use this file to discover all available pages before exploring further. # REST API Last updated Sep 15, 2026|Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/get-started/rest-api/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) This guide will instruct you through setting up and deploying your first Workers AI project. You will use the Workers AI REST API to experiment with a large language model (LLM). ## Prerequisites Sign up for a [Cloudflare account ↗︎](https://dash.cloudflare.com/sign-up/workers-and-pages) if you have not already. ## 1. Get API token and Account ID You need your API token and Account ID to use the REST API. To get these values: 1. In the Cloudflare dashboard, go to the **Workers AI** page. [Go to **Workers AI** ↗](https://dash.cloudflare.com/?to=/:account/ai/workers-ai) 2. Select **Use REST API**. 3. Get your API token: 1. Select **Create a Workers AI API Token**. 2. Review the prefilled information. 3. Select **Create API Token**. 4. Select **Copy API Token**. 5. Save that value for future use. This token will be visible [on your profile](https://developers.cloudflare.com/fundamentals/api/get-started/create-token/). 4. For **Get Account ID**, copy the value for **Account ID**. Save that value for future use. Note If you choose to [create an API token](https://developers.cloudflare.com/fundamentals/api/get-started/create-token/) instead of using the template, that token will need permissions for both `Workers AI - Read` and `Workers AI - Edit`. ## 2. Run a model via API After creating your API token, authenticate and make requests to the API using your API token in the request. You will use the [Execute AI model](https://developers.cloudflare.com/api/resources/ai/methods/run/) endpoint to run the [`@cf/meta/llama-3.1-8b-instruct`](https://developers.cloudflare.com/workers-ai/models/llama-3.1-8b-instruct/) model: ```bash curl https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai/run/@cf/meta/llama-3.1-8b-instruct \ -H 'Authorization: Bearer {API_TOKEN}' \ -d '{ "prompt": "Where did the phrase Hello World come from" }' ``` Replace the values for `{ACCOUNT_ID}` and `{API_TOKEN}`. The API response will look like the following: ```json { "result": { "response": "Hello, World first appeared in 1974 at Bell Labs when Brian Kernighan included it in the C programming language example. It became widely used as a basic test program due to simplicity and clarity. It represents an inviting greeting from a program to the world." }, "success": true, "errors": [], "messages": [] } ``` This example execution uses the `@cf/meta/llama-3.1-8b-instruct` model, but you can use any of the models in the [Workers AI models catalog](https://developers.cloudflare.com/workers-ai/models/). If using another model, you will need to replace `{model}` with your desired model name. By completing this guide, you have created a Cloudflare account (if you did not have one already) and an API token that grants Workers AI read permissions to your account. You executed the [`@cf/meta/llama-3.1-8b-instruct`](https://developers.cloudflare.com/workers-ai/models/llama-3.1-8b-instruct/) model using a cURL command from the terminal and received an answer to your prompt in a JSON response. ## Related resources - [Models](https://developers.cloudflare.com/workers-ai/models/) - Browse the Workers AI models catalog. - [AI SDK](https://developers.cloudflare.com/workers-ai/configuration/ai-sdk) - Learn how to integrate with an AI model. Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/get-started/rest-api/#page","headline":"REST API","description":"Use the Cloudflare Workers AI REST API to deploy a large language model (LLM).","url":"https://developers.cloudflare.com/workers-ai/get-started/rest-api/","inLanguage":"en","image":"https://developers.cloudflare.com/workers-ai/get-started/rest-api/og.png?v=5b1fb19ca805e26f","dateModified":"2026-09-15","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Deploy your first Cloudflare Workers AI project using the CLI. title: Workers Bindings image: https://developers.cloudflare.com/workers-ai/get-started/workers-wrangler/og.png?v=afa2e8cca2301ad1 --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/workers-ai/llms.txt > Use this file to discover all available pages before exploring further. # Workers Bindings Last updated Aug 27, 2026|Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/get-started/workers-wrangler/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) This guide will instruct you through setting up and deploying your first Workers AI project. You will use [Workers](https://developers.cloudflare.com/workers/), a Workers AI binding, and a large language model (LLM) to deploy your first AI-powered application on the Cloudflare global network. 1. Sign up for a [Cloudflare account ↗︎](https://dash.cloudflare.com/sign-up/workers-and-pages). 2. Install [`Node.js` ↗︎](https://docs.npmjs.com/downloading-and-installing-node-js-and-npm).
Node.js version manager Use Vite+ (vp env) ↗︎ to install and manage Node.js versions. For supported Node.js versions, refer to the Wrangler system requirements.
## 1. Create a Worker project You will create a new Worker project using the `create-cloudflare` CLI (C3). [C3 ↗︎](https://github.com/cloudflare/workers-sdk/tree/main/packages/create-cloudflare) is a command-line tool designed to help you set up and deploy new applications to Cloudflare. Create a new project named `hello-ai` by running: npmyarnpnpm ``` npm create cloudflare@latest -- hello-ai ``` ``` yarn create cloudflare hello-ai ``` ``` pnpm create cloudflare@latest hello-ai ``` Running `npm create cloudflare@latest` will prompt you to install the [`create-cloudflare` package ↗︎](https://www.npmjs.com/package/create-cloudflare), and lead you through setup. C3 will also install [Wrangler](https://developers.cloudflare.com/workers/wrangler/), the Cloudflare Developer Platform CLI. For setup, select the following options: - For *What would you like to start with?*, choose `Hello World example`. - For *Which template would you like to use?*, choose `Worker only`. - For *Which language do you want to use?*, choose `TypeScript`. - For *Do you want to use git for version control?*, choose `Yes`. - For *Do you want to add an AGENTS.md file to help AI coding tools understand Cloudflare APIs?*, choose `Yes`. - For *Do you want to deploy your application?*, choose `No` (we will be making some changes before deploying). This will create a new `hello-ai` directory. Your new `hello-ai` directory will include: - A `"Hello World"` [Worker](https://developers.cloudflare.com/workers/get-started/guide/#3-write-code) at `src/index.ts`. - A [`wrangler.jsonc`](https://developers.cloudflare.com/workers/wrangler/configuration/) configuration file. Go to your application directory: ```sh cd hello-ai ``` ## 2. Connect your Worker to Workers AI You must create an AI binding for your Worker to connect to Workers AI. [Bindings](https://developers.cloudflare.com/workers/runtime-apis/bindings/) allow your Workers to interact with resources, like Workers AI, on the Cloudflare Developer Platform. To bind Workers AI to your Worker, add the following to the end of your Wrangler file: ```jsonc { "ai": { "binding": "AI" } } ``` ```toml [ai] binding = "AI" ``` Your binding is [available in your Worker code](https://developers.cloudflare.com/workers/reference/migrate-to-module-workers/#bindings-in-es-modules-format) on [`env.AI`](https://developers.cloudflare.com/workers/runtime-apis/handlers/fetch/). You can also bind Workers AI to a Pages Function. For more information, refer to [Functions Bindings](https://developers.cloudflare.com/pages/functions/bindings/#workers-ai). ## 3. Run an inference task in your Worker You are now ready to run an inference task in your Worker. In this case, you will use an LLM, [`gemma-4-26b-a4b-it`](https://developers.cloudflare.com/workers-ai/models/gemma-4-26b-a4b-it/), to answer a question. Update the `index.ts` file in your `hello-ai` application directory with the following code: *index.jsjs* ```js export default { async fetch(request, env) { const response = await env.AI.run("@cf/google/gemma-4-26b-a4b-it", { messages: [ { role: "system", content: "You are a helpful assistant.", }, { role: "user", content: "What is the origin of the phrase Hello, World", }, ], chat_template_kwargs: { enable_thinking: false, }, }); return Response.json(response); }, }; ``` *index.tsts* ```ts export interface Env { // If you set another name in the Wrangler config file as the value for 'binding', // replace "AI" with the variable name you defined. AI: Ai; } export default { async fetch(request, env): Promise { const response = await env.AI.run("@cf/google/gemma-4-26b-a4b-it", { messages: [ { role: "system", content: "You are a helpful assistant.", }, { role: "user", content: "What is the origin of the phrase Hello, World", }, ], chat_template_kwargs: { enable_thinking: false, }, }); return Response.json(response); }, } satisfies ExportedHandler; ``` Up to this point, you have created an AI binding for your Worker and configured your Worker to execute the Gemma 4 26B A4B model with reasoning disabled. You can now test your project locally before you deploy globally. ## 4. Develop locally with Wrangler While in your project directory, test Workers AI locally by running [`wrangler dev`](https://developers.cloudflare.com/workers/wrangler/commands/general/#dev): ```sh npx wrangler dev ``` Workers AI local development usage charges Using Workers AI always accesses your Cloudflare account in order to run AI models and will incur usage charges even in local development. You will be prompted to log in after you run `wrangler dev`. When you run `npx wrangler dev`, Wrangler will give you a URL (most likely `localhost:8787`) to review your Worker. After you go to the URL Wrangler provides, the response will have a shape similar to the following example: ```json { "id": "", "object": "chat.completion", "created": 0, "model": "@cf/google/gemma-4-26b-a4b-it", "choices": [ { "index": 0, "message": { "role": "assistant", "content": "", "refusal": null }, "finish_reason": "stop", "logprobs": null } ] } ``` ## 5. Deploy your AI Worker Before deploying your AI Worker globally, log in with your Cloudflare account by running: ```sh npx wrangler login ``` You will be directed to a web page asking you to log in to the Cloudflare dashboard. After you have logged in, you will be asked if Wrangler can make changes to your Cloudflare account. Scroll down and select **Allow** to continue. Finally, deploy your Worker to make your project accessible on the Internet. To deploy your Worker, run: ```sh npx wrangler deploy ``` ```sh https://hello-ai..workers.dev ``` Your Worker will be deployed to your custom [`workers.dev`](https://developers.cloudflare.com/workers/configuration/routing/workers-dev/) subdomain. You can now visit the URL to run your AI Worker. By finishing this tutorial, you have created a Worker, connected it to Workers AI through an AI binding, and run an inference task using the Gemma 4 26B A4B model. ## Related resources - [Cloudflare Developers community on Discord ↗︎](https://discord.cloudflare.com) - Submit feature requests, report bugs, and share your feedback directly with the Cloudflare team by joining the Cloudflare Discord server. - [Models](https://developers.cloudflare.com/workers-ai/models/) - Browse the Workers AI models catalog. - [AI SDK](https://developers.cloudflare.com/workers-ai/configuration/ai-sdk) - Learn how to integrate with an AI model. Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/get-started/workers-wrangler/#page","headline":"Workers Bindings","description":"Deploy your first Cloudflare Workers AI project using the CLI.","url":"https://developers.cloudflare.com/workers-ai/get-started/workers-wrangler/","inLanguage":"en","image":"https://developers.cloudflare.com/workers-ai/get-started/workers-wrangler/og.png?v=afa2e8cca2301ad1","dateModified":"2026-08-27","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Understand how extraction models extract, compute, estimate, and forecast typed values, and how to interpret their results. title: Extraction models image: https://developers.cloudflare.com/workers-ai/concepts/extraction-models/og.png?v=ab83122be737302a --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/workers-ai/llms.txt > Use this file to discover all available pages before exploring further. # Extraction models Last updated Oct 19, 2026|Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/concepts/extraction-models/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Extraction models turn information into specific values your application can use, such as an invoice total, a due date, or an estimated payment amount. They return typed values rather than choosing from a fixed set of options or producing open-ended content. This helps you connect AI predictions to application logic. You provide an input state and define the values you want the model to return, including each value's type and any additional constraints. The model uses these definitions and the input state to produce a corresponding typed result for each value. Typed results provide a predictable structure but do not guarantee correctness. ## When to use extraction models Use an extraction model when your application needs to extract, compute, estimate, or forecast a value, such as an amount, date, or duration. Alternatively, use a decision model to select from known options, or a generative model to produce open-ended content. The following table compares these model types using invoice examples: | Model type | Use when you need | Invoice example | | --- | --- | --- | | Extraction | Extract, compute, estimate, or forecast values, such as amounts, dates, durations, or text. | Read the total and due date from an invoice. | | Decision | Select from a predefined set of options. | Determine if an invoice requires approval, manual review, or rejection. | | Generative | Produce open-ended content. | Summarize discrepancies between the invoice, purchase order, and delivery receipt. | Your application can use returned values in application logic or pass them as inputs to another model. ## How extraction models work An extraction request combines an input state with a set of named field definitions. The input state supplies the information the model evaluates. Each field definition describes a value to return, specifies its type, and may include constraints. The model uses both to produce a corresponding result for each field. For example, you provide invoice text that states a total of $1,080 and a due date of 2026-11-01. Separately, you define an invoice total field as a number in USD and a due date field as a date. The model uses these definitions to identify the requested values in the invoice. It returns `1080` and `2026-11-01` as the corresponding typed results. Your application can then use the returned values. For example, it can compare the due date with the current date to flag an overdue invoice. ## Extraction model capabilities Extraction models support extraction, computation, estimation, and forecasting. ### Extraction Return a value explicitly present in the input, such as the total printed on an invoice. Extraction identifies an existing value rather than calculating or estimating one. ### Computation Derive a value from facts in the input. For example, calculate how many days an invoice is overdue using its due date and a current date you provide. Unlike extraction, the result does not need to appear explicitly in the input. ### Estimation Predict a value that the input does not determine, such as when an unpaid invoice will be paid. Unlike computation, estimation addresses an unknown outcome rather than deriving a result from supplied facts. ### Forecasting Predict a sequence of future values over an interval, such as an invoice's outstanding balance each week for the next month. Unlike a single estimate, a forecast returns a value for each step in the requested period. ## Interpret extraction results Types and constraints define the form a result can take. They do not guarantee that it answers your question correctly. ### Missing results For fields that extract or compute a single value, a `null` result indicates that the model considers the input insufficient to determine the answer. The information may be missing, or the model may have failed to identify it. Lists return an empty list when the model finds no matching values. Estimates and forecasts return predictions even when the input provides little information. ### Uncertainty The field type determines how uncertainty is represented and how it can be interpreted. Fields that extract and compute a single value include confidence scores for the returned answer. Estimates and forecasts include distributions over possible values. For example, an estimated payment date might have a 10th percentile of November 5 and a 90th percentile of November 20. These dates mark the middle 80% of the predicted distribution. The actual payment date can still fall outside that interval. ### Constraints Constraints restrict the values the model can return. They do not guarantee that the answer is correct. For example, restricting an estimated payment date to November ensures that the returned date falls in November. The invoice could still be paid in December. ### Source spans Extracted text includes a source span, which identifies its start and end positions in the input. You can use these positions to find the returned text in its original context. ## Design extraction tasks ### Define clear fields Give each field a descriptive name and use its instructions to explain what value to return. For example, a field named `invoice_total` could have the instructions "Total amount due, including tax and shipping, after discounts." Specify its type and unit separately as a number in USD. ### Choose appropriate types and constraints Set each field's type according to how your application uses the result. For example, use a date for a payment deadline and a duration for days overdue. Use the available field settings to specify precision and value bounds. For example, request two decimal places for an invoice amount. When bounding an estimated payment date, choose a window that reflects plausible payment dates, not just your preferred deadline. ### Include task-relevant context Include the information each task depends on in the input state. For example, calculating days overdue requires the invoice's due date and a current date you provide. State which date to use rather than assuming the model knows when the request runs. ### Match safeguards to risk Choose safeguards based on the consequences of an incorrect result. Displaying an estimated payment date may require fewer checks than using an extracted amount to authorize a payment. Decide how your application handles missing or uncertain results. Before consequential or irreversible actions, check results against other evidence or require human review. ## Limitations - Typed output does not guarantee correctness. A result can satisfy its type and constraints while still answering the question incorrectly. - Results depend on the available information and field definitions. Returning an estimate does not mean the input provides enough information to predict the outcome accurately. - Uncertainty estimates are not guarantees. Confidence scores and prediction intervals do not replace application-level checks. An interval's actual coverage can vary across tasks and datasets. - Capabilities vary by model. Consult the [Clairic model documentation](https://developers.cloudflare.com/workers-ai/models/clairic/) for supported inputs, field types, metadata, context window, and model-specific limitations. ## Next steps - Review the [Clairic model reference](https://developers.cloudflare.com/workers-ai/models/clairic/) for supported inputs, field types, model-specific limits, request and response schemas, and invoice examples. - Consult the [Workers AI REST API reference](https://developers.cloudflare.com/api/resources/ai/methods/run/) for the inference endpoint. - [Configure a Workers AI binding](https://developers.cloudflare.com/workers-ai/configuration/bindings/) to access Workers AI from a Worker. - [Use the Workers AI REST API](https://developers.cloudflare.com/workers-ai/get-started/rest-api/) to authenticate and send requests over HTTP. Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/concepts/extraction-models/#page","headline":"Extraction models","description":"Understand how extraction models extract, compute, estimate, and forecast typed values, and how to interpret their results.","url":"https://developers.cloudflare.com/workers-ai/concepts/extraction-models/","inLanguage":"en","image":"https://developers.cloudflare.com/workers-ai/concepts/extraction-models/og.png?v=ab83122be737302a","dateModified":"2026-10-19","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Browse the catalog of machine learning models available on Workers AI. title: Workers AI Models image: https://developers.cloudflare.com/workers-ai/models/og.png?v=40351c4db9a31785 --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/workers-ai/llms.txt > Use this file to discover all available pages before exploring further. # Workers AI Models Last updated Aug 12, 2026|Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/models/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Model not available is deprecated, removed, or not found. Choose an available model from the catalog below. Looking for more models, including external providers? Check out the [unified AI model catalog](https://developers.cloudflare.com/ai/models/). Task TypesCapabilitiesAuthorsNewest first We found 73 modelsClear filters ## Compare models Compare up to three models. Close Clear comparison No models found Try a different search term, or broaden your search by removing filters. [s

apertus-v1.5-8b

swiss-aiText Generation Apertus 1.5 is an 8B parameter language model designed to advance the state of multilingual, multimodal, fully open, and transparent AI. The models support a wide range of languages, handle contexts of up to 262,144 tokens, and it uses only fully open training data whilst delivering performance comparable to other models of similar size. For access, please fill out this form: https://forms.gle/kgvmkr6ucHN3xNuH6](https://developers.cloudflare.com/workers-ai/models/apertus-v1.5-8b/) - Cloudflare-hosted - +3 Compare [![Deepgram logo](https://developers.cloudflare.com/_astro/deepgram.BYzW8KfF.svg)

aura-1

DeepgramText-to-Speech Aura is a context-aware text-to-speech (TTS) model that applies natural pacing, expressiveness, and fillers based on the context of the provided text. The quality of your text input directly impacts the naturalness of the audio output.](https://developers.cloudflare.com/workers-ai/models/aura-1/) - Cloudflare-hosted - +4 Compare [![Deepgram logo](https://developers.cloudflare.com/_astro/deepgram.BYzW8KfF.svg)

aura-2-en

DeepgramText-to-Speech Aura-2 is a context-aware text-to-speech (TTS) model that applies natural pacing, expressiveness, and fillers based on the context of the provided text. The quality of your text input directly impacts the naturalness of the audio output.](https://developers.cloudflare.com/workers-ai/models/aura-2-en/) - Cloudflare-hosted - +4 Compare [![Deepgram logo](https://developers.cloudflare.com/_astro/deepgram.BYzW8KfF.svg)

aura-2-es

DeepgramText-to-Speech Aura-2 is a context-aware text-to-speech (TTS) model that applies natural pacing, expressiveness, and fillers based on the context of the provided text. The quality of your text input directly impacts the naturalness of the audio output.](https://developers.cloudflare.com/workers-ai/models/aura-2-es/) - Cloudflare-hosted - +4 Compare [![BAAI logo](https://developers.cloudflare.com/_astro/baai.BooZR_xF.svg)

bge-base-en-v1.5

BAAIText Embeddings BAAI general embedding (Base) model that transforms any given text into a 768-dimensional vector](https://developers.cloudflare.com/workers-ai/models/bge-base-en-v1.5/) - Cloudflare-hosted - +2 Compare [![BAAI logo](https://developers.cloudflare.com/_astro/baai.BooZR_xF.svg)

bge-large-en-v1.5

BAAIText Embeddings BAAI general embedding (Large) model that transforms any given text into a 1024-dimensional vector](https://developers.cloudflare.com/workers-ai/models/bge-large-en-v1.5/) - Cloudflare-hosted - +2 Compare [![BAAI logo](https://developers.cloudflare.com/_astro/baai.BooZR_xF.svg)

bge-m3

BAAIText Embeddings Multi-Functionality, Multi-Linguality, and Multi-Granularity embeddings model.](https://developers.cloudflare.com/workers-ai/models/bge-m3/) - Cloudflare-hosted - +1 Compare [![BAAI logo](https://developers.cloudflare.com/_astro/baai.BooZR_xF.svg)

bge-reranker-base

BAAIText Classification Different from embedding model, reranker uses question and document as input and directly output similarity instead of embedding. You can get a relevance score by inputting query and passage to the reranker. And the score can be mapped to a float value in \[0,1\] by sigmoid function.](https://developers.cloudflare.com/workers-ai/models/bge-reranker-base/) - Cloudflare-hosted - +1 Compare [![BAAI logo](https://developers.cloudflare.com/_astro/baai.BooZR_xF.svg)

bge-small-en-v1.5

BAAIText Embeddings BAAI general embedding (Small) model that transforms any given text into a 384-dimensional vector](https://developers.cloudflare.com/workers-ai/models/bge-small-en-v1.5/) - Cloudflare-hosted - +2 Compare [Pinned![Cloudflare logo](https://developers.cloudflare.com/_astro/cloudflare.DP8rkHys.svg)

clairic

CloudflareText Generation Clairic is a System One-style model that can extract, compute, estimate, and forecast values from inputs. It is strongly typed and outputs confidence scores and probabilities against its outputs. This model is in preview from the Cloudflare Workers AI team.](https://developers.cloudflare.com/workers-ai/models/clairic/) - Cloudflare-hosted - +2 Compare [Pinned![Cloudflare logo](https://developers.cloudflare.com/_astro/cloudflare.DP8rkHys.svg)

clef

CloudflareText Generation Clef is a 27B multimodal decision model that turns a state and a schema of typed questions into decisions. It reads the state as text, JSON, images, or video, and returns a probability for every allowed option of every question.](https://developers.cloudflare.com/workers-ai/models/clef/) - Cloudflare-hosted - +2 Compare [Pinned![Cloudflare logo](https://developers.cloudflare.com/_astro/cloudflare.DP8rkHys.svg)

clef-flash

CloudflareText Generation Clef-flash is a fast 9B multimodal decision model that turns a state and a schema of typed questions into decisions. It reads the state as text, JSON, images, or video, and returns a probability for every allowed option of every question.](https://developers.cloudflare.com/workers-ai/models/clef-flash/) - Cloudflare-hosted - +2 Compare [Pinned![Cloudflare logo](https://developers.cloudflare.com/_astro/cloudflare.DP8rkHys.svg)

clef-omni

CloudflareText Generation Clef-omni is a multimodal decision model built on a 30B-parameter mixture-of-experts (3B active) backbone. It turns a state and a schema of typed questions into decisions, reads the state as text, JSON, images, audio, or video, and returns a probability for every allowed option of every question.](https://developers.cloudflare.com/workers-ai/models/clef-omni/) - Cloudflare-hosted - +2 Compare [![DeepSeek logo](https://developers.cloudflare.com/_astro/deepseek.CkzAgvN6.svg)

deepseek-r1-distill-qwen-32b

DeepSeekText Generation DeepSeek-R1-Distill-Qwen-32B is a model distilled from DeepSeek-R1 based on Qwen2.5. It outperforms OpenAI-o1-mini across various benchmarks, achieving new state-of-the-art results for dense models.](https://developers.cloudflare.com/workers-ai/models/deepseek-r1-distill-qwen-32b/) - Cloudflare-hosted - +3 Compare [![DeepSeek logo](https://developers.cloudflare.com/_astro/deepseek.CkzAgvN6.svg)

deepseek-v4-flash-0731

DeepSeekText Generation DeepSeek-V4-Flash-0731 is the official release of DeepSeek-V4-Flash, superseding the preview version, with substantially enhanced agentic capabilities.](https://developers.cloudflare.com/workers-ai/models/deepseek-v4-flash-0731/) - Cloudflare-hosted - +5 Compare [![DeepSeek logo](https://developers.cloudflare.com/_astro/deepseek.CkzAgvN6.svg)

deepseek-v4-pro-0813

DeepSeekText Generation DeepSeek V4 Pro is a high-capability reasoning model from DeepSeek with a one million token context window, built for long-horizon agentic workflows and complex, multi-step problem-solving](https://developers.cloudflare.com/workers-ai/models/deepseek-v4-pro-0813/) - Cloudflare-hosted - +4 Compare [![DeepSeek logo](https://developers.cloudflare.com/_astro/deepseek.CkzAgvN6.svg)

deepseek-v4.1-flash-1017

DeepSeekText Generation We introduce DeepSeek-V4.1-Flash, a multimodal Mixture-of-Experts (MoE) model with 552B backbone parameters and support for contexts of up to one million tokens. The model natively processes images and text, and generates text autoregressively.](https://developers.cloudflare.com/workers-ai/models/deepseek-v4.1-flash-1017/) - Cloudflare-hosted - +4 Compare [![HuggingFace logo](https://developers.cloudflare.com/_astro/huggingface.DMS-v5TA.svg)

distilbert-sst-2-int8

HuggingFaceText Classification Distilled BERT model that was finetuned on SST-2 for sentiment classification](https://developers.cloudflare.com/workers-ai/models/distilbert-sst-2-int8/) - Cloudflare-hosted - +1 Compare [l

dreamshaper-8-lcm

lykonText-to-Image Stable Diffusion model that has been fine-tuned to be better at photorealism without sacrificing range.](https://developers.cloudflare.com/workers-ai/models/dreamshaper-8-lcm/) - Cloudflare-hosted - +2 Compare [![Google logo](https://developers.cloudflare.com/_astro/google.DyXKPTPP.svg)

embeddinggemma-300m

BetaGoogleText Embeddings EmbeddingGemma is a 300M parameter, state-of-the-art for its size, open embedding model from Google, built from Gemma 3 (with T5Gemma initialization) and the same research and technology used to create Gemini models. EmbeddingGemma produces vector representations of text, making it well-suited for search and retrieval tasks, including classification, clustering, and semantic similarity search. This model was trained with data in 100+ spoken languages.](https://developers.cloudflare.com/workers-ai/models/embeddinggemma-300m/) - Cloudflare-hosted - +2 Compare [u

eurollm-9b-it

utter-projectText Generation EuroLLM-9B is a 9B parameter model trained on 4 trillion tokens divided across the considered languages and several data sources: Web data, parallel data (en-xx and xx-en), and high-quality datasets. For access, please fill out this form: https://forms.gle/kgvmkr6ucHN3xNuH6](https://developers.cloudflare.com/workers-ai/models/eurollm-9b-it/) - Cloudflare-hosted - +2 Compare [![Deepgram logo](https://developers.cloudflare.com/_astro/deepgram.BYzW8KfF.svg)

flux

DeepgramAutomatic Speech Recognition Flux is the first conversational speech recognition model built specifically for voice agents.](https://developers.cloudflare.com/workers-ai/models/flux/) - Cloudflare-hosted - +3 Compare [![Black Forest Labs logo](https://developers.cloudflare.com/_astro/blackforestlabs.Ccs-Y4-D.svg)

flux-1-schnell

Black Forest LabsText-to-Image FLUX.1 \[schnell\] is a 12 billion parameter rectified flow transformer capable of generating images from text descriptions.](https://developers.cloudflare.com/workers-ai/models/flux-1-schnell/) - Cloudflare-hosted - +1 Compare [![Black Forest Labs logo](https://developers.cloudflare.com/_astro/blackforestlabs.Ccs-Y4-D.svg)

flux-2-dev

Black Forest LabsText-to-Image FLUX.2 \[dev\] is an image model from Black Forest Labs where you can generate highly realistic and detailed images, with multi-reference support.](https://developers.cloudflare.com/workers-ai/models/flux-2-dev/) - Cloudflare-hosted - +2 Compare [![Black Forest Labs logo](https://developers.cloudflare.com/_astro/blackforestlabs.Ccs-Y4-D.svg)

flux-2-klein-4b

Black Forest LabsText-to-Image FLUX.2 \[klein\] is an ultra-fast, distilled image model. It unifies image generation and editing in a single model, delivering state-of-the-art quality enabling interactive workflows, real-time previews, and latency-critical applications.](https://developers.cloudflare.com/workers-ai/models/flux-2-klein-4b/) - Cloudflare-hosted - +2 Compare [![Black Forest Labs logo](https://developers.cloudflare.com/_astro/blackforestlabs.Ccs-Y4-D.svg)

flux-2-klein-9b

Black Forest LabsText-to-Image FLUX.2 \[klein\] 9B is a 9 billion parameter model that can generate images from text descriptions and supports multi-reference editing capabilities.](https://developers.cloudflare.com/workers-ai/models/flux-2-klein-9b/) - Cloudflare-hosted - +2 Compare [![Google logo](https://developers.cloudflare.com/_astro/google.DyXKPTPP.svg)

gemma-2b-it-lora

BetaGoogleText Generation This is a Gemma-2B base model that Cloudflare dedicates for inference with LoRA adapters. Gemma is a family of lightweight, state-of-the-art open models from Google, built from the same research and technology used to create the Gemini models.](https://developers.cloudflare.com/workers-ai/models/gemma-2b-it-lora/) - Cloudflare-hosted - +3 Compare [![Google logo](https://developers.cloudflare.com/_astro/google.DyXKPTPP.svg)

gemma-4-26b-a4b-it

GoogleText Generation Gemma 4 is Google's most intelligent family of open models, built from Gemini 3 research to maximize intelligence-per-parameter.](https://developers.cloudflare.com/workers-ai/models/gemma-4-26b-a4b-it/) - Cloudflare-hosted - +5 Compare [![Google logo](https://developers.cloudflare.com/_astro/google.DyXKPTPP.svg)

gemma-7b-it-lora

BetaGoogleText Generation This is a Gemma-7B base model that Cloudflare dedicates for inference with LoRA adapters. Gemma is a family of lightweight, state-of-the-art open models from Google, built from the same research and technology used to create the Gemini models.](https://developers.cloudflare.com/workers-ai/models/gemma-7b-it-lora/) - Cloudflare-hosted - +3 Compare [a

gemma-sea-lion-v4-27b-it

aisingaporeText Generation SEA-LION stands for Southeast Asian Languages In One Network, which is a collection of Large Language Models (LLMs) which have been pretrained and instruct-tuned for the Southeast Asia (SEA) region.](https://developers.cloudflare.com/workers-ai/models/gemma-sea-lion-v4-27b-it/) - Cloudflare-hosted - +1 Compare [![Zhipu AI logo](https://developers.cloudflare.com/_astro/zai-org.Dj2vcayE.svg)

glm-4.7-flash

Zhipu AIText Generation GLM-4.7-Flash is a fast and efficient multilingual text generation model with a 131,072 token context window. Optimized for dialogue, instruction-following, and multi-turn tool calling across 100+ languages.](https://developers.cloudflare.com/workers-ai/models/glm-4.7-flash/) - Cloudflare-hosted - +3 Compare [![Zhipu AI logo](https://developers.cloudflare.com/_astro/zai-org.Dj2vcayE.svg)

glm-5.2

Zhipu AIText Generation Z.ai's flagship agentic coding model](https://developers.cloudflare.com/workers-ai/models/glm-5.2/) - Cloudflare-hosted - +4 Compare [![Zhipu AI logo](https://developers.cloudflare.com/_astro/zai-org.Dj2vcayE.svg)

glm-5.3

Zhipu AIText Generation GLM-5.3 is Z.ai's flagship agentic coding model, pairing a 1M-token context window with reasoning, function calling, and structured outputs to power multi-step, tool-driven development workflows.](https://developers.cloudflare.com/workers-ai/models/glm-5.3/) - Cloudflare-hosted - +3 Compare [![Zhipu AI logo](https://developers.cloudflare.com/_astro/zai-org.Dj2vcayE.svg)

glm-5.3-flash

Zhipu AIText Generation The first natively multimodal model in the GLM-5 series. With 320B total parameters and just 18B active parameters, it outperforms GLM-5.2 across benchmarks and real-world workloads at one-tenth the price, while approaching Claude Opus 4.8 on coding and agentic benchmarks.](https://developers.cloudflare.com/workers-ai/models/glm-5.3-flash/) - Cloudflare-hosted - +5 Compare [![OpenAI logo](https://developers.cloudflare.com/_astro/openai.BBwNKzBb.svg)

gpt-oss-120b

OpenAIText Generation OpenAI’s open-weight models designed for powerful reasoning, agentic tasks, and versatile developer use cases – gpt-oss-120b is for production, general purpose, high reasoning use-cases.](https://developers.cloudflare.com/workers-ai/models/gpt-oss-120b/) - Cloudflare-hosted - +4 Compare [![OpenAI logo](https://developers.cloudflare.com/_astro/openai.BBwNKzBb.svg)

gpt-oss-20b

OpenAIText Generation OpenAI’s open-weight models designed for powerful reasoning, agentic tasks, and versatile developer use cases – gpt-oss-20b is for lower latency, and local or specialized use-cases.](https://developers.cloudflare.com/workers-ai/models/gpt-oss-20b/) - Cloudflare-hosted - +4 Compare [![IBM logo](https://developers.cloudflare.com/_astro/ibm.CYdLt4EI.svg)

granite-4.0-h-micro

IBMText Generation Granite 4.0 instruct models deliver strong performance across benchmarks, achieving industry-leading results in key agentic tasks like instruction following and function calling. These efficiencies make the models well-suited for a wide range of use cases like retrieval-augmented generation (RAG), multi-agent workflows, and edge deployments.](https://developers.cloudflare.com/workers-ai/models/granite-4.0-h-micro/) - Cloudflare-hosted - +2 Compare [a

indictrans2-en-indic-1B

ai4bharatTranslation IndicTrans2 is the first open-source transformer-based multilingual NMT model that supports high-quality translations across all the 22 scheduled Indic languages](https://developers.cloudflare.com/workers-ai/models/indictrans2-en-indic-1B/) - Cloudflare-hosted - +1 Compare [![Moonshot AI logo](https://developers.cloudflare.com/_astro/moonshotai.DjWMkXUS.svg)

kimi-k2.6

Moonshot AIText Generation Kimi K2.6 is a frontier-scale open-source 1T parameter model with a 262.1k context window, multi-turn tool calling, vision inputs, and structured outputs for agentic workloads.](https://developers.cloudflare.com/workers-ai/models/kimi-k2.6/) - Cloudflare-hosted - +6 Compare [![Moonshot AI logo](https://developers.cloudflare.com/_astro/moonshotai.DjWMkXUS.svg)

kimi-k2.7-code

Moonshot AIText Generation Kimi K2.7 is a frontier-scale open-source 1T parameter model with a 262.1k context window, multi-turn tool calling, vision inputs, and structured outputs for agentic workloads.](https://developers.cloudflare.com/workers-ai/models/kimi-k2.7-code/) - Cloudflare-hosted - +5 Compare [![Meta logo](https://developers.cloudflare.com/_astro/meta.CTzB_ysm.svg)

llama-2-7b-chat-hf-lora

BetaMetaText Generation This is a Llama2 base model that Cloudflare dedicated for inference with LoRA adapters. Llama 2 is a collection of pretrained and fine-tuned generative text models ranging in scale from 7 billion to 70 billion parameters. This is the repository for the 7B fine-tuned model, optimized for dialogue use cases and converted for the Hugging Face Transformers format.](https://developers.cloudflare.com/workers-ai/models/llama-2-7b-chat-hf-lora/) - Cloudflare-hosted - +3 Compare [![Meta logo](https://developers.cloudflare.com/_astro/meta.CTzB_ysm.svg)

llama-3.1-8b-instruct-fp8

MetaText Generation Llama 3.1 8B quantized to FP8 precision](https://developers.cloudflare.com/workers-ai/models/llama-3.1-8b-instruct-fp8/) - Cloudflare-hosted - +2 Compare [![Meta logo](https://developers.cloudflare.com/_astro/meta.CTzB_ysm.svg)

llama-3.2-11b-vision-instruct

MetaText Generation The Llama 3.2-Vision instruction-tuned models are optimized for visual recognition, image reasoning, captioning, and answering general questions about an image.](https://developers.cloudflare.com/workers-ai/models/llama-3.2-11b-vision-instruct/) - Cloudflare-hosted - +3 Compare [![Meta logo](https://developers.cloudflare.com/_astro/meta.CTzB_ysm.svg)

llama-3.2-1b-instruct

MetaText Generation The Llama 3.2 instruction-tuned text only models are optimized for multilingual dialogue use cases, including agentic retrieval and summarization tasks.](https://developers.cloudflare.com/workers-ai/models/llama-3.2-1b-instruct/) - Cloudflare-hosted - +2 Compare [![Meta logo](https://developers.cloudflare.com/_astro/meta.CTzB_ysm.svg)

llama-3.2-3b-instruct

MetaText Generation The Llama 3.2 instruction-tuned text only models are optimized for multilingual dialogue use cases, including agentic retrieval and summarization tasks.](https://developers.cloudflare.com/workers-ai/models/llama-3.2-3b-instruct/) - Cloudflare-hosted - +3 Compare [![Meta logo](https://developers.cloudflare.com/_astro/meta.CTzB_ysm.svg)

llama-3.3-70b-instruct-fp8-fast

MetaText Generation Llama 3.3 70B quantized to fp8 precision, optimized to be faster.](https://developers.cloudflare.com/workers-ai/models/llama-3.3-70b-instruct-fp8-fast/) - Cloudflare-hosted - +3 Compare [![Meta logo](https://developers.cloudflare.com/_astro/meta.CTzB_ysm.svg)

llama-4-scout-17b-16e-instruct

MetaText Generation Meta's Llama 4 Scout is a 17 billion parameter model with 16 experts that is natively multimodal. These models leverage a mixture-of-experts architecture to offer industry-leading performance in text and image understanding.](https://developers.cloudflare.com/workers-ai/models/llama-4-scout-17b-16e-instruct/) - Cloudflare-hosted - +4 Compare [![Meta logo](https://developers.cloudflare.com/_astro/meta.CTzB_ysm.svg)

llama-guard-3-8b

MetaText Generation Llama Guard 3 is a Llama-3.1-8B pretrained model, fine-tuned for content safety classification. Similar to previous versions, it can be used to classify content in both LLM inputs (prompt classification) and in LLM responses (response classification). It acts as an LLM – it generates text in its output that indicates whether a given prompt or response is safe or unsafe, and if unsafe, it also lists the content categories violated.](https://developers.cloudflare.com/workers-ai/models/llama-guard-3-8b/) - Cloudflare-hosted - +2 Compare [l

llava-1.5-7b-hf

Betallava-hfImage-to-Text LLaVA is an open-source chatbot trained by fine-tuning LLaMA/Vicuna on GPT-generated multimodal instruction-following data. It is an auto-regressive language model, based on the transformer architecture.](https://developers.cloudflare.com/workers-ai/models/llava-1.5-7b-hf/) - Cloudflare-hosted - +2 Compare [![Leonardo logo](https://developers.cloudflare.com/_astro/leonardo.JZysY-g3.svg)

lucid-origin

LeonardoText-to-Image Lucid Origin from Leonardo.AI is their most adaptable and prompt-responsive model to date. Whether you're generating images with sharp graphic design, stunning full-HD renders, or highly specific creative direction, it adheres closely to your prompts, renders text with accuracy, and supports a wide array of visual styles and aesthetics – from stylized concept art to crisp product mockups.](https://developers.cloudflare.com/workers-ai/models/lucid-origin/) - Cloudflare-hosted - +2 Compare [![Meta logo](https://developers.cloudflare.com/_astro/meta.CTzB_ysm.svg)

m2m100-1.2b

MetaTranslation Multilingual encoder-decoder (seq-to-seq) model trained for Many-to-Many multilingual translation](https://developers.cloudflare.com/workers-ai/models/m2m100-1.2b/) - Cloudflare-hosted - +2 Compare [![MyShell logo](https://developers.cloudflare.com/_astro/myshell.6ROagMV2.svg)

melotts

MyShellText-to-Speech MeloTTS is a high-quality multi-lingual text-to-speech library by MyShell.ai.](https://developers.cloudflare.com/workers-ai/models/melotts/) - Cloudflare-hosted - +2 Compare [![MistralAI logo](https://developers.cloudflare.com/_astro/mistralai.Bn9UMUMu.svg)

mistral-7b-instruct-v0.2-lora

MistralAIText Generation The Mistral-7B-Instruct-v0.2 Large Language Model (LLM) is an instruct fine-tuned version of the Mistral-7B-v0.2.](https://developers.cloudflare.com/workers-ai/models/mistral-7b-instruct-v0.2-lora/) - Cloudflare-hosted - +2 Compare [![MistralAI logo](https://developers.cloudflare.com/_astro/mistralai.Bn9UMUMu.svg)

mistral-small-3.1-24b-instruct

MistralAIText Generation Building upon Mistral Small 3 (2501), Mistral Small 3.1 (2503) adds state-of-the-art vision understanding and enhances long context capabilities up to 128k tokens without compromising text performance. With 24 billion parameters, this model achieves top-tier capabilities in both text and vision tasks.](https://developers.cloudflare.com/workers-ai/models/mistral-small-3.1-24b-instruct/) - Cloudflare-hosted - +2 Compare [m

moondream3.1-9B-A2B

moondreamImage-to-Text Moondream 3 is a fast, efficient 9B mixture-of-experts vision language model (2B active parameters) that delivers frontier-level visual reasoning for tasks like object detection, pointing, OCR, and structured output.](https://developers.cloudflare.com/workers-ai/models/moondream3.1-9B-A2B/) - Cloudflare-hosted - +2 Compare [![NVIDIA logo](https://developers.cloudflare.com/_astro/nvidia.DI1bb8hH.svg)

nemotron-3-120b-a12b

NVIDIAText Generation NVIDIA Nemotron 3 Super is a hybrid MoE model with leading accuracy for multi-agent applications and specialized agentic AI systems.](https://developers.cloudflare.com/workers-ai/models/nemotron-3-120b-a12b/) - Cloudflare-hosted - +4 Compare [![Deepgram logo](https://developers.cloudflare.com/_astro/deepgram.BYzW8KfF.svg)

nova-3

DeepgramAutomatic Speech Recognition Transcribe audio using Deepgram’s speech-to-text model](https://developers.cloudflare.com/workers-ai/models/nova-3/) - Cloudflare-hosted - +4 Compare [![Leonardo logo](https://developers.cloudflare.com/_astro/leonardo.JZysY-g3.svg)

phoenix-1.0

LeonardoText-to-Image Phoenix 1.0 is a model by Leonardo.Ai that generates images with exceptional prompt adherence and coherent text.](https://developers.cloudflare.com/workers-ai/models/phoenix-1.0/) - Cloudflare-hosted - +2 Compare [d

phonellm-mini-alpha-2

daily-coText Generation PhoneLLM is an open model tuned for voice agents. The model is based on Nemotron Nano 30B base capabilities and excels at typical voice agent tasks with thinking disabled.](https://developers.cloudflare.com/workers-ai/models/phonellm-mini-alpha-2/) - Cloudflare-hosted - +2 Compare [p

plamo-embedding-1b

pfnetText Embeddings PLaMo-Embedding-1B is a Japanese text embedding model developed by Preferred Networks, Inc. It can convert Japanese text input into numerical vectors and can be used for a wide range of applications, including information retrieval, text classification, and clustering.](https://developers.cloudflare.com/workers-ai/models/plamo-embedding-1b/) - Cloudflare-hosted - +1 Compare [![Qwen logo](https://developers.cloudflare.com/_astro/qwen.ByCZjtXU.svg)

qwen2.5-coder-32b-instruct

QwenText Generation Qwen2.5-Coder is the latest series of Code-Specific Qwen large language models (formerly known as CodeQwen). As of now, Qwen2.5-Coder has covered six mainstream model sizes, 0.5, 1.5, 3, 7, 14, 32 billion parameters, to meet the needs of different developers. Qwen2.5-Coder brings the following improvements upon CodeQwen1.5:](https://developers.cloudflare.com/workers-ai/models/qwen2.5-coder-32b-instruct/) - Cloudflare-hosted - +2 Compare [![Qwen logo](https://developers.cloudflare.com/_astro/qwen.ByCZjtXU.svg)

qwen3-30b-a3b-fp8

QwenText Generation Qwen3 is the latest generation of large language models in Qwen series, offering a comprehensive suite of dense and mixture-of-experts (MoE) models. Built upon extensive training, Qwen3 delivers groundbreaking advancements in reasoning, instruction-following, agent capabilities, and multilingual support.](https://developers.cloudflare.com/workers-ai/models/qwen3-30b-a3b-fp8/) - Cloudflare-hosted - +5 Compare [![Qwen logo](https://developers.cloudflare.com/_astro/qwen.ByCZjtXU.svg)

qwen3-embedding-0.6b

QwenText Embeddings The Qwen3 Embedding model series is the latest proprietary model of the Qwen family, specifically designed for text embedding and ranking tasks.](https://developers.cloudflare.com/workers-ai/models/qwen3-embedding-0.6b/) - Cloudflare-hosted - +1 Compare [![Qwen logo](https://developers.cloudflare.com/_astro/qwen.ByCZjtXU.svg)

qwen3.8-27b

QwenText Generation Qwen 3.8 27B is a 27-billion-parameter instruction-tuned language model from Alibaba's Qwen family, designed for vision, efficient general-purpose text generation and agentic workloads.](https://developers.cloudflare.com/workers-ai/models/qwen3.8-27b/) - Cloudflare-hosted - +5 Compare [![Qwen logo](https://developers.cloudflare.com/_astro/qwen.ByCZjtXU.svg)

qwq-32b

QwenText Generation QwQ is the reasoning model of the Qwen series. Compared with conventional instruction-tuned models, QwQ, which is capable of thinking and reasoning, can achieve significantly enhanced performance in downstream tasks, especially hard problems. QwQ-32B is the medium-sized reasoning model, which is capable of achieving competitive performance against state-of-the-art reasoning models, e.g., DeepSeek-R1, o1-mini.](https://developers.cloudflare.com/workers-ai/models/qwq-32b/) - Cloudflare-hosted - +4 Compare [![Microsoft logo](https://developers.cloudflare.com/_astro/microsoft.LujcDJ--.svg)

resnet-50

MicrosoftImage Classification 50 layers deep image classification CNN trained on more than 1M images from ImageNet](https://developers.cloudflare.com/workers-ai/models/resnet-50/) - Cloudflare-hosted - +1 Compare [![Pipecat logo](https://developers.cloudflare.com/_astro/pipecat.B-PNBdef.svg)

smart-turn-v2

PipecatDumb Pipe An open source, community-driven, native audio turn detection model in 2nd version](https://developers.cloudflare.com/workers-ai/models/smart-turn-v2/) - Cloudflare-hosted - +3 Compare [![RunwayML logo](https://developers.cloudflare.com/_astro/runway.Cq8Cjov4.svg)

stable-diffusion-v1-5-inpainting

BetaRunwayMLText-to-Image Stable Diffusion Inpainting is a latent text-to-image diffusion model capable of generating photo-realistic images given any text input, with the extra capability of inpainting the pictures by using a mask.](https://developers.cloudflare.com/workers-ai/models/stable-diffusion-v1-5-inpainting/) - Cloudflare-hosted - +2 Compare [![Stability.ai logo](https://developers.cloudflare.com/_astro/stabilityai.VsBx3CKv.svg)

stable-diffusion-xl-base-1.0

BetaStability.aiText-to-Image Diffusion-based text-to-image generative model by Stability AI. Generates and modify images based on text prompts.](https://developers.cloudflare.com/workers-ai/models/stable-diffusion-xl-base-1.0/) - Cloudflare-hosted - +2 Compare [![ByteDance logo](https://developers.cloudflare.com/_astro/bytedance.T1uiROQ6.svg)

stable-diffusion-xl-lightning

BetaByteDanceText-to-Image SDXL-Lightning is a lightning-fast text-to-image generation model. It can generate high-quality 1024px images in a few steps.](https://developers.cloudflare.com/workers-ai/models/stable-diffusion-xl-lightning/) - Cloudflare-hosted - +2 Compare [![OpenAI logo](https://developers.cloudflare.com/_astro/openai.BBwNKzBb.svg)

whisper

OpenAIAutomatic Speech Recognition Whisper is a general-purpose speech recognition model. It is trained on a large dataset of diverse audio and is also a multitasking model that can perform multilingual speech recognition, speech translation, and language identification.](https://developers.cloudflare.com/workers-ai/models/whisper/) - Cloudflare-hosted - +1 Compare [![OpenAI logo](https://developers.cloudflare.com/_astro/openai.BBwNKzBb.svg)

whisper-large-v3-turbo

OpenAIAutomatic Speech Recognition Whisper is a pre-trained model for automatic speech recognition (ASR) and speech translation.](https://developers.cloudflare.com/workers-ai/models/whisper-large-v3-turbo/) - Cloudflare-hosted - +2 Compare [![OpenAI logo](https://developers.cloudflare.com/_astro/openai.BBwNKzBb.svg)

whisper-tiny-en

BetaOpenAIAutomatic Speech Recognition Whisper is a pre-trained model for automatic speech recognition (ASR) and speech translation. Trained on 680k hours of labelled data, Whisper models demonstrate a strong ability to generalize to many datasets and domains without the need for fine-tuning. This is the English-only version of the Whisper Tiny model which was trained on the task of speech recognition.](https://developers.cloudflare.com/workers-ai/models/whisper-tiny-en/) - Cloudflare-hosted - +2 Compare Compare models Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"WebPage","@id":"https://developers.cloudflare.com/workers-ai/models/#page","headline":"Workers AI Models","description":"Browse the catalog of machine learning models available on Workers AI.","url":"https://developers.cloudflare.com/workers-ai/models/","inLanguage":"en","image":"https://developers.cloudflare.com/workers-ai/models/og.png?v=40351c4db9a31785","dateModified":"2026-08-12","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: IndicTrans2 is the first open-source transformer-based multilingual NMT model that supports high-quality translations across all the 22 scheduled Indic languages title: indictrans2-en-indic-1B image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/llms.txt > Use this file to discover all available pages before exploring further. a # indictrans2-en-indic-1B Translation • ai4bharat Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/models/indictrans2-en-indic-1B/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Model ID `@cf/ai4bharat/indictrans2-en-indic-1B` Copy - Cloudflare-hosted - Zero data retention IndicTrans2 is the first open-source transformer-based multilingual NMT model that supports high-quality translations across all the 22 scheduled Indic languages | Model Info | | | --- | --- | | Zero data retention | Yes | | Unit Pricing | $0.342 per M input tokens, $0.342 per M output tokens | ## Usage ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const response = await env.AI.run( "@cf/ai4bharat/indictrans2-en-indic-1B", { text: "Hello, how are you?", target_language: "hin_Deva", } ); return new Response(JSON.stringify(response)); }, } satisfies ExportedHandler; ``` ```py import requests API_BASE_URL = "https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai/run/" headers = {"Authorization": "Bearer {API_TOKEN}"} def run(model, input): response = requests.post(f"{API_BASE_URL}{model}", headers=headers, json=input) return response.json() output = run('@cf/ai4bharat/indictrans2-en-indic-1B', { "text": "Hello, how are you?", "target_language": "hin_Deva" }) print(output) ``` ```sh curl https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/run/@cf/ai4bharat/indictrans2-en-indic-1B \ -X POST \ -H "Authorization: Bearer $CLOUDFLARE_API_TOKEN" \ -d '{ "text": "Hello, how are you?", "target_language": "hin_Deva" }' ``` ## Parameters ▶text `one of`required target\_language `string`requireddefault: hin\_Devaenum: asm\_Beng, awa\_Deva, ben\_Beng, bho\_Deva, brx\_Deva, doi\_Deva, eng\_Latn, gom\_Deva, gon\_Deva, guj\_Gujr, hin\_Deva, hne\_Deva, kan\_Knda, kas\_Arab, kas\_Deva, kha\_Latn, lus\_Latn, mag\_Deva, mai\_Deva, mal\_Mlym, mar\_Deva, mni\_Beng, mni\_Mtei, npi\_Deva, ory\_Orya, pan\_Guru, san\_Deva, sat\_Olck, snd\_Arab, snd\_Deva, tam\_Taml, tel\_Telu, urd\_Arab, unr\_DevaTarget langauge to translate to ▶translations\[] `array`Translated texts ## API Schemas (Raw) Input [Open](https://developers.cloudflare.com/workers-ai/models/indictrans2-en-indic-1B/schema-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/indictrans2-en-indic-1B/schema-input.json) Output [Open](https://developers.cloudflare.com/workers-ai/models/indictrans2-en-indic-1B/schema-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/indictrans2-en-indic-1B/schema-output.json) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/models/indictrans2-en-indic-1B/#page","headline":"indictrans2-en-indic-1B","description":"IndicTrans2 is the first open-source transformer-based multilingual NMT model that supports high-quality translations across all the 22 scheduled Indic languages","url":"https://developers.cloudflare.com/workers-ai/models/indictrans2-en-indic-1B/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: SEA-LION stands for Southeast Asian Languages In One Network, which is a collection of Large Language Models (LLMs) which have been pretrained and instruct-tuned for the Southeast Asia (SEA) region. title: gemma-sea-lion-v4-27b-it image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/llms.txt > Use this file to discover all available pages before exploring further. a # gemma-sea-lion-v4-27b-it Text Generation • aisingapore Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/models/gemma-sea-lion-v4-27b-it/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Model ID `@cf/aisingapore/gemma-sea-lion-v4-27b-it` Copy - Cloudflare-hosted - Zero data retention SEA-LION stands for Southeast Asian Languages In One Network, which is a collection of Large Language Models (LLMs) which have been pretrained and instruct-tuned for the Southeast Asia (SEA) region. | Model Info | | | --- | --- | | Context Window [ ↗](https://developers.cloudflare.com/workers-ai/platform/glossary/) | 128,000 tokens | | Zero data retention | Yes | | Unit Pricing | $0.351 per M input tokens, $0.555 per M output tokens | ## Playground Try out this model with Workers AI LLM Playground. It does not require any setup or authentication and is an instant way to preview and test a model directly in the browser. [Launch the LLM Playground](https://playground.ai.cloudflare.com/?model=@cf/aisingapore/gemma-sea-lion-v4-27b-it) ## Usage ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const messages = [ { role: "system", content: "You are a friendly assistant" }, { role: "user", content: "What is the origin of the phrase Hello, World", }, ]; const stream = await env.AI.run("@cf/aisingapore/gemma-sea-lion-v4-27b-it", { messages, stream: true, }); return new Response(stream, { headers: { "content-type": "text/event-stream" }, }); }, } satisfies ExportedHandler; ``` ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const messages = [ { role: "system", content: "You are a friendly assistant" }, { role: "user", content: "What is the origin of the phrase Hello, World", }, ]; const response = await env.AI.run("@cf/aisingapore/gemma-sea-lion-v4-27b-it", { messages }); return Response.json(response); }, } satisfies ExportedHandler; ``` ```py import os import requests ACCOUNT_ID = "your-account-id" AUTH_TOKEN = os.environ.get("CLOUDFLARE_AUTH_TOKEN") prompt = "Tell me all about PEP-8" response = requests.post( f"https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai/run/@cf/aisingapore/gemma-sea-lion-v4-27b-it", headers={"Authorization": f"Bearer {AUTH_TOKEN}"}, json={ "messages": [ {"role": "system", "content": "You are a friendly assistant"}, {"role": "user", "content": prompt} ] } ) result = response.json() print(result) ``` ```sh curl https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/run/@cf/aisingapore/gemma-sea-lion-v4-27b-it \ -X POST \ -H "Authorization: Bearer $CLOUDFLARE_AUTH_TOKEN" \ -d '{ "messages": [{ "role": "system", "content": "You are a friendly assistant" }, { "role": "user", "content": "Why is pizza so good" }]}' ``` OpenAI compatible endpoints Workers AI also supports OpenAI compatible API endpoints for `/v1/chat/completions` and `/v1/embeddings`. For more details, refer to [Configurations](https://developers.cloudflare.com/workers-ai/configuration/open-ai-compatibility/) . ## Parameters
Synchronous — Send a request and receive a complete response prompt stringrequiredminLength: 1The input text prompt for the model to generate a response. lora stringName of the LoRA (Low-Rank Adaptation) model to fine-tune the base model. ▶response\_format{} object raw booleandefault: falseIf true, a chat template is not applied and you must adhere to the specific model's expected formatting. stream booleandefault: falseIf true, the response will be streamed back incrementally using SSE, Server Sent Events. max\_tokens integerdefault: 2000The maximum number of tokens to generate in the response. temperature numberdefault: 0.6minimum: 0maximum: 5Controls the randomness of the output; higher values produce more random results. top\_p numberminimum: 0.001maximum: 1Adjusts the creativity of the AI's responses by controlling how many possible words it considers. Lower values make outputs more predictable; higher values allow for more varied and creative responses. top\_k integerminimum: 1maximum: 50Limits the AI to choose from the top 'k' most probable words. Lower values make responses more focused; higher values introduce more variety and potential surprises. seed integerminimum: 1maximum: 9999999999Random seed for reproducibility of the generation. repetition\_penalty numberminimum: 0maximum: 2Penalty for repeated tokens; higher values discourage repetition. frequency\_penalty numberminimum: -2maximum: 2Decreases the likelihood of the model repeating the same lines verbatim. presence\_penalty numberminimum: -2maximum: 2Increases the likelihood of the model introducing new topics. id stringUnique identifier for the completion object stringenum: chat.completionObject type identifier created numberUnix timestamp of when the completion was created model stringModel used for the completion ▶choices\[] arrayList of completion choices ▶usage{} objectUsage statistics for the inference request prompt\_logprobs{} objectLog probabilities for the prompt (if requested)
Streaming — Send a request with `stream: true` and receive server-sent events prompt stringrequiredminLength: 1The input text prompt for the model to generate a response. lora stringName of the LoRA (Low-Rank Adaptation) model to fine-tune the base model. ▶response\_format{} object raw booleandefault: falseIf true, a chat template is not applied and you must adhere to the specific model's expected formatting. stream booleandefault: falseIf true, the response will be streamed back incrementally using SSE, Server Sent Events. max\_tokens integerdefault: 2000The maximum number of tokens to generate in the response. temperature numberdefault: 0.6minimum: 0maximum: 5Controls the randomness of the output; higher values produce more random results. top\_p numberminimum: 0.001maximum: 1Adjusts the creativity of the AI's responses by controlling how many possible words it considers. Lower values make outputs more predictable; higher values allow for more varied and creative responses. top\_k integerminimum: 1maximum: 50Limits the AI to choose from the top 'k' most probable words. Lower values make responses more focused; higher values introduce more variety and potential surprises. seed integerminimum: 1maximum: 9999999999Random seed for reproducibility of the generation. repetition\_penalty numberminimum: 0maximum: 2Penalty for repeated tokens; higher values discourage repetition. frequency\_penalty numberminimum: -2maximum: 2Decreases the likelihood of the model repeating the same lines verbatim. presence\_penalty numberminimum: -2maximum: 2Increases the likelihood of the model introducing new topics. type string contentType text/event-stream format binary
Batch — Send multiple requests in a single API call ▶requests\[] arrayrequired id stringUnique identifier for the completion object stringenum: chat.completionObject type identifier created numberUnix timestamp of when the completion was created model stringModel used for the completion ▶choices\[] arrayList of completion choices ▶usage{} objectUsage statistics for the inference request prompt\_logprobs{} objectLog probabilities for the prompt (if requested)
## API Schemas (Raw) SynchronousInput [Open](https://developers.cloudflare.com/workers-ai/models/gemma-sea-lion-v4-27b-it/sync-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/gemma-sea-lion-v4-27b-it/sync-input.json) SynchronousOutput [Open](https://developers.cloudflare.com/workers-ai/models/gemma-sea-lion-v4-27b-it/sync-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/gemma-sea-lion-v4-27b-it/sync-output.json) StreamingInput [Open](https://developers.cloudflare.com/workers-ai/models/gemma-sea-lion-v4-27b-it/streaming-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/gemma-sea-lion-v4-27b-it/streaming-input.json) StreamingOutput [Open](https://developers.cloudflare.com/workers-ai/models/gemma-sea-lion-v4-27b-it/streaming-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/gemma-sea-lion-v4-27b-it/streaming-output.json) BatchInput [Open](https://developers.cloudflare.com/workers-ai/models/gemma-sea-lion-v4-27b-it/batch-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/gemma-sea-lion-v4-27b-it/batch-input.json) BatchOutput [Open](https://developers.cloudflare.com/workers-ai/models/gemma-sea-lion-v4-27b-it/batch-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/gemma-sea-lion-v4-27b-it/batch-output.json) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/models/gemma-sea-lion-v4-27b-it/#page","headline":"gemma-sea-lion-v4-27b-it","description":"SEA-LION stands for Southeast Asian Languages In One Network, which is a collection of Large Language Models (LLMs) which have been pretrained and instruct-tuned for the Southeast Asia (SEA) region.","url":"https://developers.cloudflare.com/workers-ai/models/gemma-sea-lion-v4-27b-it/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: BAAI general embedding (Base) model that transforms any given text into a 768-dimensional vector title: bge-base-en-v1.5 image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/llms.txt > Use this file to discover all available pages before exploring further. ![BAAI logo](https://developers.cloudflare.com/_astro/baai.BooZR_xF.svg) # bge-base-en-v1.5 Text Embeddings • BAAI Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/models/bge-base-en-v1.5/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Model ID `@cf/baai/bge-base-en-v1.5` Copy - Cloudflare-hosted - Batch - Zero data retention BAAI general embedding (Base) model that transforms any given text into a 768-dimensional vector | Model Info | | | --- | --- | | Context Window [ ↗](https://developers.cloudflare.com/workers-ai/platform/glossary/) | 153,600 tokens | | More information | [link ↗](https://huggingface.co/BAAI/bge-base-en-v1.5) | | Maximum Input Tokens | 512 | | Output Dimensions | 768 | | Zero data retention | Yes | | Batch | Yes | | Unit Pricing | $0.0666 per M input tokens | ## Usage ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { // Can be a string or array of strings] const stories = [ "This is a story about an orange cloud", "This is a story about a llama", "This is a story about a hugging emoji", ]; const embeddings = await env.AI.run( "@cf/baai/bge-base-en-v1.5", { text: stories, } ); return Response.json(embeddings); }, } satisfies ExportedHandler; ``` ```py import os import requests ACCOUNT_ID = "your-account-id" AUTH_TOKEN = os.environ.get("CLOUDFLARE_AUTH_TOKEN") stories = [ 'This is a story about an orange cloud', 'This is a story about a llama', 'This is a story about a hugging emoji' ] response = requests.post( f"https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai/run/@cf/baai/bge-base-en-v1.5", headers={"Authorization": f"Bearer {AUTH_TOKEN}"}, json={"text": stories} ) print(response.json()) ``` ```sh curl https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/run/@cf/baai/bge-base-en-v1.5 \ -X POST \ -H "Authorization: Bearer $CLOUDFLARE_API_TOKEN" \ -d '{ "text": ["This is a story about an orange cloud", "This is a story about a llama", "This is a story about a hugging emoji"] }' ``` OpenAI compatible endpoints Workers AI also supports OpenAI compatible API endpoints for `/v1/chat/completions` and `/v1/embeddings`. For more details, refer to [Configurations](https://developers.cloudflare.com/workers-ai/configuration/open-ai-compatibility/). ## Parameters
Synchronous — Send a request and receive a complete response ▶text one ofrequired pooling stringdefault: meanenum: mean, clsThe pooling method used in the embedding process. `cls` pooling will generate more accurate embeddings on larger inputs - however, embeddings created with cls pooling are not compatible with embeddings generated with mean pooling. The default pooling method is `mean` in order for this to not be a breaking change, but we highly suggest using the new `cls` pooling for better accuracy. ▶shape\[] array ▶data\[] arrayEmbeddings of the requested text values pooling stringenum: mean, clsThe pooling method used in the embedding process.
Batch — Send multiple requests in a single API call ▶requests\[] arrayrequiredBatch of the embeddings requests to run using async-queue ▶shape\[] array ▶data\[] arrayEmbeddings of the requested text values pooling stringenum: mean, clsThe pooling method used in the embedding process.
## API Schemas (Raw) SynchronousInput [Open](https://developers.cloudflare.com/workers-ai/models/bge-base-en-v1.5/sync-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/bge-base-en-v1.5/sync-input.json) SynchronousOutput [Open](https://developers.cloudflare.com/workers-ai/models/bge-base-en-v1.5/sync-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/bge-base-en-v1.5/sync-output.json) BatchInput [Open](https://developers.cloudflare.com/workers-ai/models/bge-base-en-v1.5/batch-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/bge-base-en-v1.5/batch-input.json) BatchOutput [Open](https://developers.cloudflare.com/workers-ai/models/bge-base-en-v1.5/batch-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/bge-base-en-v1.5/batch-output.json) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/models/bge-base-en-v1.5/#page","headline":"bge-base-en-v1.5","description":"BAAI general embedding (Base) model that transforms any given text into a 768-dimensional vector","url":"https://developers.cloudflare.com/workers-ai/models/bge-base-en-v1.5/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: BAAI general embedding (Large) model that transforms any given text into a 1024-dimensional vector title: bge-large-en-v1.5 image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/llms.txt > Use this file to discover all available pages before exploring further. ![BAAI logo](https://developers.cloudflare.com/_astro/baai.BooZR_xF.svg) # bge-large-en-v1.5 Text Embeddings • BAAI Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/models/bge-large-en-v1.5/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Model ID `@cf/baai/bge-large-en-v1.5` Copy - Cloudflare-hosted - Batch - Zero data retention BAAI general embedding (Large) model that transforms any given text into a 1024-dimensional vector | Model Info | | | --- | --- | | More information | [link ↗](https://huggingface.co/BAAI/bge-large-en-v1.5) | | Maximum Input Tokens | 512 | | Output Dimensions | 1,024 | | Zero data retention | Yes | | Batch | Yes | | Unit Pricing | $0.204 per M input tokens | ## Usage ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { // Can be a string or array of strings] const stories = [ "This is a story about an orange cloud", "This is a story about a llama", "This is a story about a hugging emoji", ]; const embeddings = await env.AI.run( "@cf/baai/bge-large-en-v1.5", { text: stories, } ); return Response.json(embeddings); }, } satisfies ExportedHandler; ``` ```py import os import requests ACCOUNT_ID = "your-account-id" AUTH_TOKEN = os.environ.get("CLOUDFLARE_AUTH_TOKEN") stories = [ 'This is a story about an orange cloud', 'This is a story about a llama', 'This is a story about a hugging emoji' ] response = requests.post( f"https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai/run/@cf/baai/bge-large-en-v1.5", headers={"Authorization": f"Bearer {AUTH_TOKEN}"}, json={"text": stories} ) print(response.json()) ``` ```sh curl https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/run/@cf/baai/bge-large-en-v1.5 \ -X POST \ -H "Authorization: Bearer $CLOUDFLARE_API_TOKEN" \ -d '{ "text": ["This is a story about an orange cloud", "This is a story about a llama", "This is a story about a hugging emoji"] }' ``` OpenAI compatible endpoints Workers AI also supports OpenAI compatible API endpoints for `/v1/chat/completions` and `/v1/embeddings`. For more details, refer to [Configurations](https://developers.cloudflare.com/workers-ai/configuration/open-ai-compatibility/). ## Parameters
Synchronous — Send a request and receive a complete response ▶text one ofrequired pooling stringdefault: meanenum: mean, clsThe pooling method used in the embedding process. `cls` pooling will generate more accurate embeddings on larger inputs - however, embeddings created with cls pooling are not compatible with embeddings generated with mean pooling. The default pooling method is `mean` in order for this to not be a breaking change, but we highly suggest using the new `cls` pooling for better accuracy. ▶shape\[] array ▶data\[] arrayEmbeddings of the requested text values pooling stringenum: mean, clsThe pooling method used in the embedding process.
Batch — Send multiple requests in a single API call ▶requests\[] arrayrequiredBatch of the embeddings requests to run using async-queue ▶shape\[] array ▶data\[] arrayEmbeddings of the requested text values pooling stringenum: mean, clsThe pooling method used in the embedding process.
## API Schemas (Raw) SynchronousInput [Open](https://developers.cloudflare.com/workers-ai/models/bge-large-en-v1.5/sync-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/bge-large-en-v1.5/sync-input.json) SynchronousOutput [Open](https://developers.cloudflare.com/workers-ai/models/bge-large-en-v1.5/sync-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/bge-large-en-v1.5/sync-output.json) BatchInput [Open](https://developers.cloudflare.com/workers-ai/models/bge-large-en-v1.5/batch-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/bge-large-en-v1.5/batch-input.json) BatchOutput [Open](https://developers.cloudflare.com/workers-ai/models/bge-large-en-v1.5/batch-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/bge-large-en-v1.5/batch-output.json) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/models/bge-large-en-v1.5/#page","headline":"bge-large-en-v1.5","description":"BAAI general embedding (Large) model that transforms any given text into a 1024-dimensional vector","url":"https://developers.cloudflare.com/workers-ai/models/bge-large-en-v1.5/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Multi-Functionality, Multi-Linguality, and Multi-Granularity embeddings model. title: bge-m3 image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/llms.txt > Use this file to discover all available pages before exploring further. ![BAAI logo](https://developers.cloudflare.com/_astro/baai.BooZR_xF.svg) # bge-m3 Text Embeddings • BAAI Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/models/bge-m3/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Model ID `@cf/baai/bge-m3` Copy - Cloudflare-hosted - Zero data retention Multi-Functionality, Multi-Linguality, and Multi-Granularity embeddings model. | Model Info | | | --- | --- | | Context Window [ ↗](https://developers.cloudflare.com/workers-ai/platform/glossary/) | 60,000 tokens | | Zero data retention | Yes | | Unit Pricing | $0.0118 per M input tokens | ## Usage ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { // Can be a string or array of strings] const stories = [ "This is a story about an orange cloud", "This is a story about a llama", "This is a story about a hugging emoji", ]; const embeddings = await env.AI.run( "@cf/baai/bge-m3", { text: stories, } ); return Response.json(embeddings); }, } satisfies ExportedHandler; ``` ```py import os import requests ACCOUNT_ID = "your-account-id" AUTH_TOKEN = os.environ.get("CLOUDFLARE_AUTH_TOKEN") stories = [ 'This is a story about an orange cloud', 'This is a story about a llama', 'This is a story about a hugging emoji' ] response = requests.post( f"https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai/run/@cf/baai/bge-m3", headers={"Authorization": f"Bearer {AUTH_TOKEN}"}, json={"text": stories} ) print(response.json()) ``` ```sh curl https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/run/@cf/baai/bge-m3 \ -X POST \ -H "Authorization: Bearer $CLOUDFLARE_API_TOKEN" \ -d '{ "text": ["This is a story about an orange cloud", "This is a story about a llama", "This is a story about a hugging emoji"] }' ``` OpenAI compatible endpoints Workers AI also supports OpenAI compatible API endpoints for `/v1/chat/completions` and `/v1/embeddings`. For more details, refer to [Configurations](https://developers.cloudflare.com/workers-ai/configuration/open-ai-compatibility/). ## Parameters
Synchronous — Send a request and receive a complete response query stringminLength: 1A query you wish to perform against the provided contexts. If no query is provided the model with respond with embeddings for contexts ▶contexts\[] arrayrequiredList of provided contexts. Note that the index in this array is important, as the response will refer to it. truncate\_inputs booleandefault: falseWhen provided with too long context should the model error out or truncate the context to fit? request\_id stringThe async request id that can be used to obtain the results.
Batch — Send multiple requests in a single API call ▶requests\[] arrayrequiredBatch of the embeddings requests to run using async-queue request\_id stringThe async request id that can be used to obtain the results.
## API Schemas (Raw) SynchronousInput [Open](https://developers.cloudflare.com/workers-ai/models/bge-m3/sync-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/bge-m3/sync-input.json) SynchronousOutput [Open](https://developers.cloudflare.com/workers-ai/models/bge-m3/sync-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/bge-m3/sync-output.json) BatchInput [Open](https://developers.cloudflare.com/workers-ai/models/bge-m3/batch-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/bge-m3/batch-input.json) BatchOutput [Open](https://developers.cloudflare.com/workers-ai/models/bge-m3/batch-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/bge-m3/batch-output.json) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/models/bge-m3/#page","headline":"bge-m3","description":"Multi-Functionality, Multi-Linguality, and Multi-Granularity embeddings model.","url":"https://developers.cloudflare.com/workers-ai/models/bge-m3/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: "Different from embedding model, reranker uses question and document as input and directly output similarity instead of embedding. You can get a relevance score by inputting query and passage to the reranker. And the score can be mapped to a float value in [0,1] by sigmoid function.\n\n" title: bge-reranker-base image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/llms.txt > Use this file to discover all available pages before exploring further. ![BAAI logo](https://developers.cloudflare.com/_astro/baai.BooZR_xF.svg) # bge-reranker-base Text Classification • BAAI Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/models/bge-reranker-base/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Model ID `@cf/baai/bge-reranker-base` Copy - Cloudflare-hosted - Zero data retention Different from embedding model, reranker uses question and document as input and directly output similarity instead of embedding. You can get a relevance score by inputting query and passage to the reranker. And the score can be mapped to a float value in \[0,1] by sigmoid function. | Model Info | | | --- | --- | | Zero data retention | Yes | | Unit Pricing | $0.00311 per M input tokens | ## Usage ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const query = 'Which one is cooler?' const contexts = [ { text: 'a cyberpunk lizzard' }, { text: 'a cyberpunk cat' } ]; const response = await env.AI.run('@cf/baai/bge-reranker-base', { query, contexts }); return Response.json(response); }, } satisfies ExportedHandler; ``` ```py import os import requests ACCOUNT_ID = "your-account-id" AUTH_TOKEN = os.environ.get("CLOUDFLARE_AUTH_TOKEN") response = requests.post( f"https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai/run/@cf/baai/bge-reranker-base", headers={"Authorization": f"Bearer {AUTH_TOKEN}"}, json={ "query": "Which one is better?", "contexts": [ {"text": "a cyberpunk lizzard"}, {"text": "a cyberpunk car"}, ] } ) result = response.json() print(result) ``` ```sh curl https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/run/@cf/baai/bge-reranker-base \ -X POST \ -H "Authorization: Bearer $CLOUDFLARE_AUTH_TOKEN" \ -d '{ "query": "Which one is better?", "contexts": [{ "text": "a cyberpunk lizzard" }, {"text": "a cyberpunk cat"}]}' ``` ## Parameters query `string`requiredminLength: 1A query you wish to perform against the provided contexts. top\_k `integer`minimum: 1Number of returned results starting with the best score. ▶contexts\[] `array`requiredList of provided contexts. Note that the index in this array is important, as the response will refer to it. ▶response\[] `array` ## API Schemas (Raw) Input [Open](https://developers.cloudflare.com/workers-ai/models/bge-reranker-base/schema-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/bge-reranker-base/schema-input.json) Output [Open](https://developers.cloudflare.com/workers-ai/models/bge-reranker-base/schema-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/bge-reranker-base/schema-output.json) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/models/bge-reranker-base/#page","headline":"bge-reranker-base","description":"Different from embedding model, reranker uses question and document as input and directly output similarity instead of embedding. You can get a relevance score by inputting query and passage to the reranker. And the score can be mapped to a float value in [0,1] by sigmoid function.","url":"https://developers.cloudflare.com/workers-ai/models/bge-reranker-base/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: BAAI general embedding (Small) model that transforms any given text into a 384-dimensional vector title: bge-small-en-v1.5 image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/llms.txt > Use this file to discover all available pages before exploring further. ![BAAI logo](https://developers.cloudflare.com/_astro/baai.BooZR_xF.svg) # bge-small-en-v1.5 Text Embeddings • BAAI Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/models/bge-small-en-v1.5/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Model ID `@cf/baai/bge-small-en-v1.5` Copy - Cloudflare-hosted - Batch - Zero data retention BAAI general embedding (Small) model that transforms any given text into a 384-dimensional vector | Model Info | | | --- | --- | | More information | [link ↗](https://huggingface.co/BAAI/bge-small-en-v1.5) | | Maximum Input Tokens | 512 | | Output Dimensions | 384 | | Zero data retention | Yes | | Batch | Yes | | Unit Pricing | $0.0202 per M input tokens | ## Usage ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { // Can be a string or array of strings] const stories = [ "This is a story about an orange cloud", "This is a story about a llama", "This is a story about a hugging emoji", ]; const embeddings = await env.AI.run( "@cf/baai/bge-small-en-v1.5", { text: stories, } ); return Response.json(embeddings); }, } satisfies ExportedHandler; ``` ```py import os import requests ACCOUNT_ID = "your-account-id" AUTH_TOKEN = os.environ.get("CLOUDFLARE_AUTH_TOKEN") stories = [ 'This is a story about an orange cloud', 'This is a story about a llama', 'This is a story about a hugging emoji' ] response = requests.post( f"https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai/run/@cf/baai/bge-small-en-v1.5", headers={"Authorization": f"Bearer {AUTH_TOKEN}"}, json={"text": stories} ) print(response.json()) ``` ```sh curl https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/run/@cf/baai/bge-small-en-v1.5 \ -X POST \ -H "Authorization: Bearer $CLOUDFLARE_API_TOKEN" \ -d '{ "text": ["This is a story about an orange cloud", "This is a story about a llama", "This is a story about a hugging emoji"] }' ``` OpenAI compatible endpoints Workers AI also supports OpenAI compatible API endpoints for `/v1/chat/completions` and `/v1/embeddings`. For more details, refer to [Configurations](https://developers.cloudflare.com/workers-ai/configuration/open-ai-compatibility/). ## Parameters
Synchronous — Send a request and receive a complete response ▶text one ofrequired pooling stringdefault: meanenum: mean, clsThe pooling method used in the embedding process. `cls` pooling will generate more accurate embeddings on larger inputs - however, embeddings created with cls pooling are not compatible with embeddings generated with mean pooling. The default pooling method is `mean` in order for this to not be a breaking change, but we highly suggest using the new `cls` pooling for better accuracy. ▶shape\[] array ▶data\[] arrayEmbeddings of the requested text values pooling stringenum: mean, clsThe pooling method used in the embedding process.
Batch — Send multiple requests in a single API call ▶requests\[] arrayrequiredBatch of the embeddings requests to run using async-queue ▶shape\[] array ▶data\[] arrayEmbeddings of the requested text values pooling stringenum: mean, clsThe pooling method used in the embedding process.
## API Schemas (Raw) SynchronousInput [Open](https://developers.cloudflare.com/workers-ai/models/bge-small-en-v1.5/sync-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/bge-small-en-v1.5/sync-input.json) SynchronousOutput [Open](https://developers.cloudflare.com/workers-ai/models/bge-small-en-v1.5/sync-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/bge-small-en-v1.5/sync-output.json) BatchInput [Open](https://developers.cloudflare.com/workers-ai/models/bge-small-en-v1.5/batch-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/bge-small-en-v1.5/batch-input.json) BatchOutput [Open](https://developers.cloudflare.com/workers-ai/models/bge-small-en-v1.5/batch-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/bge-small-en-v1.5/batch-output.json) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/models/bge-small-en-v1.5/#page","headline":"bge-small-en-v1.5","description":"BAAI general embedding (Small) model that transforms any given text into a 384-dimensional vector","url":"https://developers.cloudflare.com/workers-ai/models/bge-small-en-v1.5/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: "FLUX.1 [schnell] is a 12 billion parameter rectified flow transformer capable of generating images from text descriptions. " title: flux-1-schnell image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/llms.txt > Use this file to discover all available pages before exploring further. ![Black Forest Labs logo](https://developers.cloudflare.com/_astro/blackforestlabs.Ccs-Y4-D.svg) # flux-1-schnell Text-to-Image • Black Forest Labs Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/models/flux-1-schnell/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Model ID `@cf/black-forest-labs/flux-1-schnell` Copy - Cloudflare-hosted - Zero data retention FLUX.1 \[schnell] is a 12 billion parameter rectified flow transformer capable of generating images from text descriptions. | Model Info | | | --- | --- | | Terms and License | [link ↗](https://bfl.ai/legal/terms-of-service) | | Zero data retention | Yes | ## Usage ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const response = await env.AI.run('@cf/black-forest-labs/flux-1-schnell', { prompt: 'a cyberpunk lizard', seed: Math.floor(Math.random() * 10) }); // response.image is base64 encoded which can be used directly as an data URI const dataURI = `data:image/jpeg;charset=utf-8;base64,${response.image}`; return Response.json({ dataURI }); }, } satisfies ExportedHandler; ``` ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const response = await env.AI.run('@cf/black-forest-labs/flux-1-schnell', { prompt: 'a cyberpunk lizard', seed: Math.floor(Math.random() * 10) }); // Convert from base64 string const binaryString = atob(response.image); // Create byte representation const img = Uint8Array.from(binaryString, (m) => m.codePointAt(0)); return new Response(img, { headers: { 'Content-Type': 'image/jpeg', }, }); }, } satisfies ExportedHandler; ``` ```sh curl https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/run/@cf/black-forest-labs/flux-1-schnell \ -X POST \ -H "Authorization: Bearer $CLOUDFLARE_API_TOKEN" \ -d '{ "prompt": "cyberpunk cat", "seed": "Random positive integer" }' ``` ## Parameters prompt `string`requiredminLength: 1maxLength: 2048A text description of the image you want to generate. steps `integer`maximum: 8The number of diffusion steps; higher values can improve quality but take longer. Default is 4 image `string`The generated image in Base64 format. ## API Schemas (Raw) Input [Open](https://developers.cloudflare.com/workers-ai/models/flux-1-schnell/schema-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/flux-1-schnell/schema-input.json) Output [Open](https://developers.cloudflare.com/workers-ai/models/flux-1-schnell/schema-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/flux-1-schnell/schema-output.json) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/models/flux-1-schnell/#page","headline":"flux-1-schnell","description":"FLUX.1 [schnell] is a 12 billion parameter rectified flow transformer capable of generating images from text descriptions.","url":"https://developers.cloudflare.com/workers-ai/models/flux-1-schnell/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: FLUX.2 [dev] is an image model from Black Forest Labs where you can generate highly realistic and detailed images, with multi-reference support. title: flux-2-dev image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/llms.txt > Use this file to discover all available pages before exploring further. ![Black Forest Labs logo](https://developers.cloudflare.com/_astro/blackforestlabs.Ccs-Y4-D.svg) # flux-2-dev Text-to-Image • Black Forest Labs Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/models/flux-2-dev/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Model ID `@cf/black-forest-labs/flux-2-dev` Copy - Cloudflare-hosted - Partner - Zero data retention FLUX.2 \[dev] is an image model from Black Forest Labs where you can generate highly realistic and detailed images, with multi-reference support. | Model Info | | | --- | --- | | Terms and License | [link ↗](https://bfl.ai/legal/terms-of-service) | | Zero data retention | Yes | | Partner | Yes | ## Parameters ▶multipart{} `object`required image `string`Generated image as Base64 string. ## API Schemas (Raw) Input [Open](https://developers.cloudflare.com/workers-ai/models/flux-2-dev/schema-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/flux-2-dev/schema-input.json) Output [Open](https://developers.cloudflare.com/workers-ai/models/flux-2-dev/schema-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/flux-2-dev/schema-output.json) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/models/flux-2-dev/#page","headline":"flux-2-dev","description":"FLUX.2 [dev] is an image model from Black Forest Labs where you can generate highly realistic and detailed images, with multi-reference support.","url":"https://developers.cloudflare.com/workers-ai/models/flux-2-dev/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: FLUX.2 [klein] is an ultra-fast, distilled image model. It unifies image generation and editing in a single model, delivering state-of-the-art quality enabling interactive workflows, real-time previews, and latency-critical applications. title: flux-2-klein-4b image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/llms.txt > Use this file to discover all available pages before exploring further. ![Black Forest Labs logo](https://developers.cloudflare.com/_astro/blackforestlabs.Ccs-Y4-D.svg) # flux-2-klein-4b Text-to-Image • Black Forest Labs Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/models/flux-2-klein-4b/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Model ID `@cf/black-forest-labs/flux-2-klein-4b` Copy - Cloudflare-hosted - Partner - Zero data retention FLUX.2 \[klein] is an ultra-fast, distilled image model. It unifies image generation and editing in a single model, delivering state-of-the-art quality enabling interactive workflows, real-time previews, and latency-critical applications. | Model Info | | | --- | --- | | Terms and License | [link ↗](https://bfl.ai/legal/terms-of-service) | | Zero data retention | Yes | | Partner | Yes | ## Parameters ▶multipart{} `object`required image `string`Generated image as Base64 string. ## API Schemas (Raw) Input [Open](https://developers.cloudflare.com/workers-ai/models/flux-2-klein-4b/schema-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/flux-2-klein-4b/schema-input.json) Output [Open](https://developers.cloudflare.com/workers-ai/models/flux-2-klein-4b/schema-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/flux-2-klein-4b/schema-output.json) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/models/flux-2-klein-4b/#page","headline":"flux-2-klein-4b","description":"FLUX.2 [klein] is an ultra-fast, distilled image model. It unifies image generation and editing in a single model, delivering state-of-the-art quality enabling interactive workflows, real-time previews, and latency-critical applications.","url":"https://developers.cloudflare.com/workers-ai/models/flux-2-klein-4b/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: FLUX.2 [klein] 9B is a 9 billion parameter model that can generate images from text descriptions and supports multi-reference editing capabilities. title: flux-2-klein-9b image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/llms.txt > Use this file to discover all available pages before exploring further. ![Black Forest Labs logo](https://developers.cloudflare.com/_astro/blackforestlabs.Ccs-Y4-D.svg) # flux-2-klein-9b Text-to-Image • Black Forest Labs Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/models/flux-2-klein-9b/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Model ID `@cf/black-forest-labs/flux-2-klein-9b` Copy - Cloudflare-hosted - Partner - Zero data retention FLUX.2 \[klein] 9B is a 9 billion parameter model that can generate images from text descriptions and supports multi-reference editing capabilities. | Model Info | | | --- | --- | | Terms and License | [link ↗](https://bfl.ai/legal/terms-of-service) | | Zero data retention | Yes | | Partner | Yes | ## Parameters ▶multipart{} `object`required image `string`Generated image as Base64 string. ## API Schemas (Raw) Input [Open](https://developers.cloudflare.com/workers-ai/models/flux-2-klein-9b/schema-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/flux-2-klein-9b/schema-input.json) Output [Open](https://developers.cloudflare.com/workers-ai/models/flux-2-klein-9b/schema-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/flux-2-klein-9b/schema-output.json) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/models/flux-2-klein-9b/#page","headline":"flux-2-klein-9b","description":"FLUX.2 [klein] 9B is a 9 billion parameter model that can generate images from text descriptions and supports multi-reference editing capabilities.","url":"https://developers.cloudflare.com/workers-ai/models/flux-2-klein-9b/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: SDXL-Lightning is a lightning-fast text-to-image generation model. It can generate high-quality 1024px images in a few steps. title: stable-diffusion-xl-lightning image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/llms.txt > Use this file to discover all available pages before exploring further. ![ByteDance logo](https://developers.cloudflare.com/_astro/bytedance.T1uiROQ6.svg) # stable-diffusion-xl-lightning Beta Text-to-Image • ByteDance Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/models/stable-diffusion-xl-lightning/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Model ID `@cf/bytedance/stable-diffusion-xl-lightning` Copy - Cloudflare-hosted - Planned deprecation - Zero data retention SDXL-Lightning is a lightning-fast text-to-image generation model. It can generate high-quality 1024px images in a few steps. | Model Info | | | --- | --- | | Planned Deprecation | 11/20/2026 | | More information | [link ↗](https://huggingface.co/ByteDance/SDXL-Lightning) | | Zero data retention | Yes | | Beta | Yes | | Unit Pricing | $0.00 per step | ## Parameters prompt `string`requiredminLength: 1A text description of the image you want to generate negative\_prompt `string`Text describing elements to avoid in the generated image height `integer`minimum: 256maximum: 2048The height of the generated image in pixels width `integer`minimum: 256maximum: 2048The width of the generated image in pixels ▶image\[] `array`For use with img2img tasks. An array of integers that represent the image data constrained to 8-bit unsigned integer values image\_b64 `string`For use with img2img tasks. A base64-encoded string of the input image ▶mask\[] `array`An array representing An array of integers that represent mask image data for inpainting constrained to 8-bit unsigned integer values num\_steps `integer`default: 20maximum: 20The number of diffusion steps; higher values can improve quality but take longer strength `number`default: 1A value between 0 and 1 indicating how strongly to apply the transformation during img2img tasks; lower values make the output closer to the input image guidance `number`default: 7.5Controls how closely the generated image should adhere to the prompt; higher values make the image more aligned with the prompt seed `integer`Random seed for reproducibility of the image generation The binding returns a `ReadableStream` with the output (check the model's output schema). ## API Schemas (Raw) Input [Open](https://developers.cloudflare.com/workers-ai/models/stable-diffusion-xl-lightning/schema-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/stable-diffusion-xl-lightning/schema-input.json) Output [Open](https://developers.cloudflare.com/workers-ai/models/stable-diffusion-xl-lightning/schema-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/stable-diffusion-xl-lightning/schema-output.json) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/models/stable-diffusion-xl-lightning/#page","headline":"stable-diffusion-xl-lightning","description":"SDXL-Lightning is a lightning-fast text-to-image generation model. It can generate high-quality 1024px images in a few steps.","url":"https://developers.cloudflare.com/workers-ai/models/stable-diffusion-xl-lightning/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Clairic is a System One-style model that can extract, compute, estimate, and forecast values from inputs. It is strongly typed and outputs confidence scores and probabilities against its outputs. This model is in preview from the Cloudflare Workers AI team. title: clairic image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/llms.txt > Use this file to discover all available pages before exploring further. ![Cloudflare logo](https://developers.cloudflare.com/_astro/cloudflare.DP8rkHys.svg) # clairic Text Generation • Cloudflare Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/models/clairic/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Model ID `@cf/cloudflare/clairic` Copy - Cloudflare-hosted - Vision - Zero data retention Clairic is a System One-style model that can extract, compute, estimate, and forecast values from inputs. It is strongly typed and outputs confidence scores and probabilities against its outputs. This model is in preview from the Cloudflare Workers AI team. | Model Info | | | --- | --- | | Context Window [ ↗](https://developers.cloudflare.com/workers-ai/platform/glossary/) | 65,536 tokens | | Terms and License | [link](#terms) | | Vision | Yes | | Zero data retention | Yes | | Unit Pricing | $0.10 per M input tokens | ## Usage ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const response = await env.AI.run("@cf/cloudflare/clairic", { model: "clairic", state: "Invoice INV-2291 from Northwind Supply, issued 2026-09-02, net 30. 4 pallets at $250.00 each, shipping $80.00. Contact: Dana Ruiz, Priya Shah.", fields: { invoice_number: { type: "text" }, due_date: { type: "date", instructions: "Issue date plus the net payment days", }, total: { type: "number", decimals: 2, unit: "USD", instructions: "Amount due including shipping", }, contacts: { type: "list", items: "text" }, po_number: { type: "text", instructions: "Purchase order number" }, days_to_pay: { type: "estimate", min: 0, max: 120, step: 1, instructions: "Days after issue until this invoice is paid", }, }, }); // response.answers.invoice_number -> exact span of the state, with character offsets // response.answers.due_date -> date worked out from the issue date and terms // response.answers.po_number -> null: the invoice has no PO number // response.answers.days_to_pay -> median and 80% interval (p10 to p90) return Response.json(response); }, } satisfies ExportedHandler; ``` ```py import os import requests ACCOUNT_ID = "your-account-id" AUTH_TOKEN = os.environ.get("CLOUDFLARE_AUTH_TOKEN") response = requests.post( f"https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai/run/@cf/cloudflare/clairic", headers={"Authorization": f"Bearer {AUTH_TOKEN}"}, json={ "model": "clairic", "state": "Invoice INV-2291 from Northwind Supply, issued 2026-09-02, net 30. 4 pallets at $250.00 each, shipping $80.00. Contact: Dana Ruiz, Priya Shah.", "fields": { "invoice_number": {"type": "text"}, "due_date": { "type": "date", "instructions": "Issue date plus the net payment days", }, "total": { "type": "number", "decimals": 2, "unit": "USD", "instructions": "Amount due including shipping", }, "contacts": {"type": "list", "items": "text"}, "po_number": {"type": "text", "instructions": "Purchase order number"}, "days_to_pay": { "type": "estimate", "min": 0, "max": 120, "step": 1, "instructions": "Days after issue until this invoice is paid", }, }, }, ) print(response.json()) ``` ```sh curl https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/run/@cf/cloudflare/clairic \ -X POST \ -H "Authorization: Bearer $CLOUDFLARE_AUTH_TOKEN" \ -d '{ "model": "clairic", "state": "Invoice INV-2291 from Northwind Supply, issued 2026-09-02, net 30. 4 pallets at $250.00 each, shipping $80.00. Contact: Dana Ruiz, Priya Shah.", "fields": { "invoice_number": { "type": "text" }, "due_date": { "type": "date", "instructions": "Issue date plus the net payment days" }, "total": { "type": "number", "decimals": 2, "unit": "USD", "instructions": "Amount due including shipping" }, "contacts": { "type": "list", "items": "text" }, "po_number": { "type": "text", "instructions": "Purchase order number" }, "days_to_pay": { "type": "estimate", "min": 0, "max": 120, "step": 1, "instructions": "Days after issue until this invoice is paid" } } }' ``` ```json { "model": "clairic", "answers": { "invoice_number": {"type": "text", "p_null": 3.8e-10, "span": [8, 16], "guess": "INV-2291", "confidence": 1, "value": "INV-2291"}, "due_date": {"type": "date", "p_null": 1.9e-08, "guess": "2026-10-02", "confidence": 0.9999, "value": "2026-10-02"}, "total": {"type": "number", "p_null": 2.3e-08, "guess": 1080, "confidence": 0.9999, "value": 1080}, "contacts": {"type": "list", "value": ["Dana Ruiz", "Priya Shah"], "spans": [[120, 129], [131, 141]], "confidence": 0.9999}, "po_number": {"type": "text", "p_null": 1, "span": [8, 16], "guess": "INV-2291", "confidence": 0.9995, "value": null}, "days_to_pay": {"type": "estimate", "value": 30, "mean": 29.535, "median": 30, "p10": 21, "p90": 38, "p_below": 5.9e-06, "p_above": 1.2e-05} }, "truncated": false, "usage": {"state_tokens": 61, "image_tokens": 0, "schema_tokens": 472, "input_tokens": 655, "output_tokens": 0, "passes": 3} } ``` ## Parameters model `string`requireddefault: clairicpattern: ^(clairic|@cf/cloudflare/clairic)$Required. The model selector: "clairic". state `string`default: The text to read and predict from. Must be non-empty unless images are sent. Text answers are spans of this string. ▶fields{} `object`requiredminProperties: 1Required. Map of field name to a typed field spec. At least one field; no upper limit. Fields that do not fit next to the state in one 8,192-token pass are split across parallel passes. ▶images\[] `array`maxItems: 4Optional embedded PNG, JPEG or WebP images placed before the state (max 4; 4 MiB each, 8 MiB total decoded; request body max 16 MiB). Remote URLs are not accepted. reference\_date `string`ISO date that anchors dates written without a year. ▶options{} `object` model `string` ▶answers{} `object`One answer per field, keyed by field name. value is null when the input does not determine it. truncated `boolean` state\_text `string` ▶usage{} `object` ## API Schemas (Raw) Input [Open](https://developers.cloudflare.com/workers-ai/models/clairic/schema-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/clairic/schema-input.json) Output [Open](https://developers.cloudflare.com/workers-ai/models/clairic/schema-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/clairic/schema-output.json) ## Terms You may access or use this model at your sole discretion. Cloudflare does not make any representations, warranties, or guarantees regarding the model, including, but not limited to, as to its continued availability and integrity. If Cloudflare makes the model available to you, it is on an “AS IS” and “AS AVAILABLE” basis, and Cloudflare has no obligation to indemnify you for any claims arising from your use of the model. Cloudflare may cease providing it without entitling you to any refund, credit, or other compensation. Cloudflare will not be directly or indirectly responsible or liable to you in any manner, for any harms, damages, loss, lost profits, special or consequential damages, or claims, arising out of or in connection with the use of, or reliance on the performance of the model. Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/models/clairic/#page","headline":"clairic","description":"Clairic is a System One-style model that can extract, compute, estimate, and forecast values from inputs. It is strongly typed and outputs confidence scores and probabilities against its outputs. This model is in preview from the Cloudflare Workers AI team.","url":"https://developers.cloudflare.com/workers-ai/models/clairic/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Clef is a 27B multimodal decision model that turns a state and a schema of typed questions into decisions. It reads the state as text, JSON, images, or video, and returns a probability for every allowed option of every question. title: clef image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/llms.txt > Use this file to discover all available pages before exploring further. ![Cloudflare logo](https://developers.cloudflare.com/_astro/cloudflare.DP8rkHys.svg) # clef Text Generation • Cloudflare Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/models/clef/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Model ID `@cf/cloudflare/clef` Copy - Cloudflare-hosted - Vision - Zero data retention Clef is a 27B multimodal decision model that turns a state and a schema of typed questions into decisions. It reads the state as text, JSON, images, or video, and returns a probability for every allowed option of every question. | Model Info | | | --- | --- | | Context Window [ ↗](https://developers.cloudflare.com/workers-ai/platform/glossary/) | 65,536 tokens | | Terms and License | [link ↗](https://huggingface.co/Cloudflare/clef/blob/main/LICENSE) | | More information | [link ↗](https://huggingface.co/Cloudflare/clef) | | Vision | Yes | | Zero data retention | Yes | | Unit Pricing | $0.24 per M input tokens | How media inputs are billed Clef models convert media inputs to input tokens and bill them at the model's input token rate. Clef models do not charge for output tokens. For per-model rates, refer to [pricing](https://developers.cloudflare.com/workers-ai/platform/pricing/). Images are tokenized as follows: 1. **Resize**: The image keeps its aspect ratio, but: - Each side is rounded to a multiple of 32 pixels. - If the area is under about 65,000 pixels (256x256), it is scaled up to reach that. - If the area is over about 1 megapixel, it is scaled down to fit. 2. **Count patches**: Each 32x32-pixel block is one token: `tokens = (width / 32) x (height / 32)`. 3. **Add markers**: 3 tokens are added to mark where the image starts and ends. Each image is capped at 1,024 tokens. Media tokens count toward the model's context window, together with the questions. If they exceed the context window, the request fails. Otherwise, the text `state` is truncated to fit the remaining space. ## Usage ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const response = await env.AI.run("@cf/cloudflare/clef", { model: "clef", state: "Checkout has been failing for every customer for the last hour.", questions: { urgent: { type: "noul", instructions: "Is this support request urgent?", }, team: { type: "choice", instructions: "Which team should handle this request?", criteria: { billing: "Payments, invoices, and refunds", technical: "Outages, errors, and configuration", sales: "Plans and upgrades", }, }, severity: { type: "score", instructions: "How severe is the customer impact?", criteria: ["No impact", "Minor", "Major", "Critical"], }, }, }); // response.answers.urgent -> probability the request is urgent // response.answers.team -> chosen team with per-option probabilities // response.answers.severity -> probability-weighted score (0 = lowest level) return Response.json(response); }, } satisfies ExportedHandler; ``` ```py import os import requests ACCOUNT_ID = "your-account-id" AUTH_TOKEN = os.environ.get("CLOUDFLARE_AUTH_TOKEN") response = requests.post( f"https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai/run/@cf/cloudflare/clef", headers={"Authorization": f"Bearer {AUTH_TOKEN}"}, json={ "model": "clef", "state": "Checkout has been failing for every customer for the last hour.", "questions": { "urgent": { "type": "noul", "instructions": "Is this support request urgent?", }, "team": { "type": "choice", "instructions": "Which team should handle this request?", "criteria": { "billing": "Payments, invoices, and refunds", "technical": "Outages, errors, and configuration", "sales": "Plans and upgrades", }, }, "severity": { "type": "score", "instructions": "How severe is the customer impact?", "criteria": ["No impact", "Minor", "Major", "Critical"], }, }, }, ) print(response.json()) ``` ```sh curl https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/run/@cf/cloudflare/clef \ -X POST \ -H "Authorization: Bearer $CLOUDFLARE_AUTH_TOKEN" \ -d '{ "model": "clef", "state": "Checkout has been failing for every customer for the last hour.", "questions": { "urgent": { "type": "noul", "instructions": "Is this support request urgent?" }, "team": { "type": "choice", "instructions": "Which team should handle this request?", "criteria": { "billing": "Payments, invoices, and refunds", "technical": "Outages, errors, and configuration", "sales": "Plans and upgrades" } }, "severity": { "type": "score", "instructions": "How severe is the customer impact?", "criteria": ["No impact", "Minor", "Major", "Critical"] } } }' ``` ## Parameters model `string`requiredpattern: ^\\s\*(clef|clef-flash)\\s\*$Required. The model selector: "clef" for @cf/cloudflare/clef, "clef-flash" for @cf/cloudflare/clef-flash. state requiredRequired. The content to evaluate: a string, or structured data (object/array) such as records, chat logs, or application state. Long text state is truncated to fit the model's token limit. ▶questions{} `object`requiredminProperties: 1maxProperties: 64Map of question id to a typed question (noul, choice, or score). 1 to 64 questions; ids may use letters, digits, '\_', '.', '-' (max 100 chars). Answers are returned under the same ids. ▶images\[] `array`maxItems: 4Clef extension to the System One API. Optional embedded PNG, JPEG, or WebP images placed before the state (max 4; 4 MiB and 16 megapixels each, 8 MiB total decoded; whole request body max 13 MiB). Remote URLs are not accepted. model `string`The model that performed the evaluation. ▶answers{} `object`One answer per question, keyed by the question ids from the request. ▶usage{} `object` ## API Schemas (Raw) Input [Open](https://developers.cloudflare.com/workers-ai/models/clef/schema-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/clef/schema-input.json) Output [Open](https://developers.cloudflare.com/workers-ai/models/clef/schema-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/clef/schema-output.json) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/models/clef/#page","headline":"clef","description":"Clef is a 27B multimodal decision model that turns a state and a schema of typed questions into decisions. It reads the state as text, JSON, images, or video, and returns a probability for every allowed option of every question.","url":"https://developers.cloudflare.com/workers-ai/models/clef/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Clef-flash is a fast 9B multimodal decision model that turns a state and a schema of typed questions into decisions. It reads the state as text, JSON, images, or video, and returns a probability for every allowed option of every question. title: clef-flash image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/llms.txt > Use this file to discover all available pages before exploring further. ![Cloudflare logo](https://developers.cloudflare.com/_astro/cloudflare.DP8rkHys.svg) # clef-flash Text Generation • Cloudflare Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/models/clef-flash/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Model ID `@cf/cloudflare/clef-flash` Copy - Cloudflare-hosted - Vision - Zero data retention Clef-flash is a fast 9B multimodal decision model that turns a state and a schema of typed questions into decisions. It reads the state as text, JSON, images, or video, and returns a probability for every allowed option of every question. | Model Info | | | --- | --- | | Context Window [ ↗](https://developers.cloudflare.com/workers-ai/platform/glossary/) | 24,576 tokens | | Terms and License | [link ↗](https://huggingface.co/Cloudflare/clef-flash/blob/main/LICENSE) | | More information | [link ↗](https://huggingface.co/Cloudflare/clef-flash) | | Vision | Yes | | Zero data retention | Yes | | Unit Pricing | $0.038 per M input tokens | How media inputs are billed Clef models convert media inputs to input tokens and bill them at the model's input token rate. Clef models do not charge for output tokens. For per-model rates, refer to [pricing](https://developers.cloudflare.com/workers-ai/platform/pricing/). Images are tokenized as follows: 1. **Resize**: The image keeps its aspect ratio, but: - Each side is rounded to a multiple of 32 pixels. - If the area is under about 65,000 pixels (256x256), it is scaled up to reach that. - If the area is over about 1 megapixel, it is scaled down to fit. 2. **Count patches**: Each 32x32-pixel block is one token: `tokens = (width / 32) x (height / 32)`. 3. **Add markers**: 3 tokens are added to mark where the image starts and ends. Each image is capped at 1,024 tokens. Media tokens count toward the model's context window, together with the questions. If they exceed the context window, the request fails. Otherwise, the text `state` is truncated to fit the remaining space. ## Usage ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const response = await env.AI.run("@cf/cloudflare/clef-flash", { model: "clef-flash", state: "Checkout has been failing for every customer for the last hour.", questions: { urgent: { type: "noul", instructions: "Is this support request urgent?", }, team: { type: "choice", instructions: "Which team should handle this request?", criteria: { billing: "Payments, invoices, and refunds", technical: "Outages, errors, and configuration", sales: "Plans and upgrades", }, }, severity: { type: "score", instructions: "How severe is the customer impact?", criteria: ["No impact", "Minor", "Major", "Critical"], }, }, }); // response.answers.urgent -> probability the request is urgent // response.answers.team -> chosen team with per-option probabilities // response.answers.severity -> probability-weighted score (0 = lowest level) return Response.json(response); }, } satisfies ExportedHandler; ``` ```py import os import requests ACCOUNT_ID = "your-account-id" AUTH_TOKEN = os.environ.get("CLOUDFLARE_AUTH_TOKEN") response = requests.post( f"https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai/run/@cf/cloudflare/clef-flash", headers={"Authorization": f"Bearer {AUTH_TOKEN}"}, json={ "model": "clef-flash", "state": "Checkout has been failing for every customer for the last hour.", "questions": { "urgent": { "type": "noul", "instructions": "Is this support request urgent?", }, "team": { "type": "choice", "instructions": "Which team should handle this request?", "criteria": { "billing": "Payments, invoices, and refunds", "technical": "Outages, errors, and configuration", "sales": "Plans and upgrades", }, }, "severity": { "type": "score", "instructions": "How severe is the customer impact?", "criteria": ["No impact", "Minor", "Major", "Critical"], }, }, }, ) print(response.json()) ``` ```sh curl https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/run/@cf/cloudflare/clef-flash \ -X POST \ -H "Authorization: Bearer $CLOUDFLARE_AUTH_TOKEN" \ -d '{ "model": "clef-flash", "state": "Checkout has been failing for every customer for the last hour.", "questions": { "urgent": { "type": "noul", "instructions": "Is this support request urgent?" }, "team": { "type": "choice", "instructions": "Which team should handle this request?", "criteria": { "billing": "Payments, invoices, and refunds", "technical": "Outages, errors, and configuration", "sales": "Plans and upgrades" } }, "severity": { "type": "score", "instructions": "How severe is the customer impact?", "criteria": ["No impact", "Minor", "Major", "Critical"] } } }' ``` ## Parameters model `string`requiredpattern: ^\\s\*(clef|clef-flash)\\s\*$Required. The model selector: "clef" for @cf/cloudflare/clef, "clef-flash" for @cf/cloudflare/clef-flash. state requiredRequired. The content to evaluate: a string, or structured data (object/array) such as records, chat logs, or application state. Long text state is truncated to fit the model's token limit. ▶questions{} `object`requiredminProperties: 1maxProperties: 64Map of question id to a typed question (noul, choice, or score). 1 to 64 questions; ids may use letters, digits, '\_', '.', '-' (max 100 chars). Answers are returned under the same ids. ▶images\[] `array`maxItems: 4Clef extension to the System One API. Optional embedded PNG, JPEG, or WebP images placed before the state (max 4; 4 MiB and 16 megapixels each, 8 MiB total decoded; whole request body max 13 MiB). Remote URLs are not accepted. model `string`The model that performed the evaluation. ▶answers{} `object`One answer per question, keyed by the question ids from the request. ▶usage{} `object` ## API Schemas (Raw) Input [Open](https://developers.cloudflare.com/workers-ai/models/clef-flash/schema-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/clef-flash/schema-input.json) Output [Open](https://developers.cloudflare.com/workers-ai/models/clef-flash/schema-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/clef-flash/schema-output.json) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/models/clef-flash/#page","headline":"clef-flash","description":"Clef-flash is a fast 9B multimodal decision model that turns a state and a schema of typed questions into decisions. It reads the state as text, JSON, images, or video, and returns a probability for every allowed option of every question.","url":"https://developers.cloudflare.com/workers-ai/models/clef-flash/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Clef-omni is a multimodal decision model built on a 30B-parameter mixture-of-experts (3B active) backbone. It turns a state and a schema of typed questions into decisions, reads the state as text, JSON, images, audio, or video, and returns a probability for every allowed option of every question. title: clef-omni image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/llms.txt > Use this file to discover all available pages before exploring further. ![Cloudflare logo](https://developers.cloudflare.com/_astro/cloudflare.DP8rkHys.svg) # clef-omni Text Generation • Cloudflare Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/models/clef-omni/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Model ID `@cf/cloudflare/clef-omni` Copy - Cloudflare-hosted - Vision - Zero data retention Clef-omni is a multimodal decision model built on a 30B-parameter mixture-of-experts (3B active) backbone. It turns a state and a schema of typed questions into decisions, reads the state as text, JSON, images, audio, or video, and returns a probability for every allowed option of every question. | Model Info | | | --- | --- | | Context Window [ ↗](https://developers.cloudflare.com/workers-ai/platform/glossary/) | 64,000 tokens | | Terms and License | [link ↗](https://huggingface.co/Cloudflare/clef-omni/blob/main/LICENSE) | | More information | [link ↗](https://huggingface.co/Cloudflare/clef-omni) | | Vision | Yes | | Zero data retention | Yes | | Unit Pricing | $0.15 per M input tokens | How media inputs are billed Clef models convert media inputs to input tokens and bill them at the model's input token rate. Clef models do not charge for output tokens. For per-model rates, refer to [pricing](https://developers.cloudflare.com/workers-ai/platform/pricing/). Images are tokenized as follows: 1. **Resize**: The image keeps its aspect ratio, but: - Each side is rounded to a multiple of 32 pixels. - If the area is under about 65,000 pixels (256x256), it is scaled up to reach that. - If the area is over about 1 megapixel, it is scaled down to fit. 2. **Count patches**: Each 32x32-pixel block is one token: `tokens = (width / 32) x (height / 32)`. 3. **Add markers**: 3 tokens are added to mark where the image starts and ends. Each image is capped at 1,024 tokens. Audio and video are tokenized by duration: - **Audio**: about 780 tokens per minute. - **Video**: up to about 15,400 tokens per minute at maximum resolution, plus about 780 tokens per minute if the video has sound. A 480p clip uses about 8,600 frame tokens per minute (144 tokens per second). Each request accepts up to 4 audio clips and 2 videos. Media tokens count toward the model's context window, together with the questions. If they exceed the context window, the request fails. Otherwise, the text `state` is truncated to fit the remaining space. ## Usage ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const response = await env.AI.run("@cf/cloudflare/clef-omni", { model: "clef-omni", state: "Checkout has been failing for every customer for the last hour.", questions: { urgent: { type: "noul", instructions: "Is this support request urgent?", }, team: { type: "choice", instructions: "Which team should handle this request?", criteria: { billing: "Payments, invoices, and refunds", technical: "Outages, errors, and configuration", sales: "Plans and upgrades", }, }, severity: { type: "score", instructions: "How severe is the customer impact?", criteria: ["No impact", "Minor", "Major", "Critical"], }, }, }); // response.answers.urgent -> probability the request is urgent // response.answers.team -> chosen team with per-option probabilities // response.answers.severity -> probability-weighted score (0 = lowest level) return Response.json(response); }, } satisfies ExportedHandler; ``` ```py import os import requests ACCOUNT_ID = "your-account-id" AUTH_TOKEN = os.environ.get("CLOUDFLARE_AUTH_TOKEN") response = requests.post( f"https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai/run/@cf/cloudflare/clef-omni", headers={"Authorization": f"Bearer {AUTH_TOKEN}"}, json={ "model": "clef-omni", "state": "Checkout has been failing for every customer for the last hour.", "questions": { "urgent": { "type": "noul", "instructions": "Is this support request urgent?", }, "team": { "type": "choice", "instructions": "Which team should handle this request?", "criteria": { "billing": "Payments, invoices, and refunds", "technical": "Outages, errors, and configuration", "sales": "Plans and upgrades", }, }, "severity": { "type": "score", "instructions": "How severe is the customer impact?", "criteria": ["No impact", "Minor", "Major", "Critical"], }, }, }, ) print(response.json()) ``` ```sh curl https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/run/@cf/cloudflare/clef-omni \ -X POST \ -H "Authorization: Bearer $CLOUDFLARE_AUTH_TOKEN" \ -d '{ "model": "clef-omni", "state": "Checkout has been failing for every customer for the last hour.", "questions": { "urgent": { "type": "noul", "instructions": "Is this support request urgent?" }, "team": { "type": "choice", "instructions": "Which team should handle this request?", "criteria": { "billing": "Payments, invoices, and refunds", "technical": "Outages, errors, and configuration", "sales": "Plans and upgrades" } }, "severity": { "type": "score", "instructions": "How severe is the customer impact?", "criteria": ["No impact", "Minor", "Major", "Critical"] } } }' ``` ## Parameters model `string`requireddefault: clef-omnipattern: ^\\s\*clef-omni\\s\*$Required. The model selector: "clef-omni" for @cf/cloudflare/clef-omni. state requiredRequired. The content to evaluate: a string, or structured data (object/array) such as records, chat logs, or application state. Long text state is truncated to fit the model's token limit. ▶questions{} `object`requiredminProperties: 1maxProperties: 64Map of question id to a typed question (noul, choice, or score). 1 to 64 questions; ids may use letters, digits, '\_', '.', '-' (max 100 chars). Answers are returned under the same ids. ▶images\[] `array`maxItems: 4Clef extension to the System One API. Optional embedded PNG, JPEG, or WebP images placed before the state (max 4; 4 MiB and 16 megapixels each, 8 MiB total decoded). Each image costs 64 to 1,024 input tokens. Remote URLs are not accepted. ▶audio\[] `array`maxItems: 4Clef-Omni extension. Optional embedded audio clips (max 4; 8 MiB and 300 seconds each). Audio and video clips together may total at most 16 MiB decoded. Remote URLs are not accepted. ▶videos\[] `array`maxItems: 2Clef-Omni extension. Optional embedded videos (max 2; 16 MiB and 60 seconds each), sampled at 2 frames per second. A video's soundtrack is heard with its frames when every video in the request has one. Audio and video clips together may total at most 16 MiB decoded. Each second of video costs up to 256 input tokens. Remote URLs are not accepted. model `string`The model that performed the evaluation. ▶answers{} `object`One answer per question, keyed by the question ids from the request. ▶usage{} `object` ## API Schemas (Raw) Input [Open](https://developers.cloudflare.com/workers-ai/models/clef-omni/schema-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/clef-omni/schema-input.json) Output [Open](https://developers.cloudflare.com/workers-ai/models/clef-omni/schema-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/clef-omni/schema-output.json) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/models/clef-omni/#page","headline":"clef-omni","description":"Clef-omni is a multimodal decision model built on a 30B-parameter mixture-of-experts (3B active) backbone. It turns a state and a schema of typed questions into decisions, reads the state as text, JSON, images, audio, or video, and returns a probability for every allowed option of every question.","url":"https://developers.cloudflare.com/workers-ai/models/clef-omni/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: PhoneLLM is an open model tuned for voice agents. The model is based on Nemotron Nano 30B base capabilities and excels at typical voice agent tasks with thinking disabled. title: phonellm-mini-alpha-2 image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/llms.txt > Use this file to discover all available pages before exploring further. d # phonellm-mini-alpha-2 Text Generation • daily-co Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/models/phonellm-mini-alpha-2/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Model ID `@cf/daily-co/phonellm-mini-alpha-2` Copy - Cloudflare-hosted - Function calling - Zero data retention PhoneLLM is an open model tuned for voice agents. The model is based on Nemotron Nano 30B base capabilities and excels at typical voice agent tasks with thinking disabled. | Model Info | | | --- | --- | | Context Window [ ↗](https://developers.cloudflare.com/workers-ai/platform/glossary/) | 262,140 tokens | | Function calling [↗](https://developers.cloudflare.com/workers-ai/function-calling/) | Yes | | Zero data retention | Yes | | Unit Pricing | $0.06 per M input tokens, $0.24 per M output tokens | ## Playground Try out this model with Workers AI LLM Playground. It does not require any setup or authentication and is an instant way to preview and test a model directly in the browser. [Launch the LLM Playground](https://playground.ai.cloudflare.com/?model=@cf/daily-co/phonellm-mini-alpha-2) ## Usage ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const messages = [ { role: "system", content: "You are a friendly assistant" }, { role: "user", content: "What is the origin of the phrase Hello, World", }, ]; const stream = await env.AI.run("@cf/daily-co/phonellm-mini-alpha-2", { messages, stream: true, }); return new Response(stream, { headers: { "content-type": "text/event-stream" }, }); }, } satisfies ExportedHandler; ``` ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const messages = [ { role: "system", content: "You are a friendly assistant" }, { role: "user", content: "What is the origin of the phrase Hello, World", }, ]; const response = await env.AI.run("@cf/daily-co/phonellm-mini-alpha-2", { messages }); return Response.json(response); }, } satisfies ExportedHandler; ``` ```py import os import requests ACCOUNT_ID = "your-account-id" AUTH_TOKEN = os.environ.get("CLOUDFLARE_AUTH_TOKEN") prompt = "Tell me all about PEP-8" response = requests.post( f"https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai/run/@cf/daily-co/phonellm-mini-alpha-2", headers={"Authorization": f"Bearer {AUTH_TOKEN}"}, json={ "messages": [ {"role": "system", "content": "You are a friendly assistant"}, {"role": "user", "content": prompt} ] } ) result = response.json() print(result) ``` ```sh curl https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/run/@cf/daily-co/phonellm-mini-alpha-2 \ -X POST \ -H "Authorization: Bearer $CLOUDFLARE_AUTH_TOKEN" \ -d '{ "messages": [{ "role": "system", "content": "You are a friendly assistant" }, { "role": "user", "content": "Why is pizza so good" }]}' ``` OpenAI compatible endpoints Workers AI also supports OpenAI compatible API endpoints for `/v1/chat/completions` and `/v1/embeddings`. For more details, refer to [Configurations](https://developers.cloudflare.com/workers-ai/configuration/open-ai-compatibility/) . ## Parameters ### Input model `string`ID of the model to use (for example, '@cf/daily-co/phonellm-mini-alpha-2'). ▶audio{} `object`Parameters for audio output. Required when modalities includes 'audio'. frequency\_penalty `number | null`Penalizes new tokens based on their existing frequency in the text so far. logit\_bias `object | null`Modify the likelihood of specified tokens appearing in the completion. Maps token IDs to bias values from -100 to 100. logprobs `boolean | null`Whether to return log probabilities of the output tokens. top\_logprobs `integer | null`How many top log probabilities to return at each token position (0-20). Requires logprobs=true. max\_tokens `integer | null`The maximum number of tokens to generate. max\_completion\_tokens `integer | null`An upper bound for the number of tokens that can be generated for a completion. metadata `object | null`Set of key-value pairs that can be attached to the object. modalities `array | null`Output types requested from the model. n `integer | null`How many chat completion choices to generate for each input message. parallel\_tool\_calls `boolean`default: trueWhether to enable parallel function calling during tool use. ▶prediction{} `object` presence\_penalty `number | null`Penalizes new tokens based on whether they appear in the text so far. ▶chat\_template\_kwargs{} `object`Chat-template controls. ▶response\_format `one of`Specifies the format the model must output. seed `integer | null`If specified, the system will make a best effort to sample deterministically. ▶stop `one of` store `boolean | null`Whether to store the output for model distillation or evaluation. stream `boolean | null`If true, partial message deltas will be sent as server-sent events. ▶stream\_options{} `object` temperature `number | null`Sampling temperature between 0 and 2. ▶tool\_choice `one of`Controls which (if any) tool is called by the model. 'none' = no tools, 'auto' = model decides, 'required' = must call a tool. ▶tools\[] `array`A list of tools the model may call. top\_p `number | null`Nucleus sampling: considers the results of the tokens with top\_p probability mass. user `string`A unique identifier representing your end-user, for abuse monitoring. ▶web\_search\_options{} `object`Options for the web search tool (when using built-in web search). ▶function\_call `one of` ▶functions\[] `array`minItems: 1maxItems: 128 prompt `string`requiredminLength: 1The input text prompt for the model to generate a response. ### Output
Synchronous — Send a request and receive a complete response id stringA unique identifier for the chat completion. object string created integerUnix timestamp (seconds) of when the completion was created. model stringThe model used for the chat completion. ▶choices\[] arrayminItems: 1 ▶usage{} object system\_fingerprint string | null
Streaming — Send a request with `stream: true` and receive server-sent events type string contentType text/event-stream format binary
## API Schemas (Raw) SynchronousInput [Open](https://developers.cloudflare.com/workers-ai/models/phonellm-mini-alpha-2/sync-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/phonellm-mini-alpha-2/sync-input.json) SynchronousOutput [Open](https://developers.cloudflare.com/workers-ai/models/phonellm-mini-alpha-2/sync-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/phonellm-mini-alpha-2/sync-output.json) StreamingInput [Open](https://developers.cloudflare.com/workers-ai/models/phonellm-mini-alpha-2/streaming-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/phonellm-mini-alpha-2/streaming-input.json) StreamingOutput [Open](https://developers.cloudflare.com/workers-ai/models/phonellm-mini-alpha-2/streaming-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/phonellm-mini-alpha-2/streaming-output.json) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/models/phonellm-mini-alpha-2/#page","headline":"phonellm-mini-alpha-2","description":"PhoneLLM is an open model tuned for voice agents. The model is based on Nemotron Nano 30B base capabilities and excels at typical voice agent tasks with thinking disabled.","url":"https://developers.cloudflare.com/workers-ai/models/phonellm-mini-alpha-2/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Aura is a context-aware text-to-speech (TTS) model that applies natural pacing, expressiveness, and fillers based on the context of the provided text. The quality of your text input directly impacts the naturalness of the audio output. title: aura-1 image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/llms.txt > Use this file to discover all available pages before exploring further. ![Deepgram logo](https://developers.cloudflare.com/_astro/deepgram.BYzW8KfF.svg) # aura-1 Text-to-Speech • Deepgram Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/models/aura-1/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Model ID `@cf/deepgram/aura-1` Copy - Cloudflare-hosted - Batch - Partner - Real-time - Zero data retention Aura is a context-aware text-to-speech (TTS) model that applies natural pacing, expressiveness, and fillers based on the context of the provided text. The quality of your text input directly impacts the naturalness of the audio output. | Model Info | | | --- | --- | | Terms and License | [link ↗](https://deepgram.com/terms) | | Zero data retention | Yes | | Batch | Yes | | Partner | Yes | | Real-time | Yes | | Unit Pricing | $0.015 per 1k characters | ## Usage ```ts export default { async fetch(request, env, ctx): Promise { const resp = await env.AI.run("@cf/deepgram/aura-1", { "text":"Hello World!" }, { returnRawResponse: true }); return resp; }, } satisfies ExportedHandler; ``` ```sh curl --request POST --url 'https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai/run/@cf/deepgram/aura-1' --header 'Authorization: Bearer {TOKEN}' --header 'Content-Type: application/json' --data '{ "text":"Hello world!" }' ``` ## Parameters speaker `string`default: angusenum: angus, asteria, arcas, orion, orpheus, athena, luna, zeus, perseus, helios, hera, stellaSpeaker used to produce the audio. encoding `string`enum: linear16, flac, mulaw, alaw, mp3, opus, aacEncoding of the output audio. container `string`enum: none, wav, oggContainer specifies the file format wrapper for the output audio. The available options depend on the encoding type.. text `string`requiredThe text content to be converted to speech sample\_rate `number`Sample Rate specifies the sample rate for the output audio. Based on the encoding, different sample rates are supported. For some encodings, the sample rate is not configurable bit\_rate `number`The bitrate of the audio in bits per second. Choose from predefined ranges or specific values based on the encoding type. The binding returns a `ReadableStream` with the audio in MPEG format (check the model's output schema). ## API Schemas (Raw) Input [Open](https://developers.cloudflare.com/workers-ai/models/aura-1/schema-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/aura-1/schema-input.json) Output [Open](https://developers.cloudflare.com/workers-ai/models/aura-1/schema-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/aura-1/schema-output.json) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/models/aura-1/#page","headline":"aura-1","description":"Aura is a context-aware text-to-speech (TTS) model that applies natural pacing, expressiveness, and fillers based on the context of the provided text. The quality of your text input directly impacts the naturalness of the audio output.","url":"https://developers.cloudflare.com/workers-ai/models/aura-1/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Aura-2 is a context-aware text-to-speech (TTS) model that applies natural pacing, expressiveness, and fillers based on the context of the provided text. The quality of your text input directly impacts the naturalness of the audio output. title: aura-2-en image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/llms.txt > Use this file to discover all available pages before exploring further. ![Deepgram logo](https://developers.cloudflare.com/_astro/deepgram.BYzW8KfF.svg) # aura-2-en Text-to-Speech • Deepgram Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/models/aura-2-en/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Model ID `@cf/deepgram/aura-2-en` Copy - Cloudflare-hosted - Batch - Partner - Real-time - Zero data retention Aura-2 is a context-aware text-to-speech (TTS) model that applies natural pacing, expressiveness, and fillers based on the context of the provided text. The quality of your text input directly impacts the naturalness of the audio output. | Model Info | | | --- | --- | | Terms and License | [link ↗](https://deepgram.com/terms) | | Zero data retention | Yes | | Batch | Yes | | Partner | Yes | | Real-time | Yes | | Unit Pricing | $0.03 per 1k characters | ## Parameters speaker `string`default: lunaenum: amalthea, andromeda, apollo, arcas, aries, asteria, athena, atlas, aurora, callista, cora, cordelia, delia, draco, electra, harmonia, helena, hera, hermes, hyperion, iris, janus, juno, jupiter, luna, mars, minerva, neptune, odysseus, ophelia, orion, orpheus, pandora, phoebe, pluto, saturn, thalia, theia, vesta, zeusSpeaker used to produce the audio. encoding `string`enum: linear16, flac, mulaw, alaw, mp3, opus, aacEncoding of the output audio. container `string`enum: none, wav, oggContainer specifies the file format wrapper for the output audio. The available options depend on the encoding type.. text `string`The text content to be converted to speech sample\_rate `number`Sample Rate specifies the sample rate for the output audio. Based on the encoding, different sample rates are supported. For some encodings, the sample rate is not configurable bit\_rate `number`The bitrate of the audio in bits per second. Choose from predefined ranges or specific values based on the encoding type. The binding returns a `ReadableStream` with the audio in MPEG format (check the model's output schema). ## API Schemas (Raw) Input [Open](https://developers.cloudflare.com/workers-ai/models/aura-2-en/schema-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/aura-2-en/schema-input.json) Output [Open](https://developers.cloudflare.com/workers-ai/models/aura-2-en/schema-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/aura-2-en/schema-output.json) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/models/aura-2-en/#page","headline":"aura-2-en","description":"Aura-2 is a context-aware text-to-speech (TTS) model that applies natural pacing, expressiveness, and fillers based on the context of the provided text. The quality of your text input directly impacts the naturalness of the audio output.","url":"https://developers.cloudflare.com/workers-ai/models/aura-2-en/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Aura-2 is a context-aware text-to-speech (TTS) model that applies natural pacing, expressiveness, and fillers based on the context of the provided text. The quality of your text input directly impacts the naturalness of the audio output. title: aura-2-es image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/llms.txt > Use this file to discover all available pages before exploring further. ![Deepgram logo](https://developers.cloudflare.com/_astro/deepgram.BYzW8KfF.svg) # aura-2-es Text-to-Speech • Deepgram Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/models/aura-2-es/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Model ID `@cf/deepgram/aura-2-es` Copy - Cloudflare-hosted - Batch - Partner - Real-time - Zero data retention Aura-2 is a context-aware text-to-speech (TTS) model that applies natural pacing, expressiveness, and fillers based on the context of the provided text. The quality of your text input directly impacts the naturalness of the audio output. | Model Info | | | --- | --- | | Terms and License | [link ↗](https://deepgram.com/terms) | | Zero data retention | Yes | | Batch | Yes | | Partner | Yes | | Real-time | Yes | | Unit Pricing | $0.03 per 1k characters | ## Parameters speaker `string`default: aquilaenum: sirio, nestor, carina, celeste, alvaro, diana, aquila, selena, estrella, javierSpeaker used to produce the audio. encoding `string`enum: linear16, flac, mulaw, alaw, mp3, opus, aacEncoding of the output audio. container `string`enum: none, wav, oggContainer specifies the file format wrapper for the output audio. The available options depend on the encoding type.. text `string`The text content to be converted to speech sample\_rate `number`Sample Rate specifies the sample rate for the output audio. Based on the encoding, different sample rates are supported. For some encodings, the sample rate is not configurable bit\_rate `number`The bitrate of the audio in bits per second. Choose from predefined ranges or specific values based on the encoding type. The binding returns a `ReadableStream` with the audio in MPEG format (check the model's output schema). ## API Schemas (Raw) Input [Open](https://developers.cloudflare.com/workers-ai/models/aura-2-es/schema-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/aura-2-es/schema-input.json) Output [Open](https://developers.cloudflare.com/workers-ai/models/aura-2-es/schema-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/aura-2-es/schema-output.json) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/models/aura-2-es/#page","headline":"aura-2-es","description":"Aura-2 is a context-aware text-to-speech (TTS) model that applies natural pacing, expressiveness, and fillers based on the context of the provided text. The quality of your text input directly impacts the naturalness of the audio output.","url":"https://developers.cloudflare.com/workers-ai/models/aura-2-es/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Flux is the first conversational speech recognition model built specifically for voice agents. title: flux image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/llms.txt > Use this file to discover all available pages before exploring further. ![Deepgram logo](https://developers.cloudflare.com/_astro/deepgram.BYzW8KfF.svg) # flux Automatic Speech Recognition • Deepgram Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/models/flux/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Model ID `@cf/deepgram/flux` Copy - Cloudflare-hosted - Partner - Real-time - Zero data retention Flux is the first conversational speech recognition model built specifically for voice agents. | Model Info | | | --- | --- | | Terms and License | [link ↗](https://deepgram.com/terms) | | Zero data retention | Yes | | Partner | Yes | | Real-time | Yes | | Unit Pricing | $0.0077 per audio minute (websocket) | ## Parameters encoding `string`enum: linear16Encoding of the audio stream. Currently only supports raw signed little-endian 16-bit PCM. sample\_rate `string`pattern: ^\[0-9]+$Sample rate of the audio stream in Hz. eager\_eot\_threshold `string`End-of-turn confidence required to fire an eager end-of-turn event. When set, enables EagerEndOfTurn and TurnResumed events. Valid Values 0.3 - 0.9. eot\_threshold `string`default: 0.7End-of-turn confidence required to finish a turn. Valid Values 0.5 - 0.9. eot\_timeout\_ms `string`default: 5000pattern: ^\[0-9]+$A turn will be finished when this much time has passed after speech, regardless of EOT confidence. keyterm `string`Keyterm prompting can improve recognition of specialized terminology. Pass multiple keyterm query parameters to boost multiple keyterms. mip\_opt\_out `string`default: falseenum: true, falseOpts out requests from the Deepgram Model Improvement Program. Refer to Deepgram Docs for pricing impacts before setting this to true. https://dpgr.am/deepgram-mip tag `string`Label your requests for the purpose of identification during usage reporting request\_id `string`The unique identifier of the request (uuid) sequence\_id `integer`minimum: 0Starts at 0 and increments for each message the server sends to the client. event `string`enum: Update, StartOfTurn, EagerEndOfTurn, TurnResumed, EndOfTurnThe type of event being reported. turn\_index `integer`minimum: 0The index of the current turn audio\_window\_start `number`Start time in seconds of the audio range that was transcribed audio\_window\_end `number`End time in seconds of the audio range that was transcribed transcript `string`Text that was said over the course of the current turn ▶words\[] `array`The words in the transcript end\_of\_turn\_confidence `number`Confidence that no more speech is coming in this turn ## API Schemas (Raw) Input [Open](https://developers.cloudflare.com/workers-ai/models/flux/schema-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/flux/schema-input.json) Output [Open](https://developers.cloudflare.com/workers-ai/models/flux/schema-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/flux/schema-output.json) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/models/flux/#page","headline":"flux","description":"Flux is the first conversational speech recognition model built specifically for voice agents.","url":"https://developers.cloudflare.com/workers-ai/models/flux/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Transcribe audio using Deepgram’s speech-to-text model title: nova-3 image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/llms.txt > Use this file to discover all available pages before exploring further. ![Deepgram logo](https://developers.cloudflare.com/_astro/deepgram.BYzW8KfF.svg) # nova-3 Automatic Speech Recognition • Deepgram Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/models/nova-3/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Model ID `@cf/deepgram/nova-3` Copy - Cloudflare-hosted - Batch - Partner - Real-time - Zero data retention Transcribe audio using Deepgram’s speech-to-text model | Model Info | | | --- | --- | | Terms and License | [link ↗](https://deepgram.com/terms) | | Zero data retention | Yes | | Batch | Yes | | Partner | Yes | | Real-time | Yes | | Unit Pricing | $0.0052 per audio minute, $0.0092 per audio minute (websocket) | Note The [pricing of this model](https://developers.cloudflare.com/workers-ai/platform/pricing) is different based on transport. Transport-based pricing does not apply to all models. - WebSocket: $0.0092 per audio minute output (836.36 neurons per audio minute output) - Regular HTTP: $0.0052 per audio minute output (472.73 neurons per audio minute output) ## Parameters ▶audio{} `object`required custom\_topic\_mode `string`enum: extended, strictSets how the model will interpret strings submitted to the custom\_topic param. When strict, the model will only return topics submitted using the custom\_topic param. When extended, the model will return its own detected topics in addition to those submitted using the custom\_topic param. custom\_topic `string`Custom topics you want the model to detect within your input audio or text if present Submit up to 100 custom\_intent\_mode `string`enum: extended, strictSets how the model will interpret intents submitted to the custom\_intent param. When strict, the model will only return intents submitted using the custom\_intent param. When extended, the model will return its own detected intents in addition those submitted using the custom\_intents param custom\_intent `string`Custom intents you want the model to detect within your input audio if present detect\_entities `boolean`Identifies and extracts key entities from content in submitted audio detect\_language `boolean`Identifies the dominant language spoken in submitted audio diarize `boolean`Recognize speaker changes. Each word in the transcript will be assigned a speaker number starting at 0 dictation `boolean`Identify and extract key entities from content in submitted audio encoding `string`enum: linear16, flac, mulaw, amr-nb, amr-wb, opus, speex, g729Specify the expected encoding of your submitted audio extra `string`Arbitrary key-value pairs that are attached to the API response for usage in downstream processing filler\_words `boolean`Filler Words can help transcribe interruptions in your audio, like 'uh' and 'um' keyterm `string`Key term prompting can boost or suppress specialized terminology and brands. keywords `string`Keywords can boost or suppress specialized terminology and brands. language `string`The BCP-47 language tag that hints at the primary spoken language. Depending on the Model and API endpoint you choose only certain languages are available. measurements `boolean`Spoken measurements will be converted to their corresponding abbreviations. mip\_opt\_out `boolean`Opts out requests from the Deepgram Model Improvement Program. Refer to our Docs for pricing impacts before setting this to true. https://dpgr.am/deepgram-mip. mode `string`enum: general, medical, financeMode of operation for the model representing broad area of topic that will be talked about in the supplied audio multichannel `boolean`Transcribe each audio channel independently. numerals `boolean`Numerals converts numbers from written format to numerical format. paragraphs `boolean`Splits audio into paragraphs to improve transcript readability. profanity\_filter `boolean`Profanity Filter looks for recognized profanity and converts it to the nearest recognized non-profane word or removes it from the transcript completely. punctuate `boolean`Add punctuation and capitalization to the transcript. redact `string`Redaction removes sensitive information from your transcripts. replace `string`Search for terms or phrases in submitted audio and replaces them. search `string`Search for terms or phrases in submitted audio. sentiment `boolean`Recognizes the sentiment throughout a transcript or text. smart\_format `boolean`Apply formatting to transcript output. When set to true, additional formatting will be applied to transcripts to improve readability. topics `boolean`Detect topics throughout a transcript or text. utterances `boolean`Segments speech into meaningful semantic units. utt\_split `number`Seconds to wait before detecting a pause between words in submitted audio. channels `number`The number of channels in the submitted audio interim\_results `boolean`Specifies whether the streaming endpoint should provide ongoing transcription updates as more audio is received. When set to true, the endpoint sends continuous updates, meaning transcription results may evolve over time. Note: Supported only for webosockets. endpointing `string`Indicates how long model will wait to detect whether a speaker has finished speaking or pauses for a significant period of time. When set to a value, the streaming endpoint immediately finalizes the transcription for the processed time range and returns the transcript with a speech\_final parameter set to true. Can also be set to false to disable endpointing vad\_events `boolean`Indicates that speech has started. You'll begin receiving Speech Started messages upon speech starting. Note: Supported only for webosockets. utterance\_end\_ms `boolean`Indicates how long model will wait to send an UtteranceEnd message after a word has been transcribed. Use with interim\_results. Note: Supported only for webosockets. ▶results{} `object` ## API Schemas (Raw) Input [Open](https://developers.cloudflare.com/workers-ai/models/nova-3/schema-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/nova-3/schema-input.json) Output [Open](https://developers.cloudflare.com/workers-ai/models/nova-3/schema-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/nova-3/schema-output.json) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/models/nova-3/#page","headline":"nova-3","description":"Transcribe audio using Deepgram’s speech-to-text model","url":"https://developers.cloudflare.com/workers-ai/models/nova-3/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: DeepSeek-R1-Distill-Qwen-32B is a model distilled from DeepSeek-R1 based on Qwen2.5. It outperforms OpenAI-o1-mini across various benchmarks, achieving new state-of-the-art results for dense models. title: deepseek-r1-distill-qwen-32b image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/llms.txt > Use this file to discover all available pages before exploring further. ![DeepSeek logo](https://developers.cloudflare.com/_astro/deepseek.CkzAgvN6.svg) # deepseek-r1-distill-qwen-32b Text Generation • DeepSeek Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/models/deepseek-r1-distill-qwen-32b/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Model ID `@cf/deepseek-ai/deepseek-r1-distill-qwen-32b` Copy - Cloudflare-hosted - Reasoning - Planned deprecation - Zero data retention DeepSeek-R1-Distill-Qwen-32B is a model distilled from DeepSeek-R1 based on Qwen2.5. It outperforms OpenAI-o1-mini across various benchmarks, achieving new state-of-the-art results for dense models. | Model Info | | | --- | --- | | Planned Deprecation | 11/20/2026 | | Context Window [ ↗](https://developers.cloudflare.com/workers-ai/platform/glossary/) | 80,000 tokens | | Terms and License | [link ↗](https://github.com/deepseek-ai/DeepSeek-R1/blob/main/LICENSE) | | Reasoning | Yes | | Zero data retention | Yes | | Unit Pricing | $0.497 per M input tokens, $4.881 per M output tokens | ## Playground Try out this model with Workers AI LLM Playground. It does not require any setup or authentication and is an instant way to preview and test a model directly in the browser. [Launch the LLM Playground](https://playground.ai.cloudflare.com/?model=@cf/deepseek-ai/deepseek-r1-distill-qwen-32b) ## Usage ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const messages = [ { role: "system", content: "You are a friendly assistant" }, { role: "user", content: "What is the origin of the phrase Hello, World", }, ]; const stream = await env.AI.run("@cf/deepseek-ai/deepseek-r1-distill-qwen-32b", { messages, stream: true, }); return new Response(stream, { headers: { "content-type": "text/event-stream" }, }); }, } satisfies ExportedHandler; ``` ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const messages = [ { role: "system", content: "You are a friendly assistant" }, { role: "user", content: "What is the origin of the phrase Hello, World", }, ]; const response = await env.AI.run("@cf/deepseek-ai/deepseek-r1-distill-qwen-32b", { messages }); return Response.json(response); }, } satisfies ExportedHandler; ``` ```py import os import requests ACCOUNT_ID = "your-account-id" AUTH_TOKEN = os.environ.get("CLOUDFLARE_AUTH_TOKEN") prompt = "Tell me all about PEP-8" response = requests.post( f"https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai/run/@cf/deepseek-ai/deepseek-r1-distill-qwen-32b", headers={"Authorization": f"Bearer {AUTH_TOKEN}"}, json={ "messages": [ {"role": "system", "content": "You are a friendly assistant"}, {"role": "user", "content": prompt} ] } ) result = response.json() print(result) ``` ```sh curl https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/run/@cf/deepseek-ai/deepseek-r1-distill-qwen-32b \ -X POST \ -H "Authorization: Bearer $CLOUDFLARE_AUTH_TOKEN" \ -d '{ "messages": [{ "role": "system", "content": "You are a friendly assistant" }, { "role": "user", "content": "Why is pizza so good" }]}' ``` OpenAI compatible endpoints Workers AI also supports OpenAI compatible API endpoints for `/v1/chat/completions` and `/v1/embeddings`. For more details, refer to [Configurations](https://developers.cloudflare.com/workers-ai/configuration/open-ai-compatibility/) . ## Parameters ### Input prompt `string`requiredminLength: 1The input text prompt for the model to generate a response. lora `string`Name of the LoRA (Low-Rank Adaptation) model to fine-tune the base model. ▶response\_format{} `object` raw `boolean`default: falseIf true, a chat template is not applied and you must adhere to the specific model's expected formatting. stream `boolean`default: falseIf true, the response will be streamed back incrementally using SSE, Server Sent Events. max\_tokens `integer`default: 256The maximum number of tokens to generate in the response. temperature `number`default: 0.6minimum: 0maximum: 5Controls the randomness of the output; higher values produce more random results. top\_p `number`minimum: 0.001maximum: 1Adjusts the creativity of the AI's responses by controlling how many possible words it considers. Lower values make outputs more predictable; higher values allow for more varied and creative responses. top\_k `integer`minimum: 1maximum: 50Limits the AI to choose from the top 'k' most probable words. Lower values make responses more focused; higher values introduce more variety and potential surprises. seed `integer`minimum: 1maximum: 9999999999Random seed for reproducibility of the generation. repetition\_penalty `number`minimum: 0maximum: 2Penalty for repeated tokens; higher values discourage repetition. frequency\_penalty `number`minimum: -2maximum: 2Decreases the likelihood of the model repeating the same lines verbatim. presence\_penalty `number`minimum: -2maximum: 2Increases the likelihood of the model introducing new topics. ### Output
Synchronous — Send a request and receive a complete response response stringThe generated text response from the model ▶usage{} objectUsage statistics for the inference request ▶tool\_calls\[] arrayAn array of tool calls requests made during the response generation
Streaming — Send a request with `stream: true` and receive server-sent events type string format binary
## API Schemas (Raw) SynchronousInput [Open](https://developers.cloudflare.com/workers-ai/models/deepseek-r1-distill-qwen-32b/sync-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/deepseek-r1-distill-qwen-32b/sync-input.json) SynchronousOutput [Open](https://developers.cloudflare.com/workers-ai/models/deepseek-r1-distill-qwen-32b/sync-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/deepseek-r1-distill-qwen-32b/sync-output.json) StreamingInput [Open](https://developers.cloudflare.com/workers-ai/models/deepseek-r1-distill-qwen-32b/streaming-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/deepseek-r1-distill-qwen-32b/streaming-input.json) StreamingOutput [Open](https://developers.cloudflare.com/workers-ai/models/deepseek-r1-distill-qwen-32b/streaming-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/deepseek-r1-distill-qwen-32b/streaming-output.json) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/models/deepseek-r1-distill-qwen-32b/#page","headline":"deepseek-r1-distill-qwen-32b","description":"DeepSeek-R1-Distill-Qwen-32B is a model distilled from DeepSeek-R1 based on Qwen2.5. It outperforms OpenAI-o1-mini across various benchmarks, achieving new state-of-the-art results for dense models.","url":"https://developers.cloudflare.com/workers-ai/models/deepseek-r1-distill-qwen-32b/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: "DeepSeek-V4-Flash-0731 is the official release of DeepSeek-V4-Flash, superseding the preview version, with substantially enhanced agentic capabilities. " title: deepseek-v4-flash-0731 image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/llms.txt > Use this file to discover all available pages before exploring further. ![DeepSeek logo](https://developers.cloudflare.com/_astro/deepseek.CkzAgvN6.svg) # deepseek-v4-flash-0731 Text Generation • DeepSeek Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/models/deepseek-v4-flash-0731/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Model ID `@cf/deepseek-ai/deepseek-v4-flash-0731` Copy - Cloudflare-hosted - Batch - Function calling - Reasoning - Planned deprecation - Zero data retention DeepSeek-V4-Flash-0731 is the official release of DeepSeek-V4-Flash, superseding the preview version, with substantially enhanced agentic capabilities. Paid access required This model is not available through standard Workers Free billing. To use it, upgrade to the [Workers Paid plan](https://developers.cloudflare.com/workers/platform/pricing/#workers) or use prepaid [AI Gateway credits](https://developers.cloudflare.com/ai-gateway/features/unified-billing/). | Model Info | | | --- | --- | | Planned Deprecation | 11/20/2026 | | Context Window [ ↗](https://developers.cloudflare.com/workers-ai/platform/glossary/) | 1,048,576 tokens | | Function calling [↗](https://developers.cloudflare.com/workers-ai/function-calling/) | Yes | | Reasoning | `none``low``high` (default)`max` | | Zero data retention | Yes | | Batch | Yes | | Unit Pricing | $0.44 per M input tokens, $1.32 per M output tokens, $0.014 per M cached input tokens | ## Playground Try out this model with Workers AI LLM Playground. It does not require any setup or authentication and is an instant way to preview and test a model directly in the browser. [Launch the LLM Playground](https://playground.ai.cloudflare.com/?model=@cf/deepseek-ai/deepseek-v4-flash-0731) ## Usage ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const messages = [ { role: "system", content: "You are a friendly assistant" }, { role: "user", content: "What is the origin of the phrase Hello, World", }, ]; const stream = await env.AI.run("@cf/deepseek-ai/deepseek-v4-flash-0731", { messages, stream: true, }); return new Response(stream, { headers: { "content-type": "text/event-stream" }, }); }, } satisfies ExportedHandler; ``` ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const messages = [ { role: "system", content: "You are a friendly assistant" }, { role: "user", content: "What is the origin of the phrase Hello, World", }, ]; const response = await env.AI.run("@cf/deepseek-ai/deepseek-v4-flash-0731", { messages }); return Response.json(response); }, } satisfies ExportedHandler; ``` ```py import os import requests ACCOUNT_ID = "your-account-id" AUTH_TOKEN = os.environ.get("CLOUDFLARE_AUTH_TOKEN") prompt = "Tell me all about PEP-8" response = requests.post( f"https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai/run/@cf/deepseek-ai/deepseek-v4-flash-0731", headers={"Authorization": f"Bearer {AUTH_TOKEN}"}, json={ "messages": [ {"role": "system", "content": "You are a friendly assistant"}, {"role": "user", "content": prompt} ] } ) result = response.json() print(result) ``` ```sh curl https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/run/@cf/deepseek-ai/deepseek-v4-flash-0731 \ -X POST \ -H "Authorization: Bearer $CLOUDFLARE_AUTH_TOKEN" \ -d '{ "messages": [{ "role": "system", "content": "You are a friendly assistant" }, { "role": "user", "content": "Why is pizza so good" }]}' ``` OpenAI compatible endpoints Workers AI also supports OpenAI compatible API endpoints for `/v1/chat/completions` and `/v1/embeddings`. For more details, refer to [Configurations](https://developers.cloudflare.com/workers-ai/configuration/open-ai-compatibility/) . ## Parameters
Synchronous — Send a request and receive a complete response Input format Prompt Simple text input for single-turn interactionsMessages Structured conversation format with roles (user, assistant, system) prompt stringrequiredminLength: 1The input text prompt for the model to generate a response. model stringID of the model to use (e.g. '@cf/zai-org/glm-4.7-flash, etc'). ▶audio{} objectParameters for audio output. Required when modalities includes 'audio'. frequency\_penalty number | nullPenalizes new tokens based on their existing frequency in the text so far. logit\_bias object | nullModify the likelihood of specified tokens appearing in the completion. Maps token IDs to bias values from -100 to 100. logprobs boolean | nullWhether to return log probabilities of the output tokens. top\_logprobs integer | nullHow many top log probabilities to return at each token position (0-20). Requires logprobs=true. max\_tokens integer | nullDeprecated in favor of max_completion_tokens. The maximum number of tokens to generate. max\_completion\_tokens integer | nullAn upper bound for the number of tokens that can be generated for a completion. metadata object | nullSet of 16 key-value pairs that can be attached to the object. modalities array | nullOutput types requested from the model (e.g. ['text'] or ['text', 'audio']). n integer | nullHow many chat completion choices to generate for each input message. parallel\_tool\_calls booleandefault: trueWhether to enable parallel function calling during tool use. ▶prediction{} object presence\_penalty number | nullPenalizes new tokens based on whether they appear in the text so far. reasoning\_effort string | nullenum: max, high, low, noneReasoning effort. Supported levels: max, high, low, none. Compatibility aliases: minimal maps to low; medium maps to high; xhigh maps to high. ▶chat\_template\_kwargs{} object ▶response\_format one ofSpecifies the format the model must output. seed integer | nullIf specified, the system will make a best effort to sample deterministically. ▶stop one of store boolean | nullWhether to store the output for model distillation / evals. stream boolean | nullIf true, partial message deltas will be sent as server-sent events. ▶stream\_options{} object temperature number | nullSampling temperature between 0 and 2. ▶tool\_choice one ofControls which (if any) tool is called by the model. 'none' = no tools, 'auto' = model decides, 'required' = must call a tool. ▶tools\[] arrayA list of tools the model may call. top\_p number | nullNucleus sampling: considers the results of the tokens with top_p probability mass. user stringA unique identifier representing your end-user, for abuse monitoring. ▶web\_search\_options{} objectOptions for the web search tool (when using built-in web search). ▶function\_call one of ▶functions\[] arrayminItems: 1maxItems: 128 id stringA unique identifier for the chat completion. object string created integerUnix timestamp (seconds) of when the completion was created. model stringThe model used for the chat completion. ▶choices\[] arrayminItems: 1 ▶usage{} object system\_fingerprint string | null
Streaming — Send a request with `stream: true` and receive server-sent events Input format Prompt Simple text input for single-turn interactionsMessages Structured conversation format with roles (user, assistant, system) prompt stringrequiredminLength: 1The input text prompt for the model to generate a response. model stringID of the model to use (e.g. '@cf/zai-org/glm-4.7-flash, etc'). ▶audio{} objectParameters for audio output. Required when modalities includes 'audio'. frequency\_penalty number | nullPenalizes new tokens based on their existing frequency in the text so far. logit\_bias object | nullModify the likelihood of specified tokens appearing in the completion. Maps token IDs to bias values from -100 to 100. logprobs boolean | nullWhether to return log probabilities of the output tokens. top\_logprobs integer | nullHow many top log probabilities to return at each token position (0-20). Requires logprobs=true. max\_tokens integer | nullDeprecated in favor of max_completion_tokens. The maximum number of tokens to generate. max\_completion\_tokens integer | nullAn upper bound for the number of tokens that can be generated for a completion. metadata object | nullSet of 16 key-value pairs that can be attached to the object. modalities array | nullOutput types requested from the model (e.g. ['text'] or ['text', 'audio']). n integer | nullHow many chat completion choices to generate for each input message. parallel\_tool\_calls booleandefault: trueWhether to enable parallel function calling during tool use. ▶prediction{} object presence\_penalty number | nullPenalizes new tokens based on whether they appear in the text so far. reasoning\_effort string | nullenum: max, high, low, noneReasoning effort. Supported levels: max, high, low, none. Compatibility aliases: minimal maps to low; medium maps to high; xhigh maps to high. ▶chat\_template\_kwargs{} object ▶response\_format one ofSpecifies the format the model must output. seed integer | nullIf specified, the system will make a best effort to sample deterministically. ▶stop one of store boolean | nullWhether to store the output for model distillation / evals. stream boolean | nullIf true, partial message deltas will be sent as server-sent events. ▶stream\_options{} object temperature number | nullSampling temperature between 0 and 2. ▶tool\_choice one ofControls which (if any) tool is called by the model. 'none' = no tools, 'auto' = model decides, 'required' = must call a tool. ▶tools\[] arrayA list of tools the model may call. top\_p number | nullNucleus sampling: considers the results of the tokens with top_p probability mass. user stringA unique identifier representing your end-user, for abuse monitoring. ▶web\_search\_options{} objectOptions for the web search tool (when using built-in web search). ▶function\_call one of ▶functions\[] arrayminItems: 1maxItems: 128 type string contentType text/event-stream format binary
Batch — Send multiple requests in a single API call ▶requests\[] array id stringA unique identifier for the chat completion. object string created integerUnix timestamp (seconds) of when the completion was created. model stringThe model used for the chat completion. ▶choices\[] arrayminItems: 1 ▶usage{} object system\_fingerprint string | null
## API Schemas (Raw) SynchronousInput [Open](https://developers.cloudflare.com/workers-ai/models/deepseek-v4-flash-0731/sync-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/deepseek-v4-flash-0731/sync-input.json) SynchronousOutput [Open](https://developers.cloudflare.com/workers-ai/models/deepseek-v4-flash-0731/sync-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/deepseek-v4-flash-0731/sync-output.json) StreamingInput [Open](https://developers.cloudflare.com/workers-ai/models/deepseek-v4-flash-0731/streaming-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/deepseek-v4-flash-0731/streaming-input.json) StreamingOutput [Open](https://developers.cloudflare.com/workers-ai/models/deepseek-v4-flash-0731/streaming-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/deepseek-v4-flash-0731/streaming-output.json) BatchInput [Open](https://developers.cloudflare.com/workers-ai/models/deepseek-v4-flash-0731/batch-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/deepseek-v4-flash-0731/batch-input.json) BatchOutput [Open](https://developers.cloudflare.com/workers-ai/models/deepseek-v4-flash-0731/batch-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/deepseek-v4-flash-0731/batch-output.json) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/models/deepseek-v4-flash-0731/#page","headline":"deepseek-v4-flash-0731","description":"DeepSeek-V4-Flash-0731 is the official release of DeepSeek-V4-Flash, superseding the preview version, with substantially enhanced agentic capabilities.","url":"https://developers.cloudflare.com/workers-ai/models/deepseek-v4-flash-0731/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: DeepSeek V4 Pro is a high-capability reasoning model from DeepSeek with a one million token context window, built for long-horizon agentic workflows and complex, multi-step problem-solving title: deepseek-v4-pro-0813 image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/llms.txt > Use this file to discover all available pages before exploring further. ![DeepSeek logo](https://developers.cloudflare.com/_astro/deepseek.CkzAgvN6.svg) # deepseek-v4-pro-0813 Text Generation • DeepSeek Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/models/deepseek-v4-pro-0813/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Model ID `@cf/deepseek-ai/deepseek-v4-pro-0813` Copy - Cloudflare-hosted - Function calling - Reasoning - Planned deprecation - Zero data retention DeepSeek V4 Pro is a high-capability reasoning model from DeepSeek with a one million token context window, built for long-horizon agentic workflows and complex, multi-step problem-solving Paid access required This model is not available through standard Workers Free billing. To use it, upgrade to the [Workers Paid plan](https://developers.cloudflare.com/workers/platform/pricing/#workers) or use prepaid [AI Gateway credits](https://developers.cloudflare.com/ai-gateway/features/unified-billing/). | Model Info | | | --- | --- | | Planned Deprecation | 11/20/2026 | | Context Window [ ↗](https://developers.cloudflare.com/workers-ai/platform/glossary/) | 1,048,576 tokens | | Function calling [↗](https://developers.cloudflare.com/workers-ai/function-calling/) | Yes | | Reasoning | `none``low``high` (default)`max` | | Zero data retention | Yes | | Unit Pricing | $1.32 per M input tokens, $3.96 per M output tokens, $0.044 per M cached input tokens | ## Playground Try out this model with Workers AI LLM Playground. It does not require any setup or authentication and is an instant way to preview and test a model directly in the browser. [Launch the LLM Playground](https://playground.ai.cloudflare.com/?model=@cf/deepseek-ai/deepseek-v4-pro-0813) ## Usage ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const messages = [ { role: "system", content: "You are a friendly assistant" }, { role: "user", content: "What is the origin of the phrase Hello, World", }, ]; const stream = await env.AI.run("@cf/deepseek-ai/deepseek-v4-pro-0813", { messages, stream: true, }); return new Response(stream, { headers: { "content-type": "text/event-stream" }, }); }, } satisfies ExportedHandler; ``` ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const messages = [ { role: "system", content: "You are a friendly assistant" }, { role: "user", content: "What is the origin of the phrase Hello, World", }, ]; const response = await env.AI.run("@cf/deepseek-ai/deepseek-v4-pro-0813", { messages }); return Response.json(response); }, } satisfies ExportedHandler; ``` ```py import os import requests ACCOUNT_ID = "your-account-id" AUTH_TOKEN = os.environ.get("CLOUDFLARE_AUTH_TOKEN") prompt = "Tell me all about PEP-8" response = requests.post( f"https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai/run/@cf/deepseek-ai/deepseek-v4-pro-0813", headers={"Authorization": f"Bearer {AUTH_TOKEN}"}, json={ "messages": [ {"role": "system", "content": "You are a friendly assistant"}, {"role": "user", "content": prompt} ] } ) result = response.json() print(result) ``` ```sh curl https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/run/@cf/deepseek-ai/deepseek-v4-pro-0813 \ -X POST \ -H "Authorization: Bearer $CLOUDFLARE_AUTH_TOKEN" \ -d '{ "messages": [{ "role": "system", "content": "You are a friendly assistant" }, { "role": "user", "content": "Why is pizza so good" }]}' ``` OpenAI compatible endpoints Workers AI also supports OpenAI compatible API endpoints for `/v1/chat/completions` and `/v1/embeddings`. For more details, refer to [Configurations](https://developers.cloudflare.com/workers-ai/configuration/open-ai-compatibility/) . ## Parameters
Synchronous — Send a request and receive a complete response Input format Prompt Simple text input for single-turn interactionsMessages Structured conversation format with roles (user, assistant, system) prompt stringrequiredminLength: 1The input text prompt for the model to generate a response. model stringID of the model to use (e.g. '@cf/zai-org/glm-4.7-flash, etc'). ▶audio{} objectParameters for audio output. Required when modalities includes 'audio'. frequency\_penalty number | nullPenalizes new tokens based on their existing frequency in the text so far. logit\_bias object | nullModify the likelihood of specified tokens appearing in the completion. Maps token IDs to bias values from -100 to 100. logprobs boolean | nullWhether to return log probabilities of the output tokens. top\_logprobs integer | nullHow many top log probabilities to return at each token position (0-20). Requires logprobs=true. max\_tokens integer | nullDeprecated in favor of max_completion_tokens. The maximum number of tokens to generate. max\_completion\_tokens integer | nullAn upper bound for the number of tokens that can be generated for a completion. metadata object | nullSet of 16 key-value pairs that can be attached to the object. modalities array | nullOutput types requested from the model (e.g. ['text'] or ['text', 'audio']). n integer | nullHow many chat completion choices to generate for each input message. parallel\_tool\_calls booleandefault: trueWhether to enable parallel function calling during tool use. ▶prediction{} object presence\_penalty number | nullPenalizes new tokens based on whether they appear in the text so far. reasoning\_effort string | nullenum: max, high, low, noneReasoning effort. Supported levels: max, high, low, none. Compatibility aliases: minimal maps to low; medium maps to high; xhigh maps to high. ▶chat\_template\_kwargs{} object ▶response\_format one ofSpecifies the format the model must output. seed integer | nullIf specified, the system will make a best effort to sample deterministically. ▶stop one of store boolean | nullWhether to store the output for model distillation / evals. stream boolean | nullIf true, partial message deltas will be sent as server-sent events. ▶stream\_options{} object temperature number | nullSampling temperature between 0 and 2. ▶tool\_choice one ofControls which (if any) tool is called by the model. 'none' = no tools, 'auto' = model decides, 'required' = must call a tool. ▶tools\[] arrayA list of tools the model may call. top\_p number | nullNucleus sampling: considers the results of the tokens with top_p probability mass. user stringA unique identifier representing your end-user, for abuse monitoring. ▶web\_search\_options{} objectOptions for the web search tool (when using built-in web search). ▶function\_call one of ▶functions\[] arrayminItems: 1maxItems: 128 id stringA unique identifier for the chat completion. object string created integerUnix timestamp (seconds) of when the completion was created. model stringThe model used for the chat completion. ▶choices\[] arrayminItems: 1 ▶usage{} object system\_fingerprint string | null
Streaming — Send a request with `stream: true` and receive server-sent events Input format Prompt Simple text input for single-turn interactionsMessages Structured conversation format with roles (user, assistant, system) prompt stringrequiredminLength: 1The input text prompt for the model to generate a response. model stringID of the model to use (e.g. '@cf/zai-org/glm-4.7-flash, etc'). ▶audio{} objectParameters for audio output. Required when modalities includes 'audio'. frequency\_penalty number | nullPenalizes new tokens based on their existing frequency in the text so far. logit\_bias object | nullModify the likelihood of specified tokens appearing in the completion. Maps token IDs to bias values from -100 to 100. logprobs boolean | nullWhether to return log probabilities of the output tokens. top\_logprobs integer | nullHow many top log probabilities to return at each token position (0-20). Requires logprobs=true. max\_tokens integer | nullDeprecated in favor of max_completion_tokens. The maximum number of tokens to generate. max\_completion\_tokens integer | nullAn upper bound for the number of tokens that can be generated for a completion. metadata object | nullSet of 16 key-value pairs that can be attached to the object. modalities array | nullOutput types requested from the model (e.g. ['text'] or ['text', 'audio']). n integer | nullHow many chat completion choices to generate for each input message. parallel\_tool\_calls booleandefault: trueWhether to enable parallel function calling during tool use. ▶prediction{} object presence\_penalty number | nullPenalizes new tokens based on whether they appear in the text so far. reasoning\_effort string | nullenum: max, high, low, noneReasoning effort. Supported levels: max, high, low, none. Compatibility aliases: minimal maps to low; medium maps to high; xhigh maps to high. ▶chat\_template\_kwargs{} object ▶response\_format one ofSpecifies the format the model must output. seed integer | nullIf specified, the system will make a best effort to sample deterministically. ▶stop one of store boolean | nullWhether to store the output for model distillation / evals. stream boolean | nullIf true, partial message deltas will be sent as server-sent events. ▶stream\_options{} object temperature number | nullSampling temperature between 0 and 2. ▶tool\_choice one ofControls which (if any) tool is called by the model. 'none' = no tools, 'auto' = model decides, 'required' = must call a tool. ▶tools\[] arrayA list of tools the model may call. top\_p number | nullNucleus sampling: considers the results of the tokens with top_p probability mass. user stringA unique identifier representing your end-user, for abuse monitoring. ▶web\_search\_options{} objectOptions for the web search tool (when using built-in web search). ▶function\_call one of ▶functions\[] arrayminItems: 1maxItems: 128 type string contentType text/event-stream format binary
Batch — Send multiple requests in a single API call ▶requests\[] array id stringA unique identifier for the chat completion. object string created integerUnix timestamp (seconds) of when the completion was created. model stringThe model used for the chat completion. ▶choices\[] arrayminItems: 1 ▶usage{} object system\_fingerprint string | null
## API Schemas (Raw) SynchronousInput [Open](https://developers.cloudflare.com/workers-ai/models/deepseek-v4-pro-0813/sync-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/deepseek-v4-pro-0813/sync-input.json) SynchronousOutput [Open](https://developers.cloudflare.com/workers-ai/models/deepseek-v4-pro-0813/sync-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/deepseek-v4-pro-0813/sync-output.json) StreamingInput [Open](https://developers.cloudflare.com/workers-ai/models/deepseek-v4-pro-0813/streaming-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/deepseek-v4-pro-0813/streaming-input.json) StreamingOutput [Open](https://developers.cloudflare.com/workers-ai/models/deepseek-v4-pro-0813/streaming-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/deepseek-v4-pro-0813/streaming-output.json) BatchInput [Open](https://developers.cloudflare.com/workers-ai/models/deepseek-v4-pro-0813/batch-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/deepseek-v4-pro-0813/batch-input.json) BatchOutput [Open](https://developers.cloudflare.com/workers-ai/models/deepseek-v4-pro-0813/batch-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/deepseek-v4-pro-0813/batch-output.json) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/models/deepseek-v4-pro-0813/#page","headline":"deepseek-v4-pro-0813","description":"DeepSeek V4 Pro is a high-capability reasoning model from DeepSeek with a one million token context window, built for long-horizon agentic workflows and complex, multi-step problem-solving","url":"https://developers.cloudflare.com/workers-ai/models/deepseek-v4-pro-0813/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: We introduce DeepSeek-V4.1-Flash, a multimodal Mixture-of-Experts (MoE) model with 552B backbone parameters and support for contexts of up to one million tokens. The model natively processes images and text, and generates text autoregressively. title: deepseek-v4.1-flash-1017 image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/llms.txt > Use this file to discover all available pages before exploring further. ![DeepSeek logo](https://developers.cloudflare.com/_astro/deepseek.CkzAgvN6.svg) # deepseek-v4.1-flash-1017 Text Generation • DeepSeek Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/models/deepseek-v4.1-flash-1017/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Model ID `@cf/deepseek-ai/deepseek-v4.1-flash-1017` Copy - Cloudflare-hosted - Function calling - Reasoning - Vision - Zero data retention We introduce DeepSeek-V4.1-Flash, a multimodal Mixture-of-Experts (MoE) model with 552B backbone parameters and support for contexts of up to one million tokens. The model natively processes images and text, and generates text autoregressively. Paid access required This model is not available through standard Workers Free billing. To use it, upgrade to the [Workers Paid plan](https://developers.cloudflare.com/workers/platform/pricing/#workers) or use prepaid [AI Gateway credits](https://developers.cloudflare.com/ai-gateway/features/unified-billing/). | Model Info | | | --- | --- | | Context Window [ ↗](https://developers.cloudflare.com/workers-ai/platform/glossary/) | 1,048,576 tokens | | Terms and License | [link ↗](https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash-1017/blob/main/LICENSE) | | Function calling [↗](https://developers.cloudflare.com/workers-ai/function-calling/) | Yes | | Reasoning | `none``low``high` (default)`max` | | Vision | Yes | | Zero data retention | Yes | | Unit Pricing | $0.30 per M input tokens, $1.20 per M output tokens, $0.006 per M cached input tokens | ## Playground Try out this model with Workers AI LLM Playground. It does not require any setup or authentication and is an instant way to preview and test a model directly in the browser. [Launch the LLM Playground](https://playground.ai.cloudflare.com/?model=@cf/deepseek-ai/deepseek-v4.1-flash-1017) ## Usage ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const messages = [ { role: "system", content: "You are a friendly assistant" }, { role: "user", content: "What is the origin of the phrase Hello, World", }, ]; const stream = await env.AI.run("@cf/deepseek-ai/deepseek-v4.1-flash-1017", { messages, stream: true, }); return new Response(stream, { headers: { "content-type": "text/event-stream" }, }); }, } satisfies ExportedHandler; ``` ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const messages = [ { role: "system", content: "You are a friendly assistant" }, { role: "user", content: "What is the origin of the phrase Hello, World", }, ]; const response = await env.AI.run("@cf/deepseek-ai/deepseek-v4.1-flash-1017", { messages }); return Response.json(response); }, } satisfies ExportedHandler; ``` ```py import os import requests ACCOUNT_ID = "your-account-id" AUTH_TOKEN = os.environ.get("CLOUDFLARE_AUTH_TOKEN") prompt = "Tell me all about PEP-8" response = requests.post( f"https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai/run/@cf/deepseek-ai/deepseek-v4.1-flash-1017", headers={"Authorization": f"Bearer {AUTH_TOKEN}"}, json={ "messages": [ {"role": "system", "content": "You are a friendly assistant"}, {"role": "user", "content": prompt} ] } ) result = response.json() print(result) ``` ```sh curl https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/run/@cf/deepseek-ai/deepseek-v4.1-flash-1017 \ -X POST \ -H "Authorization: Bearer $CLOUDFLARE_AUTH_TOKEN" \ -d '{ "messages": [{ "role": "system", "content": "You are a friendly assistant" }, { "role": "user", "content": "Why is pizza so good" }]}' ``` OpenAI compatible endpoints Workers AI also supports OpenAI compatible API endpoints for `/v1/chat/completions` and `/v1/embeddings`. For more details, refer to [Configurations](https://developers.cloudflare.com/workers-ai/configuration/open-ai-compatibility/) . ## Parameters
Synchronous — Send a request and receive a complete response Input format Prompt Simple text input for single-turn interactionsMessages Structured conversation format with roles (user, assistant, system) prompt stringrequiredminLength: 1The input text prompt for the model to generate a response. model stringID of the model to use (e.g. '@cf/zai-org/glm-4.7-flash, etc'). ▶audio{} objectParameters for audio output. Required when modalities includes 'audio'. frequency\_penalty number | nullPenalizes new tokens based on their existing frequency in the text so far. logit\_bias object | nullModify the likelihood of specified tokens appearing in the completion. Maps token IDs to bias values from -100 to 100. logprobs boolean | nullWhether to return log probabilities of the output tokens. top\_logprobs integer | nullHow many top log probabilities to return at each token position (0-20). Requires logprobs=true. max\_tokens integer | nullDeprecated in favor of max_completion_tokens. The maximum number of tokens to generate. max\_completion\_tokens integer | nullAn upper bound for the number of tokens that can be generated for a completion. metadata object | nullSet of 16 key-value pairs that can be attached to the object. modalities array | nullOutput types requested from the model (e.g. ['text'] or ['text', 'audio']). n integer | nullHow many chat completion choices to generate for each input message. parallel\_tool\_calls booleandefault: trueWhether to enable parallel function calling during tool use. ▶prediction{} object presence\_penalty number | nullPenalizes new tokens based on whether they appear in the text so far. reasoning\_effort string | nullenum: max, high, low, noneReasoning effort. Supported levels: max, high, low, none. Compatibility aliases: auto maps to high; minimal maps to none; medium maps to high; xhigh maps to max. ▶chat\_template\_kwargs{} object ▶response\_format one ofSpecifies the format the model must output. seed integer | nullIf specified, the system will make a best effort to sample deterministically. ▶stop one of store boolean | nullWhether to store the output for model distillation / evals. stream boolean | nullIf true, partial message deltas will be sent as server-sent events. ▶stream\_options{} object temperature number | nullSampling temperature between 0 and 2. ▶tool\_choice one ofControls which (if any) tool is called by the model. 'none' = no tools, 'auto' = model decides, 'required' = must call a tool. ▶tools\[] arrayA list of tools the model may call. top\_p number | nullNucleus sampling: considers the results of the tokens with top_p probability mass. user stringA unique identifier representing your end-user, for abuse monitoring. ▶web\_search\_options{} objectOptions for the web search tool (when using built-in web search). ▶function\_call one of ▶functions\[] arrayminItems: 1maxItems: 128 id stringA unique identifier for the chat completion. object string created integerUnix timestamp (seconds) of when the completion was created. model stringThe model used for the chat completion. ▶choices\[] arrayminItems: 1 ▶usage{} object system\_fingerprint string | null
Streaming — Send a request with `stream: true` and receive server-sent events Input format Prompt Simple text input for single-turn interactionsMessages Structured conversation format with roles (user, assistant, system) prompt stringrequiredminLength: 1The input text prompt for the model to generate a response. model stringID of the model to use (e.g. '@cf/zai-org/glm-4.7-flash, etc'). ▶audio{} objectParameters for audio output. Required when modalities includes 'audio'. frequency\_penalty number | nullPenalizes new tokens based on their existing frequency in the text so far. logit\_bias object | nullModify the likelihood of specified tokens appearing in the completion. Maps token IDs to bias values from -100 to 100. logprobs boolean | nullWhether to return log probabilities of the output tokens. top\_logprobs integer | nullHow many top log probabilities to return at each token position (0-20). Requires logprobs=true. max\_tokens integer | nullDeprecated in favor of max_completion_tokens. The maximum number of tokens to generate. max\_completion\_tokens integer | nullAn upper bound for the number of tokens that can be generated for a completion. metadata object | nullSet of 16 key-value pairs that can be attached to the object. modalities array | nullOutput types requested from the model (e.g. ['text'] or ['text', 'audio']). n integer | nullHow many chat completion choices to generate for each input message. parallel\_tool\_calls booleandefault: trueWhether to enable parallel function calling during tool use. ▶prediction{} object presence\_penalty number | nullPenalizes new tokens based on whether they appear in the text so far. reasoning\_effort string | nullenum: max, high, low, noneReasoning effort. Supported levels: max, high, low, none. Compatibility aliases: auto maps to high; minimal maps to none; medium maps to high; xhigh maps to max. ▶chat\_template\_kwargs{} object ▶response\_format one ofSpecifies the format the model must output. seed integer | nullIf specified, the system will make a best effort to sample deterministically. ▶stop one of store boolean | nullWhether to store the output for model distillation / evals. stream boolean | nullIf true, partial message deltas will be sent as server-sent events. ▶stream\_options{} object temperature number | nullSampling temperature between 0 and 2. ▶tool\_choice one ofControls which (if any) tool is called by the model. 'none' = no tools, 'auto' = model decides, 'required' = must call a tool. ▶tools\[] arrayA list of tools the model may call. top\_p number | nullNucleus sampling: considers the results of the tokens with top_p probability mass. user stringA unique identifier representing your end-user, for abuse monitoring. ▶web\_search\_options{} objectOptions for the web search tool (when using built-in web search). ▶function\_call one of ▶functions\[] arrayminItems: 1maxItems: 128 type string contentType text/event-stream format binary
Batch — Send multiple requests in a single API call ▶requests\[] array id stringA unique identifier for the chat completion. object string created integerUnix timestamp (seconds) of when the completion was created. model stringThe model used for the chat completion. ▶choices\[] arrayminItems: 1 ▶usage{} object system\_fingerprint string | null
## API Schemas (Raw) SynchronousInput [Open](https://developers.cloudflare.com/workers-ai/models/deepseek-v4.1-flash-1017/sync-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/deepseek-v4.1-flash-1017/sync-input.json) SynchronousOutput [Open](https://developers.cloudflare.com/workers-ai/models/deepseek-v4.1-flash-1017/sync-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/deepseek-v4.1-flash-1017/sync-output.json) StreamingInput [Open](https://developers.cloudflare.com/workers-ai/models/deepseek-v4.1-flash-1017/streaming-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/deepseek-v4.1-flash-1017/streaming-input.json) StreamingOutput [Open](https://developers.cloudflare.com/workers-ai/models/deepseek-v4.1-flash-1017/streaming-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/deepseek-v4.1-flash-1017/streaming-output.json) BatchInput [Open](https://developers.cloudflare.com/workers-ai/models/deepseek-v4.1-flash-1017/batch-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/deepseek-v4.1-flash-1017/batch-input.json) BatchOutput [Open](https://developers.cloudflare.com/workers-ai/models/deepseek-v4.1-flash-1017/batch-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/deepseek-v4.1-flash-1017/batch-output.json) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/models/deepseek-v4.1-flash-1017/#page","headline":"deepseek-v4.1-flash-1017","description":"We introduce DeepSeek-V4.1-Flash, a multimodal Mixture-of-Experts (MoE) model with 552B backbone parameters and support for contexts of up to one million tokens. The model natively processes images and text, and generates text autoregressively.","url":"https://developers.cloudflare.com/workers-ai/models/deepseek-v4.1-flash-1017/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: EmbeddingGemma is a 300M parameter, state-of-the-art for its size, open embedding model from Google, built from Gemma 3 (with T5Gemma initialization) and the same research and technology used to create Gemini models. EmbeddingGemma produces vector representations of text, making it well-suited for search and retrieval tasks, including classification, clustering, and semantic similarity search. This model was trained with data in 100+ spoken languages. title: embeddinggemma-300m image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/llms.txt > Use this file to discover all available pages before exploring further. ![Google logo](https://developers.cloudflare.com/_astro/google.DyXKPTPP.svg) # embeddinggemma-300m Beta Text Embeddings • Google Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/models/embeddinggemma-300m/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Model ID `@cf/google/embeddinggemma-300m` Copy - Cloudflare-hosted - Planned deprecation - Zero data retention EmbeddingGemma is a 300M parameter, state-of-the-art for its size, open embedding model from Google, built from Gemma 3 (with T5Gemma initialization) and the same research and technology used to create Gemini models. EmbeddingGemma produces vector representations of text, making it well-suited for search and retrieval tasks, including classification, clustering, and semantic similarity search. This model was trained with data in 100+ spoken languages. | Model Info | | | --- | --- | | Planned Deprecation | 11/20/2026 | | Zero data retention | Yes | | Beta | Yes | ## Usage ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { // Can be a string or array of strings] const stories = [ "This is a story about an orange cloud", "This is a story about a llama", "This is a story about a hugging emoji", ]; const embeddings = await env.AI.run( "@cf/google/embeddinggemma-300m", { text: stories, } ); return Response.json(embeddings); }, } satisfies ExportedHandler; ``` ```py import os import requests ACCOUNT_ID = "your-account-id" AUTH_TOKEN = os.environ.get("CLOUDFLARE_AUTH_TOKEN") stories = [ 'This is a story about an orange cloud', 'This is a story about a llama', 'This is a story about a hugging emoji' ] response = requests.post( f"https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai/run/@cf/google/embeddinggemma-300m", headers={"Authorization": f"Bearer {AUTH_TOKEN}"}, json={"text": stories} ) print(response.json()) ``` ```sh curl https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/run/@cf/google/embeddinggemma-300m \ -X POST \ -H "Authorization: Bearer $CLOUDFLARE_API_TOKEN" \ -d '{ "text": ["This is a story about an orange cloud", "This is a story about a llama", "This is a story about a hugging emoji"] }' ``` OpenAI compatible endpoints Workers AI also supports OpenAI compatible API endpoints for `/v1/chat/completions` and `/v1/embeddings`. For more details, refer to [Configurations](https://developers.cloudflare.com/workers-ai/configuration/open-ai-compatibility/). ## Parameters ▶text `one of`required ▶shape\[] `array` ▶data\[] `array`Embeddings of the requested text values ## API Schemas (Raw) Input [Open](https://developers.cloudflare.com/workers-ai/models/embeddinggemma-300m/schema-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/embeddinggemma-300m/schema-input.json) Output [Open](https://developers.cloudflare.com/workers-ai/models/embeddinggemma-300m/schema-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/embeddinggemma-300m/schema-output.json) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/models/embeddinggemma-300m/#page","headline":"embeddinggemma-300m","description":"EmbeddingGemma is a 300M parameter, state-of-the-art for its size, open embedding model from Google, built from Gemma 3 (with T5Gemma initialization) and the same research and technology used to create Gemini models. EmbeddingGemma produces vector representations of text, making it well-suited for search and retrieval tasks, including classification, clustering, and semantic similarity search. This model was trained with data in 100+ spoken languages.","url":"https://developers.cloudflare.com/workers-ai/models/embeddinggemma-300m/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: This is a Gemma-2B base model that Cloudflare dedicates for inference with LoRA adapters. Gemma is a family of lightweight, state-of-the-art open models from Google, built from the same research and technology used to create the Gemini models. title: gemma-2b-it-lora image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/llms.txt > Use this file to discover all available pages before exploring further. ![Google logo](https://developers.cloudflare.com/_astro/google.DyXKPTPP.svg) # gemma-2b-it-lora Beta Text Generation • Google Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/models/gemma-2b-it-lora/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Model ID `@cf/google/gemma-2b-it-lora` Copy - Cloudflare-hosted - LoRA - Planned deprecation - Zero data retention This is a Gemma-2B base model that Cloudflare dedicates for inference with LoRA adapters. Gemma is a family of lightweight, state-of-the-art open models from Google, built from the same research and technology used to create the Gemini models. | Model Info | | | --- | --- | | Planned Deprecation | 11/20/2026 | | Context Window [ ↗](https://developers.cloudflare.com/workers-ai/platform/glossary/) | 8,192 tokens | | Zero data retention | Yes | | LoRA | Yes | | Beta | Yes | ## Playground Try out this model with Workers AI LLM Playground. It does not require any setup or authentication and is an instant way to preview and test a model directly in the browser. [Launch the LLM Playground](https://playground.ai.cloudflare.com/?model=@cf/google/gemma-2b-it-lora) ## Usage ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const messages = [ { role: "system", content: "You are a friendly assistant" }, { role: "user", content: "What is the origin of the phrase Hello, World", }, ]; const stream = await env.AI.run("@cf/google/gemma-2b-it-lora", { messages, stream: true, }); return new Response(stream, { headers: { "content-type": "text/event-stream" }, }); }, } satisfies ExportedHandler; ``` ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const messages = [ { role: "system", content: "You are a friendly assistant" }, { role: "user", content: "What is the origin of the phrase Hello, World", }, ]; const response = await env.AI.run("@cf/google/gemma-2b-it-lora", { messages }); return Response.json(response); }, } satisfies ExportedHandler; ``` ```py import os import requests ACCOUNT_ID = "your-account-id" AUTH_TOKEN = os.environ.get("CLOUDFLARE_AUTH_TOKEN") prompt = "Tell me all about PEP-8" response = requests.post( f"https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai/run/@cf/google/gemma-2b-it-lora", headers={"Authorization": f"Bearer {AUTH_TOKEN}"}, json={ "messages": [ {"role": "system", "content": "You are a friendly assistant"}, {"role": "user", "content": prompt} ] } ) result = response.json() print(result) ``` ```sh curl https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/run/@cf/google/gemma-2b-it-lora \ -X POST \ -H "Authorization: Bearer $CLOUDFLARE_AUTH_TOKEN" \ -d '{ "messages": [{ "role": "system", "content": "You are a friendly assistant" }, { "role": "user", "content": "Why is pizza so good" }]}' ``` OpenAI compatible endpoints Workers AI also supports OpenAI compatible API endpoints for `/v1/chat/completions` and `/v1/embeddings`. For more details, refer to [Configurations](https://developers.cloudflare.com/workers-ai/configuration/open-ai-compatibility/) . ## Parameters ### Input prompt `string`requiredminLength: 1The input text prompt for the model to generate a response. lora `string`Name of the LoRA (Low-Rank Adaptation) model to fine-tune the base model. ▶response\_format{} `object` raw `boolean`default: falseIf true, a chat template is not applied and you must adhere to the specific model's expected formatting. stream `boolean`default: falseIf true, the response will be streamed back incrementally using SSE, Server Sent Events. max\_tokens `integer`default: 256The maximum number of tokens to generate in the response. temperature `number`default: 0.6minimum: 0maximum: 5Controls the randomness of the output; higher values produce more random results. top\_p `number`minimum: 0.001maximum: 1Adjusts the creativity of the AI's responses by controlling how many possible words it considers. Lower values make outputs more predictable; higher values allow for more varied and creative responses. top\_k `integer`minimum: 1maximum: 50Limits the AI to choose from the top 'k' most probable words. Lower values make responses more focused; higher values introduce more variety and potential surprises. seed `integer`minimum: 1maximum: 9999999999Random seed for reproducibility of the generation. repetition\_penalty `number`minimum: 0maximum: 2Penalty for repeated tokens; higher values discourage repetition. frequency\_penalty `number`minimum: -2maximum: 2Decreases the likelihood of the model repeating the same lines verbatim. presence\_penalty `number`minimum: -2maximum: 2Increases the likelihood of the model introducing new topics. ### Output
Synchronous — Send a request and receive a complete response response stringThe generated text response from the model ▶usage{} objectUsage statistics for the inference request ▶tool\_calls\[] arrayAn array of tool calls requests made during the response generation
Streaming — Send a request with `stream: true` and receive server-sent events type string format binary
## API Schemas (Raw) SynchronousInput [Open](https://developers.cloudflare.com/workers-ai/models/gemma-2b-it-lora/sync-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/gemma-2b-it-lora/sync-input.json) SynchronousOutput [Open](https://developers.cloudflare.com/workers-ai/models/gemma-2b-it-lora/sync-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/gemma-2b-it-lora/sync-output.json) StreamingInput [Open](https://developers.cloudflare.com/workers-ai/models/gemma-2b-it-lora/streaming-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/gemma-2b-it-lora/streaming-input.json) StreamingOutput [Open](https://developers.cloudflare.com/workers-ai/models/gemma-2b-it-lora/streaming-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/gemma-2b-it-lora/streaming-output.json) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/models/gemma-2b-it-lora/#page","headline":"gemma-2b-it-lora","description":"This is a Gemma-2B base model that Cloudflare dedicates for inference with LoRA adapters. Gemma is a family of lightweight, state-of-the-art open models from Google, built from the same research and technology used to create the Gemini models.","url":"https://developers.cloudflare.com/workers-ai/models/gemma-2b-it-lora/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Gemma 4 is Google's most intelligent family of open models, built from Gemini 3 research to maximize intelligence-per-parameter. title: gemma-4-26b-a4b-it image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/llms.txt > Use this file to discover all available pages before exploring further. ![Google logo](https://developers.cloudflare.com/_astro/google.DyXKPTPP.svg) # gemma-4-26b-a4b-it Text Generation • Google Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/models/gemma-4-26b-a4b-it/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Model ID `@cf/google/gemma-4-26b-a4b-it` Copy - Cloudflare-hosted - Batch - Function calling - Reasoning - Vision - Zero data retention Gemma 4 is Google's most intelligent family of open models, built from Gemini 3 research to maximize intelligence-per-parameter. | Model Info | | | --- | --- | | Context Window [ ↗](https://developers.cloudflare.com/workers-ai/platform/glossary/) | 256,000 tokens | | Terms and License | [link ↗](https://ai.google.dev/gemma/docs/gemma_4_license) | | Function calling [↗](https://developers.cloudflare.com/workers-ai/function-calling/) | Yes | | Reasoning | Yes | | Vision | Yes | | Zero data retention | Yes | | Batch | Yes | | Unit Pricing | $0.10 per M input tokens, $0.30 per M output tokens | ## Playground Try out this model with Workers AI LLM Playground. It does not require any setup or authentication and is an instant way to preview and test a model directly in the browser. [Launch the LLM Playground](https://playground.ai.cloudflare.com/?model=@cf/google/gemma-4-26b-a4b-it) ## Usage ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const messages = [ { role: "system", content: "You are a friendly assistant" }, { role: "user", content: "What is the origin of the phrase Hello, World", }, ]; const stream = await env.AI.run("@cf/google/gemma-4-26b-a4b-it", { messages, stream: true, }); return new Response(stream, { headers: { "content-type": "text/event-stream" }, }); }, } satisfies ExportedHandler; ``` ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const messages = [ { role: "system", content: "You are a friendly assistant" }, { role: "user", content: "What is the origin of the phrase Hello, World", }, ]; const response = await env.AI.run("@cf/google/gemma-4-26b-a4b-it", { messages }); return Response.json(response); }, } satisfies ExportedHandler; ``` ```py import os import requests ACCOUNT_ID = "your-account-id" AUTH_TOKEN = os.environ.get("CLOUDFLARE_AUTH_TOKEN") prompt = "Tell me all about PEP-8" response = requests.post( f"https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai/run/@cf/google/gemma-4-26b-a4b-it", headers={"Authorization": f"Bearer {AUTH_TOKEN}"}, json={ "messages": [ {"role": "system", "content": "You are a friendly assistant"}, {"role": "user", "content": prompt} ] } ) result = response.json() print(result) ``` ```sh curl https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/run/@cf/google/gemma-4-26b-a4b-it \ -X POST \ -H "Authorization: Bearer $CLOUDFLARE_AUTH_TOKEN" \ -d '{ "messages": [{ "role": "system", "content": "You are a friendly assistant" }, { "role": "user", "content": "Why is pizza so good" }]}' ``` OpenAI compatible endpoints Workers AI also supports OpenAI compatible API endpoints for `/v1/chat/completions` and `/v1/embeddings`. For more details, refer to [Configurations](https://developers.cloudflare.com/workers-ai/configuration/open-ai-compatibility/) . ## Parameters
Synchronous — Send a request and receive a complete response Input format Prompt Simple text input for single-turn interactionsMessages Structured conversation format with roles (user, assistant, system) prompt stringrequiredminLength: 1The input text prompt for the model to generate a response. skip\_special\_tokens booleandefault: false model stringID of the model to use (e.g. '@cf/zai-org/glm-4.7-flash, etc'). ▶audio{} objectParameters for audio output. Required when modalities includes 'audio'. frequency\_penalty number | nullPenalizes new tokens based on their existing frequency in the text so far. logit\_bias object | nullModify the likelihood of specified tokens appearing in the completion. Maps token IDs to bias values from -100 to 100. logprobs boolean | nullWhether to return log probabilities of the output tokens. top\_logprobs integer | nullHow many top log probabilities to return at each token position (0-20). Requires logprobs=true. max\_tokens integer | nullDeprecated in favor of max_completion_tokens. The maximum number of tokens to generate. max\_completion\_tokens integer | nullAn upper bound for the number of tokens that can be generated for a completion. metadata object | nullSet of 16 key-value pairs that can be attached to the object. modalities array | nullOutput types requested from the model (e.g. ['text'] or ['text', 'audio']). n integer | nullHow many chat completion choices to generate for each input message. parallel\_tool\_calls booleandefault: trueWhether to enable parallel function calling during tool use. ▶prediction{} object presence\_penalty number | nullPenalizes new tokens based on whether they appear in the text so far. ▶chat\_template\_kwargs{} object ▶response\_format one ofSpecifies the format the model must output. seed integer | nullIf specified, the system will make a best effort to sample deterministically. ▶stop one of store boolean | nullWhether to store the output for model distillation / evals. stream boolean | nullIf true, partial message deltas will be sent as server-sent events. ▶stream\_options{} object temperature number | nullSampling temperature between 0 and 2. ▶tool\_choice one ofControls which (if any) tool is called by the model. 'none' = no tools, 'auto' = model decides, 'required' = must call a tool. ▶tools\[] arrayA list of tools the model may call. top\_p number | nullNucleus sampling: considers the results of the tokens with top_p probability mass. user stringA unique identifier representing your end-user, for abuse monitoring. ▶web\_search\_options{} objectOptions for the web search tool (when using built-in web search). ▶function\_call one of ▶functions\[] arrayminItems: 1maxItems: 128 id stringA unique identifier for the chat completion. object string created integerUnix timestamp (seconds) of when the completion was created. model stringThe model used for the chat completion. ▶choices\[] arrayminItems: 1 ▶usage{} object system\_fingerprint string | null
Streaming — Send a request with `stream: true` and receive server-sent events Input format Prompt Simple text input for single-turn interactionsMessages Structured conversation format with roles (user, assistant, system) prompt stringrequiredminLength: 1The input text prompt for the model to generate a response. skip\_special\_tokens booleandefault: false model stringID of the model to use (e.g. '@cf/zai-org/glm-4.7-flash, etc'). ▶audio{} objectParameters for audio output. Required when modalities includes 'audio'. frequency\_penalty number | nullPenalizes new tokens based on their existing frequency in the text so far. logit\_bias object | nullModify the likelihood of specified tokens appearing in the completion. Maps token IDs to bias values from -100 to 100. logprobs boolean | nullWhether to return log probabilities of the output tokens. top\_logprobs integer | nullHow many top log probabilities to return at each token position (0-20). Requires logprobs=true. max\_tokens integer | nullDeprecated in favor of max_completion_tokens. The maximum number of tokens to generate. max\_completion\_tokens integer | nullAn upper bound for the number of tokens that can be generated for a completion. metadata object | nullSet of 16 key-value pairs that can be attached to the object. modalities array | nullOutput types requested from the model (e.g. ['text'] or ['text', 'audio']). n integer | nullHow many chat completion choices to generate for each input message. parallel\_tool\_calls booleandefault: trueWhether to enable parallel function calling during tool use. ▶prediction{} object presence\_penalty number | nullPenalizes new tokens based on whether they appear in the text so far. ▶chat\_template\_kwargs{} object ▶response\_format one ofSpecifies the format the model must output. seed integer | nullIf specified, the system will make a best effort to sample deterministically. ▶stop one of store boolean | nullWhether to store the output for model distillation / evals. stream boolean | nullIf true, partial message deltas will be sent as server-sent events. ▶stream\_options{} object temperature number | nullSampling temperature between 0 and 2. ▶tool\_choice one ofControls which (if any) tool is called by the model. 'none' = no tools, 'auto' = model decides, 'required' = must call a tool. ▶tools\[] arrayA list of tools the model may call. top\_p number | nullNucleus sampling: considers the results of the tokens with top_p probability mass. user stringA unique identifier representing your end-user, for abuse monitoring. ▶web\_search\_options{} objectOptions for the web search tool (when using built-in web search). ▶function\_call one of ▶functions\[] arrayminItems: 1maxItems: 128 type string contentType text/event-stream format binary
Batch — Send multiple requests in a single API call ▶requests\[] array id stringA unique identifier for the chat completion. object string created integerUnix timestamp (seconds) of when the completion was created. model stringThe model used for the chat completion. ▶choices\[] arrayminItems: 1 ▶usage{} object system\_fingerprint string | null
## API Schemas (Raw) SynchronousInput [Open](https://developers.cloudflare.com/workers-ai/models/gemma-4-26b-a4b-it/sync-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/gemma-4-26b-a4b-it/sync-input.json) SynchronousOutput [Open](https://developers.cloudflare.com/workers-ai/models/gemma-4-26b-a4b-it/sync-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/gemma-4-26b-a4b-it/sync-output.json) StreamingInput [Open](https://developers.cloudflare.com/workers-ai/models/gemma-4-26b-a4b-it/streaming-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/gemma-4-26b-a4b-it/streaming-input.json) StreamingOutput [Open](https://developers.cloudflare.com/workers-ai/models/gemma-4-26b-a4b-it/streaming-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/gemma-4-26b-a4b-it/streaming-output.json) BatchInput [Open](https://developers.cloudflare.com/workers-ai/models/gemma-4-26b-a4b-it/batch-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/gemma-4-26b-a4b-it/batch-input.json) BatchOutput [Open](https://developers.cloudflare.com/workers-ai/models/gemma-4-26b-a4b-it/batch-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/gemma-4-26b-a4b-it/batch-output.json) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/models/gemma-4-26b-a4b-it/#page","headline":"gemma-4-26b-a4b-it","description":"Gemma 4 is Google's most intelligent family of open models, built from Gemini 3 research to maximize intelligence-per-parameter.","url":"https://developers.cloudflare.com/workers-ai/models/gemma-4-26b-a4b-it/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: " This is a Gemma-7B base model that Cloudflare dedicates for inference with LoRA adapters. Gemma is a family of lightweight, state-of-the-art open models from Google, built from the same research and technology used to create the Gemini models." title: gemma-7b-it-lora image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/llms.txt > Use this file to discover all available pages before exploring further. ![Google logo](https://developers.cloudflare.com/_astro/google.DyXKPTPP.svg) # gemma-7b-it-lora Beta Text Generation • Google Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/models/gemma-7b-it-lora/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Model ID `@cf/google/gemma-7b-it-lora` Copy - Cloudflare-hosted - LoRA - Planned deprecation - Zero data retention This is a Gemma-7B base model that Cloudflare dedicates for inference with LoRA adapters. Gemma is a family of lightweight, state-of-the-art open models from Google, built from the same research and technology used to create the Gemini models. | Model Info | | | --- | --- | | Planned Deprecation | 11/20/2026 | | Context Window [ ↗](https://developers.cloudflare.com/workers-ai/platform/glossary/) | 3,500 tokens | | Zero data retention | Yes | | LoRA | Yes | | Beta | Yes | ## Playground Try out this model with Workers AI LLM Playground. It does not require any setup or authentication and is an instant way to preview and test a model directly in the browser. [Launch the LLM Playground](https://playground.ai.cloudflare.com/?model=@cf/google/gemma-7b-it-lora) ## Usage ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const messages = [ { role: "system", content: "You are a friendly assistant" }, { role: "user", content: "What is the origin of the phrase Hello, World", }, ]; const stream = await env.AI.run("@cf/google/gemma-7b-it-lora", { messages, stream: true, }); return new Response(stream, { headers: { "content-type": "text/event-stream" }, }); }, } satisfies ExportedHandler; ``` ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const messages = [ { role: "system", content: "You are a friendly assistant" }, { role: "user", content: "What is the origin of the phrase Hello, World", }, ]; const response = await env.AI.run("@cf/google/gemma-7b-it-lora", { messages }); return Response.json(response); }, } satisfies ExportedHandler; ``` ```py import os import requests ACCOUNT_ID = "your-account-id" AUTH_TOKEN = os.environ.get("CLOUDFLARE_AUTH_TOKEN") prompt = "Tell me all about PEP-8" response = requests.post( f"https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai/run/@cf/google/gemma-7b-it-lora", headers={"Authorization": f"Bearer {AUTH_TOKEN}"}, json={ "messages": [ {"role": "system", "content": "You are a friendly assistant"}, {"role": "user", "content": prompt} ] } ) result = response.json() print(result) ``` ```sh curl https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/run/@cf/google/gemma-7b-it-lora \ -X POST \ -H "Authorization: Bearer $CLOUDFLARE_AUTH_TOKEN" \ -d '{ "messages": [{ "role": "system", "content": "You are a friendly assistant" }, { "role": "user", "content": "Why is pizza so good" }]}' ``` OpenAI compatible endpoints Workers AI also supports OpenAI compatible API endpoints for `/v1/chat/completions` and `/v1/embeddings`. For more details, refer to [Configurations](https://developers.cloudflare.com/workers-ai/configuration/open-ai-compatibility/) . ## Parameters ### Input prompt `string`requiredminLength: 1The input text prompt for the model to generate a response. lora `string`Name of the LoRA (Low-Rank Adaptation) model to fine-tune the base model. ▶response\_format{} `object` raw `boolean`default: falseIf true, a chat template is not applied and you must adhere to the specific model's expected formatting. stream `boolean`default: falseIf true, the response will be streamed back incrementally using SSE, Server Sent Events. max\_tokens `integer`default: 256The maximum number of tokens to generate in the response. temperature `number`default: 0.6minimum: 0maximum: 5Controls the randomness of the output; higher values produce more random results. top\_p `number`minimum: 0.001maximum: 1Adjusts the creativity of the AI's responses by controlling how many possible words it considers. Lower values make outputs more predictable; higher values allow for more varied and creative responses. top\_k `integer`minimum: 1maximum: 50Limits the AI to choose from the top 'k' most probable words. Lower values make responses more focused; higher values introduce more variety and potential surprises. seed `integer`minimum: 1maximum: 9999999999Random seed for reproducibility of the generation. repetition\_penalty `number`minimum: 0maximum: 2Penalty for repeated tokens; higher values discourage repetition. frequency\_penalty `number`minimum: -2maximum: 2Decreases the likelihood of the model repeating the same lines verbatim. presence\_penalty `number`minimum: -2maximum: 2Increases the likelihood of the model introducing new topics. ### Output
Synchronous — Send a request and receive a complete response response stringThe generated text response from the model ▶usage{} objectUsage statistics for the inference request ▶tool\_calls\[] arrayAn array of tool calls requests made during the response generation
Streaming — Send a request with `stream: true` and receive server-sent events type string format binary
## API Schemas (Raw) SynchronousInput [Open](https://developers.cloudflare.com/workers-ai/models/gemma-7b-it-lora/sync-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/gemma-7b-it-lora/sync-input.json) SynchronousOutput [Open](https://developers.cloudflare.com/workers-ai/models/gemma-7b-it-lora/sync-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/gemma-7b-it-lora/sync-output.json) StreamingInput [Open](https://developers.cloudflare.com/workers-ai/models/gemma-7b-it-lora/streaming-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/gemma-7b-it-lora/streaming-input.json) StreamingOutput [Open](https://developers.cloudflare.com/workers-ai/models/gemma-7b-it-lora/streaming-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/gemma-7b-it-lora/streaming-output.json) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/models/gemma-7b-it-lora/#page","headline":"gemma-7b-it-lora","description":"This is a Gemma-7B base model that Cloudflare dedicates for inference with LoRA adapters. Gemma is a family of lightweight, state-of-the-art open models from Google, built from the same research and technology used to create the Gemini models.","url":"https://developers.cloudflare.com/workers-ai/models/gemma-7b-it-lora/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Distilled BERT model that was finetuned on SST-2 for sentiment classification title: distilbert-sst-2-int8 image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/llms.txt > Use this file to discover all available pages before exploring further. ![HuggingFace logo](https://developers.cloudflare.com/_astro/huggingface.DMS-v5TA.svg) # distilbert-sst-2-int8 Text Classification • HuggingFace Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/models/distilbert-sst-2-int8/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Model ID `@cf/huggingface/distilbert-sst-2-int8` Copy - Cloudflare-hosted - Zero data retention Distilled BERT model that was finetuned on SST-2 for sentiment classification | Model Info | | | --- | --- | | More information | [link ↗](https://huggingface.co/Intel/distilbert-base-uncased-finetuned-sst-2-english-int8-static) | | Zero data retention | Yes | | Unit Pricing | $0.0263 per M input tokens | ## Usage ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const response = await env.AI.run( "@cf/huggingface/distilbert-sst-2-int8", { text: "This pizza is great!", } ); return Response.json(response); }, } satisfies ExportedHandler; ``` ```py API_BASE_URL = "https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai/run/" headers = {"Authorization": "Bearer {API_KEY}"} def run(model, input): response = requests.post(f"{API_BASE_URL}{model}", headers=headers, json=input) return response.json() output = run("@cf/huggingface/distilbert-sst-2-int8", { "text": "This pizza is great!" }) print(output) ``` ```sh curl https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/run/@cf/huggingface/distilbert-sst-2-int8 \ -X POST \ -H "Authorization: Bearer $CLOUDFLARE_API_TOKEN" \ -d '{ "text": "This pizza is great!" }' ``` ## Parameters text `string`requiredminLength: 1The text that you want to classify type `array` contentType `application/json` description `An array of classification results for the input text` items `[object Object]` ## API Schemas (Raw) Input [Open](https://developers.cloudflare.com/workers-ai/models/distilbert-sst-2-int8/schema-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/distilbert-sst-2-int8/schema-input.json) Output [Open](https://developers.cloudflare.com/workers-ai/models/distilbert-sst-2-int8/schema-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/distilbert-sst-2-int8/schema-output.json) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/models/distilbert-sst-2-int8/#page","headline":"distilbert-sst-2-int8","description":"Distilled BERT model that was finetuned on SST-2 for sentiment classification","url":"https://developers.cloudflare.com/workers-ai/models/distilbert-sst-2-int8/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Granite 4.0 instruct models deliver strong performance across benchmarks, achieving industry-leading results in key agentic tasks like instruction following and function calling. These efficiencies make the models well-suited for a wide range of use cases like retrieval-augmented generation (RAG), multi-agent workflows, and edge deployments. title: granite-4.0-h-micro image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/llms.txt > Use this file to discover all available pages before exploring further. ![IBM logo](https://developers.cloudflare.com/_astro/ibm.CYdLt4EI.svg) # granite-4.0-h-micro Text Generation • IBM Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/models/granite-4.0-h-micro/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Model ID `@cf/ibm-granite/granite-4.0-h-micro` Copy - Cloudflare-hosted - Function calling - Zero data retention Granite 4.0 instruct models deliver strong performance across benchmarks, achieving industry-leading results in key agentic tasks like instruction following and function calling. These efficiencies make the models well-suited for a wide range of use cases like retrieval-augmented generation (RAG), multi-agent workflows, and edge deployments. | Model Info | | | --- | --- | | Context Window [ ↗](https://developers.cloudflare.com/workers-ai/platform/glossary/) | 131,000 tokens | | Function calling [↗](https://developers.cloudflare.com/workers-ai/function-calling/) | Yes | | Zero data retention | Yes | | Unit Pricing | $0.017 per M input tokens, $0.112 per M output tokens | ## Playground Try out this model with Workers AI LLM Playground. It does not require any setup or authentication and is an instant way to preview and test a model directly in the browser. [Launch the LLM Playground](https://playground.ai.cloudflare.com/?model=@cf/ibm-granite/granite-4.0-h-micro) ## Usage ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const messages = [ { role: "system", content: "You are a friendly assistant" }, { role: "user", content: "What is the origin of the phrase Hello, World", }, ]; const stream = await env.AI.run("@cf/ibm-granite/granite-4.0-h-micro", { messages, stream: true, }); return new Response(stream, { headers: { "content-type": "text/event-stream" }, }); }, } satisfies ExportedHandler; ``` ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const messages = [ { role: "system", content: "You are a friendly assistant" }, { role: "user", content: "What is the origin of the phrase Hello, World", }, ]; const response = await env.AI.run("@cf/ibm-granite/granite-4.0-h-micro", { messages }); return Response.json(response); }, } satisfies ExportedHandler; ``` ```py import os import requests ACCOUNT_ID = "your-account-id" AUTH_TOKEN = os.environ.get("CLOUDFLARE_AUTH_TOKEN") prompt = "Tell me all about PEP-8" response = requests.post( f"https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai/run/@cf/ibm-granite/granite-4.0-h-micro", headers={"Authorization": f"Bearer {AUTH_TOKEN}"}, json={ "messages": [ {"role": "system", "content": "You are a friendly assistant"}, {"role": "user", "content": prompt} ] } ) result = response.json() print(result) ``` ```sh curl https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/run/@cf/ibm-granite/granite-4.0-h-micro \ -X POST \ -H "Authorization: Bearer $CLOUDFLARE_AUTH_TOKEN" \ -d '{ "messages": [{ "role": "system", "content": "You are a friendly assistant" }, { "role": "user", "content": "Why is pizza so good" }]}' ``` OpenAI compatible endpoints Workers AI also supports OpenAI compatible API endpoints for `/v1/chat/completions` and `/v1/embeddings`. For more details, refer to [Configurations](https://developers.cloudflare.com/workers-ai/configuration/open-ai-compatibility/) . ## Parameters ### Input prompt `string`requiredminLength: 1The input text prompt for the model to generate a response. lora `string`Name of the LoRA (Low-Rank Adaptation) model to fine-tune the base model. ▶response\_format{} `object` raw `boolean`default: falseIf true, a chat template is not applied and you must adhere to the specific model's expected formatting. stream `boolean`default: falseIf true, the response will be streamed back incrementally using SSE, Server Sent Events. max\_tokens `integer`default: 256The maximum number of tokens to generate in the response. temperature `number`default: 0.6minimum: 0maximum: 5Controls the randomness of the output; higher values produce more random results. top\_p `number`minimum: 0.001maximum: 1Adjusts the creativity of the AI's responses by controlling how many possible words it considers. Lower values make outputs more predictable; higher values allow for more varied and creative responses. top\_k `integer`minimum: 1maximum: 50Limits the AI to choose from the top 'k' most probable words. Lower values make responses more focused; higher values introduce more variety and potential surprises. seed `integer`minimum: 1maximum: 9999999999Random seed for reproducibility of the generation. repetition\_penalty `number`minimum: 0maximum: 2Penalty for repeated tokens; higher values discourage repetition. frequency\_penalty `number`minimum: -2maximum: 2Decreases the likelihood of the model repeating the same lines verbatim. presence\_penalty `number`minimum: -2maximum: 2Increases the likelihood of the model introducing new topics. ### Output
Synchronous — Send a request and receive a complete response response stringThe generated text response from the model ▶usage{} objectUsage statistics for the inference request ▶tool\_calls\[] arrayAn array of tool calls requests made during the response generation
Streaming — Send a request with `stream: true` and receive server-sent events type string format binary
## API Schemas (Raw) SynchronousInput [Open](https://developers.cloudflare.com/workers-ai/models/granite-4.0-h-micro/sync-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/granite-4.0-h-micro/sync-input.json) SynchronousOutput [Open](https://developers.cloudflare.com/workers-ai/models/granite-4.0-h-micro/sync-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/granite-4.0-h-micro/sync-output.json) StreamingInput [Open](https://developers.cloudflare.com/workers-ai/models/granite-4.0-h-micro/streaming-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/granite-4.0-h-micro/streaming-input.json) StreamingOutput [Open](https://developers.cloudflare.com/workers-ai/models/granite-4.0-h-micro/streaming-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/granite-4.0-h-micro/streaming-output.json) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/models/granite-4.0-h-micro/#page","headline":"granite-4.0-h-micro","description":"Granite 4.0 instruct models deliver strong performance across benchmarks, achieving industry-leading results in key agentic tasks like instruction following and function calling. These efficiencies make the models well-suited for a wide range of use cases like retrieval-augmented generation (RAG), multi-agent workflows, and edge deployments.","url":"https://developers.cloudflare.com/workers-ai/models/granite-4.0-h-micro/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: "Lucid Origin from Leonardo.AI is their most adaptable and prompt-responsive model to date. Whether you're generating images with sharp graphic design, stunning full-HD renders, or highly specific creative direction, it adheres closely to your prompts, renders text with accuracy, and supports a wide array of visual styles and aesthetics – from stylized concept art to crisp product mockups.\n" title: lucid-origin image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/llms.txt > Use this file to discover all available pages before exploring further. ![Leonardo logo](https://developers.cloudflare.com/_astro/leonardo.JZysY-g3.svg) # lucid-origin Text-to-Image • Leonardo Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/models/lucid-origin/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Model ID `@cf/leonardo/lucid-origin` Copy - Cloudflare-hosted - Partner - Zero data retention Lucid Origin from Leonardo.AI is their most adaptable and prompt-responsive model to date. Whether you're generating images with sharp graphic design, stunning full-HD renders, or highly specific creative direction, it adheres closely to your prompts, renders text with accuracy, and supports a wide array of visual styles and aesthetics – from stylized concept art to crisp product mockups. | Model Info | | | --- | --- | | Terms and License | [link ↗](https://leonardo.ai/terms-of-service/) | | Zero data retention | Yes | | Partner | Yes | | Unit Pricing | $0.007 per 512 by 512 tile, $0.000132 per step | ## Parameters prompt `string`requiredminLength: 1A text description of the image you want to generate. guidance `number`default: 4.5minimum: 0maximum: 10Controls how closely the generated image should adhere to the prompt; higher values make the image more aligned with the prompt seed `integer`minimum: 0Random seed for reproducibility of the image generation height `integer`default: 1120minimum: 0maximum: 2500The height of the generated image in pixels width `integer`default: 1120minimum: 0maximum: 2500The width of the generated image in pixels num\_steps `integer`minimum: 1maximum: 40The number of diffusion steps; higher values can improve quality but take longer steps `integer`minimum: 1maximum: 40The number of diffusion steps; higher values can improve quality but take longer image `string`The generated image in Base64 format. ## API Schemas (Raw) Input [Open](https://developers.cloudflare.com/workers-ai/models/lucid-origin/schema-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/lucid-origin/schema-input.json) Output [Open](https://developers.cloudflare.com/workers-ai/models/lucid-origin/schema-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/lucid-origin/schema-output.json) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/models/lucid-origin/#page","headline":"lucid-origin","description":"Lucid Origin from Leonardo.AI is their most adaptable and prompt-responsive model to date. Whether you're generating images with sharp graphic design, stunning full-HD renders, or highly specific creative direction, it adheres closely to your prompts, renders text with accuracy, and supports a wide array of visual styles and aesthetics – from stylized concept art to crisp product mockups.","url":"https://developers.cloudflare.com/workers-ai/models/lucid-origin/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Phoenix 1.0 is a model by Leonardo.Ai that generates images with exceptional prompt adherence and coherent text. title: phoenix-1.0 image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/llms.txt > Use this file to discover all available pages before exploring further. ![Leonardo logo](https://developers.cloudflare.com/_astro/leonardo.JZysY-g3.svg) # phoenix-1.0 Text-to-Image • Leonardo Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/models/phoenix-1.0/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Model ID `@cf/leonardo/phoenix-1.0` Copy - Cloudflare-hosted - Partner - Zero data retention Phoenix 1.0 is a model by Leonardo.Ai that generates images with exceptional prompt adherence and coherent text. | Model Info | | | --- | --- | | Terms and License | [link ↗](https://leonardo.ai/terms-of-service/) | | Zero data retention | Yes | | Partner | Yes | | Unit Pricing | $0.00583 per 512 by 512 tile, $0.00011 per step | ## Parameters prompt `string`requiredminLength: 1A text description of the image you want to generate. guidance `number`default: 2minimum: 2maximum: 10Controls how closely the generated image should adhere to the prompt; higher values make the image more aligned with the prompt seed `integer`minimum: 0Random seed for reproducibility of the image generation height `integer`default: 1024minimum: 0maximum: 2048The height of the generated image in pixels width `integer`default: 1024minimum: 0maximum: 2048The width of the generated image in pixels num\_steps `integer`default: 25minimum: 1maximum: 50The number of diffusion steps; higher values can improve quality but take longer negative\_prompt `string`minLength: 1Specify what to exclude from the generated images The binding returns a `ReadableStream` with the output (check the model's output schema). ## API Schemas (Raw) Input [Open](https://developers.cloudflare.com/workers-ai/models/phoenix-1.0/schema-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/phoenix-1.0/schema-input.json) Output [Open](https://developers.cloudflare.com/workers-ai/models/phoenix-1.0/schema-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/phoenix-1.0/schema-output.json) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/models/phoenix-1.0/#page","headline":"phoenix-1.0","description":"Phoenix 1.0 is a model by Leonardo.Ai that generates images with exceptional prompt adherence and coherent text.","url":"https://developers.cloudflare.com/workers-ai/models/phoenix-1.0/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: LLaVA is an open-source chatbot trained by fine-tuning LLaMA/Vicuna on GPT-generated multimodal instruction-following data. It is an auto-regressive language model, based on the transformer architecture. title: llava-1.5-7b-hf image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/llms.txt > Use this file to discover all available pages before exploring further. l # llava-1.5-7b-hf Beta Image-to-Text • llava-hf Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/models/llava-1.5-7b-hf/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Model ID `@cf/llava-hf/llava-1.5-7b-hf` Copy - Cloudflare-hosted - Planned deprecation - Zero data retention LLaVA is an open-source chatbot trained by fine-tuning LLaMA/Vicuna on GPT-generated multimodal instruction-following data. It is an auto-regressive language model, based on the transformer architecture. | Model Info | | | --- | --- | | Planned Deprecation | 11/20/2026 | | Zero data retention | Yes | | Beta | Yes | ## Parameters Option 1 stringformat: binary Binary string representing the image contents. ▶Option 2{} object description `string` ## API Schemas (Raw) Input [Open](https://developers.cloudflare.com/workers-ai/models/llava-1.5-7b-hf/schema-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/llava-1.5-7b-hf/schema-input.json) Output [Open](https://developers.cloudflare.com/workers-ai/models/llava-1.5-7b-hf/schema-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/llava-1.5-7b-hf/schema-output.json) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/models/llava-1.5-7b-hf/#page","headline":"llava-1.5-7b-hf","description":"LLaVA is an open-source chatbot trained by fine-tuning LLaMA/Vicuna on GPT-generated multimodal instruction-following data. It is an auto-regressive language model, based on the transformer architecture.","url":"https://developers.cloudflare.com/workers-ai/models/llava-1.5-7b-hf/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Stable Diffusion model that has been fine-tuned to be better at photorealism without sacrificing range. title: dreamshaper-8-lcm image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/llms.txt > Use this file to discover all available pages before exploring further. l # dreamshaper-8-lcm Text-to-Image • lykon Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/models/dreamshaper-8-lcm/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Model ID `@cf/lykon/dreamshaper-8-lcm` Copy - Cloudflare-hosted - Planned deprecation - Zero data retention Stable Diffusion model that has been fine-tuned to be better at photorealism without sacrificing range. | Model Info | | | --- | --- | | Planned Deprecation | 11/20/2026 | | More information | [link ↗](https://huggingface.co/Lykon/DreamShaper) | | Zero data retention | Yes | ## Parameters prompt `string`requiredminLength: 1A text description of the image you want to generate negative\_prompt `string`Text describing elements to avoid in the generated image height `integer`minimum: 256maximum: 2048The height of the generated image in pixels width `integer`minimum: 256maximum: 2048The width of the generated image in pixels ▶image\[] `array`For use with img2img tasks. An array of integers that represent the image data constrained to 8-bit unsigned integer values image\_b64 `string`For use with img2img tasks. A base64-encoded string of the input image ▶mask\[] `array`An array representing An array of integers that represent mask image data for inpainting constrained to 8-bit unsigned integer values num\_steps `integer`default: 20maximum: 20The number of diffusion steps; higher values can improve quality but take longer strength `number`default: 1A value between 0 and 1 indicating how strongly to apply the transformation during img2img tasks; lower values make the output closer to the input image guidance `number`default: 7.5Controls how closely the generated image should adhere to the prompt; higher values make the image more aligned with the prompt seed `integer`Random seed for reproducibility of the image generation The binding returns a `ReadableStream` with the output (check the model's output schema). ## API Schemas (Raw) Input [Open](https://developers.cloudflare.com/workers-ai/models/dreamshaper-8-lcm/schema-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/dreamshaper-8-lcm/schema-input.json) Output [Open](https://developers.cloudflare.com/workers-ai/models/dreamshaper-8-lcm/schema-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/dreamshaper-8-lcm/schema-output.json) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/models/dreamshaper-8-lcm/#page","headline":"dreamshaper-8-lcm","description":"Stable Diffusion model that has been fine-tuned to be better at photorealism without sacrificing range.","url":"https://developers.cloudflare.com/workers-ai/models/dreamshaper-8-lcm/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: "This is a Llama2 base model that Cloudflare dedicated for inference with LoRA adapters. Llama 2 is a collection of pretrained and fine-tuned generative text models ranging in scale from 7 billion to 70 billion parameters. This is the repository for the 7B fine-tuned model, optimized for dialogue use cases and converted for the Hugging Face Transformers format. " title: llama-2-7b-chat-hf-lora image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/llms.txt > Use this file to discover all available pages before exploring further. ![Meta logo](https://developers.cloudflare.com/_astro/meta.CTzB_ysm.svg) # llama-2-7b-chat-hf-lora Beta Text Generation • Meta Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/models/llama-2-7b-chat-hf-lora/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Model ID `@cf/meta-llama/llama-2-7b-chat-hf-lora` Copy - Cloudflare-hosted - LoRA - Planned deprecation - Zero data retention This is a Llama2 base model that Cloudflare dedicated for inference with LoRA adapters. Llama 2 is a collection of pretrained and fine-tuned generative text models ranging in scale from 7 billion to 70 billion parameters. This is the repository for the 7B fine-tuned model, optimized for dialogue use cases and converted for the Hugging Face Transformers format. | Model Info | | | --- | --- | | Planned Deprecation | 11/20/2026 | | Context Window [ ↗](https://developers.cloudflare.com/workers-ai/platform/glossary/) | 8,192 tokens | | Zero data retention | Yes | | LoRA | Yes | | Beta | Yes | ## Playground Try out this model with Workers AI LLM Playground. It does not require any setup or authentication and is an instant way to preview and test a model directly in the browser. [Launch the LLM Playground](https://playground.ai.cloudflare.com/?model=@cf/meta-llama/llama-2-7b-chat-hf-lora) ## Usage ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const messages = [ { role: "system", content: "You are a friendly assistant" }, { role: "user", content: "What is the origin of the phrase Hello, World", }, ]; const stream = await env.AI.run("@cf/meta-llama/llama-2-7b-chat-hf-lora", { messages, stream: true, }); return new Response(stream, { headers: { "content-type": "text/event-stream" }, }); }, } satisfies ExportedHandler; ``` ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const messages = [ { role: "system", content: "You are a friendly assistant" }, { role: "user", content: "What is the origin of the phrase Hello, World", }, ]; const response = await env.AI.run("@cf/meta-llama/llama-2-7b-chat-hf-lora", { messages }); return Response.json(response); }, } satisfies ExportedHandler; ``` ```py import os import requests ACCOUNT_ID = "your-account-id" AUTH_TOKEN = os.environ.get("CLOUDFLARE_AUTH_TOKEN") prompt = "Tell me all about PEP-8" response = requests.post( f"https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai/run/@cf/meta-llama/llama-2-7b-chat-hf-lora", headers={"Authorization": f"Bearer {AUTH_TOKEN}"}, json={ "messages": [ {"role": "system", "content": "You are a friendly assistant"}, {"role": "user", "content": prompt} ] } ) result = response.json() print(result) ``` ```sh curl https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/run/@cf/meta-llama/llama-2-7b-chat-hf-lora \ -X POST \ -H "Authorization: Bearer $CLOUDFLARE_AUTH_TOKEN" \ -d '{ "messages": [{ "role": "system", "content": "You are a friendly assistant" }, { "role": "user", "content": "Why is pizza so good" }]}' ``` OpenAI compatible endpoints Workers AI also supports OpenAI compatible API endpoints for `/v1/chat/completions` and `/v1/embeddings`. For more details, refer to [Configurations](https://developers.cloudflare.com/workers-ai/configuration/open-ai-compatibility/) . ## Parameters ### Input prompt `string`requiredminLength: 1The input text prompt for the model to generate a response. lora `string`Name of the LoRA (Low-Rank Adaptation) model to fine-tune the base model. ▶response\_format{} `object` raw `boolean`default: falseIf true, a chat template is not applied and you must adhere to the specific model's expected formatting. stream `boolean`default: falseIf true, the response will be streamed back incrementally using SSE, Server Sent Events. max\_tokens `integer`default: 256The maximum number of tokens to generate in the response. temperature `number`default: 0.6minimum: 0maximum: 5Controls the randomness of the output; higher values produce more random results. top\_p `number`minimum: 0.001maximum: 1Adjusts the creativity of the AI's responses by controlling how many possible words it considers. Lower values make outputs more predictable; higher values allow for more varied and creative responses. top\_k `integer`minimum: 1maximum: 50Limits the AI to choose from the top 'k' most probable words. Lower values make responses more focused; higher values introduce more variety and potential surprises. seed `integer`minimum: 1maximum: 9999999999Random seed for reproducibility of the generation. repetition\_penalty `number`minimum: 0maximum: 2Penalty for repeated tokens; higher values discourage repetition. frequency\_penalty `number`minimum: -2maximum: 2Decreases the likelihood of the model repeating the same lines verbatim. presence\_penalty `number`minimum: -2maximum: 2Increases the likelihood of the model introducing new topics. ### Output
Synchronous — Send a request and receive a complete response response stringThe generated text response from the model ▶usage{} objectUsage statistics for the inference request ▶tool\_calls\[] arrayAn array of tool calls requests made during the response generation
Streaming — Send a request with `stream: true` and receive server-sent events type string format binary
## API Schemas (Raw) SynchronousInput [Open](https://developers.cloudflare.com/workers-ai/models/llama-2-7b-chat-hf-lora/sync-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/llama-2-7b-chat-hf-lora/sync-input.json) SynchronousOutput [Open](https://developers.cloudflare.com/workers-ai/models/llama-2-7b-chat-hf-lora/sync-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/llama-2-7b-chat-hf-lora/sync-output.json) StreamingInput [Open](https://developers.cloudflare.com/workers-ai/models/llama-2-7b-chat-hf-lora/streaming-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/llama-2-7b-chat-hf-lora/streaming-input.json) StreamingOutput [Open](https://developers.cloudflare.com/workers-ai/models/llama-2-7b-chat-hf-lora/streaming-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/llama-2-7b-chat-hf-lora/streaming-output.json) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/models/llama-2-7b-chat-hf-lora/#page","headline":"llama-2-7b-chat-hf-lora","description":"This is a Llama2 base model that Cloudflare dedicated for inference with LoRA adapters. Llama 2 is a collection of pretrained and fine-tuned generative text models ranging in scale from 7 billion to 70 billion parameters. This is the repository for the 7B fine-tuned model, optimized for dialogue use cases and converted for the Hugging Face Transformers format.","url":"https://developers.cloudflare.com/workers-ai/models/llama-2-7b-chat-hf-lora/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Llama 3.1 8B quantized to FP8 precision title: llama-3.1-8b-instruct-fp8 image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/llms.txt > Use this file to discover all available pages before exploring further. ![Meta logo](https://developers.cloudflare.com/_astro/meta.CTzB_ysm.svg) # llama-3.1-8b-instruct-fp8 Text Generation • Meta Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/models/llama-3.1-8b-instruct-fp8/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Model ID `@cf/meta/llama-3.1-8b-instruct-fp8` Copy - Cloudflare-hosted - Planned deprecation - Zero data retention Llama 3.1 8B quantized to FP8 precision | Model Info | | | --- | --- | | Planned Deprecation | 11/20/2026 | | Context Window [ ↗](https://developers.cloudflare.com/workers-ai/platform/glossary/) | 32,000 tokens | | Terms and License | [link ↗](https://github.com/meta-llama/llama-models/blob/main/models/llama3_1/LICENSE) | | Zero data retention | Yes | | Unit Pricing | $0.152 per M input tokens, $0.287 per M output tokens | ## Playground Try out this model with Workers AI LLM Playground. It does not require any setup or authentication and is an instant way to preview and test a model directly in the browser. [Launch the LLM Playground](https://playground.ai.cloudflare.com/?model=@cf/meta/llama-3.1-8b-instruct-fp8) ## Usage ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const messages = [ { role: "system", content: "You are a friendly assistant" }, { role: "user", content: "What is the origin of the phrase Hello, World", }, ]; const stream = await env.AI.run("@cf/meta/llama-3.1-8b-instruct-fp8", { messages, stream: true, }); return new Response(stream, { headers: { "content-type": "text/event-stream" }, }); }, } satisfies ExportedHandler; ``` ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const messages = [ { role: "system", content: "You are a friendly assistant" }, { role: "user", content: "What is the origin of the phrase Hello, World", }, ]; const response = await env.AI.run("@cf/meta/llama-3.1-8b-instruct-fp8", { messages }); return Response.json(response); }, } satisfies ExportedHandler; ``` ```py import os import requests ACCOUNT_ID = "your-account-id" AUTH_TOKEN = os.environ.get("CLOUDFLARE_AUTH_TOKEN") prompt = "Tell me all about PEP-8" response = requests.post( f"https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai/run/@cf/meta/llama-3.1-8b-instruct-fp8", headers={"Authorization": f"Bearer {AUTH_TOKEN}"}, json={ "messages": [ {"role": "system", "content": "You are a friendly assistant"}, {"role": "user", "content": prompt} ] } ) result = response.json() print(result) ``` ```sh curl https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/run/@cf/meta/llama-3.1-8b-instruct-fp8 \ -X POST \ -H "Authorization: Bearer $CLOUDFLARE_AUTH_TOKEN" \ -d '{ "messages": [{ "role": "system", "content": "You are a friendly assistant" }, { "role": "user", "content": "Why is pizza so good" }]}' ``` OpenAI compatible endpoints Workers AI also supports OpenAI compatible API endpoints for `/v1/chat/completions` and `/v1/embeddings`. For more details, refer to [Configurations](https://developers.cloudflare.com/workers-ai/configuration/open-ai-compatibility/) . ## Parameters ### Input prompt `string`requiredminLength: 1The input text prompt for the model to generate a response. lora `string`Name of the LoRA (Low-Rank Adaptation) model to fine-tune the base model. ▶response\_format{} `object` raw `boolean`default: falseIf true, a chat template is not applied and you must adhere to the specific model's expected formatting. stream `boolean`default: falseIf true, the response will be streamed back incrementally using SSE, Server Sent Events. max\_tokens `integer`default: 256The maximum number of tokens to generate in the response. temperature `number`default: 0.6minimum: 0maximum: 5Controls the randomness of the output; higher values produce more random results. top\_p `number`minimum: 0.001maximum: 1Adjusts the creativity of the AI's responses by controlling how many possible words it considers. Lower values make outputs more predictable; higher values allow for more varied and creative responses. top\_k `integer`minimum: 1maximum: 50Limits the AI to choose from the top 'k' most probable words. Lower values make responses more focused; higher values introduce more variety and potential surprises. seed `integer`minimum: 1maximum: 9999999999Random seed for reproducibility of the generation. repetition\_penalty `number`minimum: 0maximum: 2Penalty for repeated tokens; higher values discourage repetition. frequency\_penalty `number`minimum: -2maximum: 2Decreases the likelihood of the model repeating the same lines verbatim. presence\_penalty `number`minimum: -2maximum: 2Increases the likelihood of the model introducing new topics. ### Output
Synchronous — Send a request and receive a complete response response stringThe generated text response from the model ▶usage{} objectUsage statistics for the inference request ▶tool\_calls\[] arrayAn array of tool calls requests made during the response generation
Streaming — Send a request with `stream: true` and receive server-sent events type string format binary
## API Schemas (Raw) SynchronousInput [Open](https://developers.cloudflare.com/workers-ai/models/llama-3.1-8b-instruct-fp8/sync-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/llama-3.1-8b-instruct-fp8/sync-input.json) SynchronousOutput [Open](https://developers.cloudflare.com/workers-ai/models/llama-3.1-8b-instruct-fp8/sync-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/llama-3.1-8b-instruct-fp8/sync-output.json) StreamingInput [Open](https://developers.cloudflare.com/workers-ai/models/llama-3.1-8b-instruct-fp8/streaming-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/llama-3.1-8b-instruct-fp8/streaming-input.json) StreamingOutput [Open](https://developers.cloudflare.com/workers-ai/models/llama-3.1-8b-instruct-fp8/streaming-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/llama-3.1-8b-instruct-fp8/streaming-output.json) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/models/llama-3.1-8b-instruct-fp8/#page","headline":"llama-3.1-8b-instruct-fp8","description":"Llama 3.1 8B quantized to FP8 precision","url":"https://developers.cloudflare.com/workers-ai/models/llama-3.1-8b-instruct-fp8/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: " The Llama 3.2-Vision instruction-tuned models are optimized for visual recognition, image reasoning, captioning, and answering general questions about an image." title: llama-3.2-11b-vision-instruct image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/llms.txt > Use this file to discover all available pages before exploring further. ![Meta logo](https://developers.cloudflare.com/_astro/meta.CTzB_ysm.svg) # llama-3.2-11b-vision-instruct Text Generation • Meta Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/models/llama-3.2-11b-vision-instruct/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Model ID `@cf/meta/llama-3.2-11b-vision-instruct` Copy - Cloudflare-hosted - LoRA - Vision - Zero data retention The Llama 3.2-Vision instruction-tuned models are optimized for visual recognition, image reasoning, captioning, and answering general questions about an image. Note To use Llama 3.2 11b Vision Instruct, you need to agree to the [Meta License](https://github.com/meta-llama/llama-models/blob/main/models/llama3_2/LICENSE) and [Acceptable Use Policy](https://github.com/meta-llama/llama-models/blob/main/models/llama3_2/USE_POLICY.md). To do so, please send an initial request to`@cf/meta/llama-3.2-11b-vision-instruct` with`"prompt" : "agree"`. After that, you'll be able to use the model as normal. ```sh curl https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/run/@cf/meta/llama-3.2-11b-vision-instruct \ -X POST \ -H "Authorization: Bearer $CLOUDFLARE_AUTH_TOKEN" \ -d '{ "prompt": "agree"}' ``` | Model Info | | | --- | --- | | Context Window [ ↗](https://developers.cloudflare.com/workers-ai/platform/glossary/) | 128,000 tokens | | Terms and License | [link ↗](https://github.com/meta-llama/llama-models/blob/main/models/llama3_2/LICENSE) | | Vision | Yes | | Zero data retention | Yes | | LoRA | Yes | | Unit Pricing | $0.0485 per M input tokens, $0.676 per M output tokens | ## Playground Try out this model with Workers AI LLM Playground. It does not require any setup or authentication and is an instant way to preview and test a model directly in the browser. [Launch the LLM Playground](https://playground.ai.cloudflare.com/?model=@cf/meta/llama-3.2-11b-vision-instruct) ## Usage ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const messages = [ { role: "system", content: "You are a friendly assistant" }, { role: "user", content: "What is the origin of the phrase Hello, World", }, ]; const stream = await env.AI.run("@cf/meta/llama-3.2-11b-vision-instruct", { messages, stream: true, }); return new Response(stream, { headers: { "content-type": "text/event-stream" }, }); }, } satisfies ExportedHandler; ``` ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const messages = [ { role: "system", content: "You are a friendly assistant" }, { role: "user", content: "What is the origin of the phrase Hello, World", }, ]; const response = await env.AI.run("@cf/meta/llama-3.2-11b-vision-instruct", { messages }); return Response.json(response); }, } satisfies ExportedHandler; ``` ```py import os import requests ACCOUNT_ID = "your-account-id" AUTH_TOKEN = os.environ.get("CLOUDFLARE_AUTH_TOKEN") prompt = "Tell me all about PEP-8" response = requests.post( f"https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai/run/@cf/meta/llama-3.2-11b-vision-instruct", headers={"Authorization": f"Bearer {AUTH_TOKEN}"}, json={ "messages": [ {"role": "system", "content": "You are a friendly assistant"}, {"role": "user", "content": prompt} ] } ) result = response.json() print(result) ``` ```sh curl https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/run/@cf/meta/llama-3.2-11b-vision-instruct \ -X POST \ -H "Authorization: Bearer $CLOUDFLARE_AUTH_TOKEN" \ -d '{ "messages": [{ "role": "system", "content": "You are a friendly assistant" }, { "role": "user", "content": "Why is pizza so good" }]}' ``` OpenAI compatible endpoints Workers AI also supports OpenAI compatible API endpoints for `/v1/chat/completions` and `/v1/embeddings`. For more details, refer to [Configurations](https://developers.cloudflare.com/workers-ai/configuration/open-ai-compatibility/) . ## Parameters ### Input prompt `string`requiredminLength: 1maxLength: 131072The input text prompt for the model to generate a response. ▶image `one of` raw `boolean`default: falseIf true, a chat template is not applied and you must adhere to the specific model's expected formatting. stream `boolean`default: falseIf true, the response will be streamed back incrementally using SSE, Server Sent Events. max\_tokens `integer`default: 256The maximum number of tokens to generate in the response. temperature `number`default: 0.6minimum: 0maximum: 5Controls the randomness of the output; higher values produce more random results. top\_p `number`minimum: 0maximum: 2Adjusts the creativity of the AI's responses by controlling how many possible words it considers. Lower values make outputs more predictable; higher values allow for more varied and creative responses. top\_k `integer`minimum: 1maximum: 50Limits the AI to choose from the top 'k' most probable words. Lower values make responses more focused; higher values introduce more variety and potential surprises. seed `integer`minimum: 1maximum: 9999999999Random seed for reproducibility of the generation. repetition\_penalty `number`minimum: 0maximum: 2Penalty for repeated tokens; higher values discourage repetition. frequency\_penalty `number`minimum: 0maximum: 2Decreases the likelihood of the model repeating the same lines verbatim. presence\_penalty `number`minimum: 0maximum: 2Increases the likelihood of the model introducing new topics. lora `string`Name of the LoRA (Low-Rank Adaptation) model to fine-tune the base model. ### Output
Synchronous — Send a request and receive a complete response response stringThe generated text response from the model ▶tool\_calls\[] arrayAn array of tool calls requests made during the response generation
Streaming — Send a request with `stream: true` and receive server-sent events type string contentType text/event-stream format binary
## API Schemas (Raw) SynchronousInput [Open](https://developers.cloudflare.com/workers-ai/models/llama-3.2-11b-vision-instruct/sync-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/llama-3.2-11b-vision-instruct/sync-input.json) SynchronousOutput [Open](https://developers.cloudflare.com/workers-ai/models/llama-3.2-11b-vision-instruct/sync-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/llama-3.2-11b-vision-instruct/sync-output.json) StreamingInput [Open](https://developers.cloudflare.com/workers-ai/models/llama-3.2-11b-vision-instruct/streaming-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/llama-3.2-11b-vision-instruct/streaming-input.json) StreamingOutput [Open](https://developers.cloudflare.com/workers-ai/models/llama-3.2-11b-vision-instruct/streaming-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/llama-3.2-11b-vision-instruct/streaming-output.json) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/models/llama-3.2-11b-vision-instruct/#page","headline":"llama-3.2-11b-vision-instruct","description":"The Llama 3.2-Vision instruction-tuned models are optimized for visual recognition, image reasoning, captioning, and answering general questions about an image.","url":"https://developers.cloudflare.com/workers-ai/models/llama-3.2-11b-vision-instruct/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: The Llama 3.2 instruction-tuned text only models are optimized for multilingual dialogue use cases, including agentic retrieval and summarization tasks. title: llama-3.2-1b-instruct image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/llms.txt > Use this file to discover all available pages before exploring further. ![Meta logo](https://developers.cloudflare.com/_astro/meta.CTzB_ysm.svg) # llama-3.2-1b-instruct Text Generation • Meta Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/models/llama-3.2-1b-instruct/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Model ID `@cf/meta/llama-3.2-1b-instruct` Copy - Cloudflare-hosted - Planned deprecation - Zero data retention The Llama 3.2 instruction-tuned text only models are optimized for multilingual dialogue use cases, including agentic retrieval and summarization tasks. | Model Info | | | --- | --- | | Planned Deprecation | 11/20/2026 | | Context Window [ ↗](https://developers.cloudflare.com/workers-ai/platform/glossary/) | 60,000 tokens | | Terms and License | [link ↗](https://github.com/meta-llama/llama-models/blob/main/models/llama3_2/LICENSE) | | Zero data retention | Yes | | Unit Pricing | $0.027 per M input tokens, $0.201 per M output tokens | ## Playground Try out this model with Workers AI LLM Playground. It does not require any setup or authentication and is an instant way to preview and test a model directly in the browser. [Launch the LLM Playground](https://playground.ai.cloudflare.com/?model=@cf/meta/llama-3.2-1b-instruct) ## Usage ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const messages = [ { role: "system", content: "You are a friendly assistant" }, { role: "user", content: "What is the origin of the phrase Hello, World", }, ]; const stream = await env.AI.run("@cf/meta/llama-3.2-1b-instruct", { messages, stream: true, }); return new Response(stream, { headers: { "content-type": "text/event-stream" }, }); }, } satisfies ExportedHandler; ``` ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const messages = [ { role: "system", content: "You are a friendly assistant" }, { role: "user", content: "What is the origin of the phrase Hello, World", }, ]; const response = await env.AI.run("@cf/meta/llama-3.2-1b-instruct", { messages }); return Response.json(response); }, } satisfies ExportedHandler; ``` ```py import os import requests ACCOUNT_ID = "your-account-id" AUTH_TOKEN = os.environ.get("CLOUDFLARE_AUTH_TOKEN") prompt = "Tell me all about PEP-8" response = requests.post( f"https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai/run/@cf/meta/llama-3.2-1b-instruct", headers={"Authorization": f"Bearer {AUTH_TOKEN}"}, json={ "messages": [ {"role": "system", "content": "You are a friendly assistant"}, {"role": "user", "content": prompt} ] } ) result = response.json() print(result) ``` ```sh curl https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/run/@cf/meta/llama-3.2-1b-instruct \ -X POST \ -H "Authorization: Bearer $CLOUDFLARE_AUTH_TOKEN" \ -d '{ "messages": [{ "role": "system", "content": "You are a friendly assistant" }, { "role": "user", "content": "Why is pizza so good" }]}' ``` OpenAI compatible endpoints Workers AI also supports OpenAI compatible API endpoints for `/v1/chat/completions` and `/v1/embeddings`. For more details, refer to [Configurations](https://developers.cloudflare.com/workers-ai/configuration/open-ai-compatibility/) . ## Parameters ### Input prompt `string`requiredminLength: 1The input text prompt for the model to generate a response. lora `string`Name of the LoRA (Low-Rank Adaptation) model to fine-tune the base model. ▶response\_format{} `object` raw `boolean`default: falseIf true, a chat template is not applied and you must adhere to the specific model's expected formatting. stream `boolean`default: falseIf true, the response will be streamed back incrementally using SSE, Server Sent Events. max\_tokens `integer`default: 256The maximum number of tokens to generate in the response. temperature `number`default: 0.6minimum: 0maximum: 5Controls the randomness of the output; higher values produce more random results. top\_p `number`minimum: 0.001maximum: 1Adjusts the creativity of the AI's responses by controlling how many possible words it considers. Lower values make outputs more predictable; higher values allow for more varied and creative responses. top\_k `integer`minimum: 1maximum: 50Limits the AI to choose from the top 'k' most probable words. Lower values make responses more focused; higher values introduce more variety and potential surprises. seed `integer`minimum: 1maximum: 9999999999Random seed for reproducibility of the generation. repetition\_penalty `number`minimum: 0maximum: 2Penalty for repeated tokens; higher values discourage repetition. frequency\_penalty `number`minimum: -2maximum: 2Decreases the likelihood of the model repeating the same lines verbatim. presence\_penalty `number`minimum: -2maximum: 2Increases the likelihood of the model introducing new topics. ### Output
Synchronous — Send a request and receive a complete response response stringThe generated text response from the model ▶usage{} objectUsage statistics for the inference request ▶tool\_calls\[] arrayAn array of tool calls requests made during the response generation
Streaming — Send a request with `stream: true` and receive server-sent events type string format binary
## API Schemas (Raw) SynchronousInput [Open](https://developers.cloudflare.com/workers-ai/models/llama-3.2-1b-instruct/sync-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/llama-3.2-1b-instruct/sync-input.json) SynchronousOutput [Open](https://developers.cloudflare.com/workers-ai/models/llama-3.2-1b-instruct/sync-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/llama-3.2-1b-instruct/sync-output.json) StreamingInput [Open](https://developers.cloudflare.com/workers-ai/models/llama-3.2-1b-instruct/streaming-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/llama-3.2-1b-instruct/streaming-input.json) StreamingOutput [Open](https://developers.cloudflare.com/workers-ai/models/llama-3.2-1b-instruct/streaming-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/llama-3.2-1b-instruct/streaming-output.json) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/models/llama-3.2-1b-instruct/#page","headline":"llama-3.2-1b-instruct","description":"The Llama 3.2 instruction-tuned text only models are optimized for multilingual dialogue use cases, including agentic retrieval and summarization tasks.","url":"https://developers.cloudflare.com/workers-ai/models/llama-3.2-1b-instruct/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: The Llama 3.2 instruction-tuned text only models are optimized for multilingual dialogue use cases, including agentic retrieval and summarization tasks. title: llama-3.2-3b-instruct image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/llms.txt > Use this file to discover all available pages before exploring further. ![Meta logo](https://developers.cloudflare.com/_astro/meta.CTzB_ysm.svg) # llama-3.2-3b-instruct Text Generation • Meta Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/models/llama-3.2-3b-instruct/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Model ID `@cf/meta/llama-3.2-3b-instruct` Copy - Cloudflare-hosted - LoRA - Planned deprecation - Zero data retention The Llama 3.2 instruction-tuned text only models are optimized for multilingual dialogue use cases, including agentic retrieval and summarization tasks. | Model Info | | | --- | --- | | Planned Deprecation | 11/20/2026 | | Context Window [ ↗](https://developers.cloudflare.com/workers-ai/platform/glossary/) | 80,000 tokens | | Terms and License | [link ↗](https://github.com/meta-llama/llama-models/blob/main/models/llama3_2/LICENSE) | | Zero data retention | Yes | | LoRA | Yes | | Unit Pricing | $0.0509 per M input tokens, $0.335 per M output tokens | ## Playground Try out this model with Workers AI LLM Playground. It does not require any setup or authentication and is an instant way to preview and test a model directly in the browser. [Launch the LLM Playground](https://playground.ai.cloudflare.com/?model=@cf/meta/llama-3.2-3b-instruct) ## Usage ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const messages = [ { role: "system", content: "You are a friendly assistant" }, { role: "user", content: "What is the origin of the phrase Hello, World", }, ]; const stream = await env.AI.run("@cf/meta/llama-3.2-3b-instruct", { messages, stream: true, }); return new Response(stream, { headers: { "content-type": "text/event-stream" }, }); }, } satisfies ExportedHandler; ``` ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const messages = [ { role: "system", content: "You are a friendly assistant" }, { role: "user", content: "What is the origin of the phrase Hello, World", }, ]; const response = await env.AI.run("@cf/meta/llama-3.2-3b-instruct", { messages }); return Response.json(response); }, } satisfies ExportedHandler; ``` ```py import os import requests ACCOUNT_ID = "your-account-id" AUTH_TOKEN = os.environ.get("CLOUDFLARE_AUTH_TOKEN") prompt = "Tell me all about PEP-8" response = requests.post( f"https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai/run/@cf/meta/llama-3.2-3b-instruct", headers={"Authorization": f"Bearer {AUTH_TOKEN}"}, json={ "messages": [ {"role": "system", "content": "You are a friendly assistant"}, {"role": "user", "content": prompt} ] } ) result = response.json() print(result) ``` ```sh curl https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/run/@cf/meta/llama-3.2-3b-instruct \ -X POST \ -H "Authorization: Bearer $CLOUDFLARE_AUTH_TOKEN" \ -d '{ "messages": [{ "role": "system", "content": "You are a friendly assistant" }, { "role": "user", "content": "Why is pizza so good" }]}' ``` OpenAI compatible endpoints Workers AI also supports OpenAI compatible API endpoints for `/v1/chat/completions` and `/v1/embeddings`. For more details, refer to [Configurations](https://developers.cloudflare.com/workers-ai/configuration/open-ai-compatibility/) . ## Parameters ### Input prompt `string`requiredminLength: 1The input text prompt for the model to generate a response. lora `string`Name of the LoRA (Low-Rank Adaptation) model to fine-tune the base model. ▶response\_format{} `object` raw `boolean`default: falseIf true, a chat template is not applied and you must adhere to the specific model's expected formatting. stream `boolean`default: falseIf true, the response will be streamed back incrementally using SSE, Server Sent Events. max\_tokens `integer`default: 256The maximum number of tokens to generate in the response. temperature `number`default: 0.6minimum: 0maximum: 5Controls the randomness of the output; higher values produce more random results. top\_p `number`minimum: 0.001maximum: 1Adjusts the creativity of the AI's responses by controlling how many possible words it considers. Lower values make outputs more predictable; higher values allow for more varied and creative responses. top\_k `integer`minimum: 1maximum: 50Limits the AI to choose from the top 'k' most probable words. Lower values make responses more focused; higher values introduce more variety and potential surprises. seed `integer`minimum: 1maximum: 9999999999Random seed for reproducibility of the generation. repetition\_penalty `number`minimum: 0maximum: 2Penalty for repeated tokens; higher values discourage repetition. frequency\_penalty `number`minimum: -2maximum: 2Decreases the likelihood of the model repeating the same lines verbatim. presence\_penalty `number`minimum: -2maximum: 2Increases the likelihood of the model introducing new topics. ### Output
Synchronous — Send a request and receive a complete response response stringThe generated text response from the model ▶usage{} objectUsage statistics for the inference request ▶tool\_calls\[] arrayAn array of tool calls requests made during the response generation
Streaming — Send a request with `stream: true` and receive server-sent events type string format binary
## API Schemas (Raw) SynchronousInput [Open](https://developers.cloudflare.com/workers-ai/models/llama-3.2-3b-instruct/sync-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/llama-3.2-3b-instruct/sync-input.json) SynchronousOutput [Open](https://developers.cloudflare.com/workers-ai/models/llama-3.2-3b-instruct/sync-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/llama-3.2-3b-instruct/sync-output.json) StreamingInput [Open](https://developers.cloudflare.com/workers-ai/models/llama-3.2-3b-instruct/streaming-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/llama-3.2-3b-instruct/streaming-input.json) StreamingOutput [Open](https://developers.cloudflare.com/workers-ai/models/llama-3.2-3b-instruct/streaming-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/llama-3.2-3b-instruct/streaming-output.json) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/models/llama-3.2-3b-instruct/#page","headline":"llama-3.2-3b-instruct","description":"The Llama 3.2 instruction-tuned text only models are optimized for multilingual dialogue use cases, including agentic retrieval and summarization tasks.","url":"https://developers.cloudflare.com/workers-ai/models/llama-3.2-3b-instruct/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Llama 3.3 70B quantized to fp8 precision, optimized to be faster. title: llama-3.3-70b-instruct-fp8-fast image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/llms.txt > Use this file to discover all available pages before exploring further. ![Meta logo](https://developers.cloudflare.com/_astro/meta.CTzB_ysm.svg) # llama-3.3-70b-instruct-fp8-fast Text Generation • Meta Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/models/llama-3.3-70b-instruct-fp8-fast/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Model ID `@cf/meta/llama-3.3-70b-instruct-fp8-fast` Copy - Cloudflare-hosted - Batch - Function calling - Zero data retention Llama 3.3 70B quantized to fp8 precision, optimized to be faster. | Model Info | | | --- | --- | | Context Window [ ↗](https://developers.cloudflare.com/workers-ai/platform/glossary/) | 24,000 tokens | | Terms and License | [link ↗](https://github.com/meta-llama/llama-models/blob/main/models/llama3_3/LICENSE) | | Function calling [↗](https://developers.cloudflare.com/workers-ai/function-calling/) | Yes | | Zero data retention | Yes | | Batch | Yes | | Unit Pricing | $0.293 per M input tokens, $2.253 per M output tokens | ## Playground Try out this model with Workers AI LLM Playground. It does not require any setup or authentication and is an instant way to preview and test a model directly in the browser. [Launch the LLM Playground](https://playground.ai.cloudflare.com/?model=@cf/meta/llama-3.3-70b-instruct-fp8-fast) ## Usage ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const messages = [ { role: "system", content: "You are a friendly assistant" }, { role: "user", content: "What is the origin of the phrase Hello, World", }, ]; const stream = await env.AI.run("@cf/meta/llama-3.3-70b-instruct-fp8-fast", { messages, stream: true, }); return new Response(stream, { headers: { "content-type": "text/event-stream" }, }); }, } satisfies ExportedHandler; ``` ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const messages = [ { role: "system", content: "You are a friendly assistant" }, { role: "user", content: "What is the origin of the phrase Hello, World", }, ]; const response = await env.AI.run("@cf/meta/llama-3.3-70b-instruct-fp8-fast", { messages }); return Response.json(response); }, } satisfies ExportedHandler; ``` ```py import os import requests ACCOUNT_ID = "your-account-id" AUTH_TOKEN = os.environ.get("CLOUDFLARE_AUTH_TOKEN") prompt = "Tell me all about PEP-8" response = requests.post( f"https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai/run/@cf/meta/llama-3.3-70b-instruct-fp8-fast", headers={"Authorization": f"Bearer {AUTH_TOKEN}"}, json={ "messages": [ {"role": "system", "content": "You are a friendly assistant"}, {"role": "user", "content": prompt} ] } ) result = response.json() print(result) ``` ```sh curl https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/run/@cf/meta/llama-3.3-70b-instruct-fp8-fast \ -X POST \ -H "Authorization: Bearer $CLOUDFLARE_AUTH_TOKEN" \ -d '{ "messages": [{ "role": "system", "content": "You are a friendly assistant" }, { "role": "user", "content": "Why is pizza so good" }]}' ``` OpenAI compatible endpoints Workers AI also supports OpenAI compatible API endpoints for `/v1/chat/completions` and `/v1/embeddings`. For more details, refer to [Configurations](https://developers.cloudflare.com/workers-ai/configuration/open-ai-compatibility/) . ## Parameters
Synchronous — Send a request and receive a complete response prompt stringrequiredminLength: 1The input text prompt for the model to generate a response. lora stringName of the LoRA (Low-Rank Adaptation) model to fine-tune the base model. ▶response\_format{} object raw booleandefault: falseIf true, a chat template is not applied and you must adhere to the specific model's expected formatting. stream booleandefault: falseIf true, the response will be streamed back incrementally using SSE, Server Sent Events. max\_tokens integerdefault: 256The maximum number of tokens to generate in the response. temperature numberdefault: 0.6minimum: 0maximum: 5Controls the randomness of the output; higher values produce more random results. top\_p numberminimum: 0.001maximum: 1Adjusts the creativity of the AI's responses by controlling how many possible words it considers. Lower values make outputs more predictable; higher values allow for more varied and creative responses. top\_k integerminimum: 1maximum: 50Limits the AI to choose from the top 'k' most probable words. Lower values make responses more focused; higher values introduce more variety and potential surprises. seed integerminimum: 1maximum: 9999999999Random seed for reproducibility of the generation. repetition\_penalty numberminimum: 0maximum: 2Penalty for repeated tokens; higher values discourage repetition. frequency\_penalty numberminimum: -2maximum: 2Decreases the likelihood of the model repeating the same lines verbatim. presence\_penalty numberminimum: -2maximum: 2Increases the likelihood of the model introducing new topics. response stringThe generated text response from the model ▶usage{} objectUsage statistics for the inference request ▶tool\_calls\[] arrayAn array of tool calls requests made during the response generation
Streaming — Send a request with `stream: true` and receive server-sent events prompt stringrequiredminLength: 1The input text prompt for the model to generate a response. lora stringName of the LoRA (Low-Rank Adaptation) model to fine-tune the base model. ▶response\_format{} object raw booleandefault: falseIf true, a chat template is not applied and you must adhere to the specific model's expected formatting. stream booleandefault: falseIf true, the response will be streamed back incrementally using SSE, Server Sent Events. max\_tokens integerdefault: 256The maximum number of tokens to generate in the response. temperature numberdefault: 0.6minimum: 0maximum: 5Controls the randomness of the output; higher values produce more random results. top\_p numberminimum: 0.001maximum: 1Adjusts the creativity of the AI's responses by controlling how many possible words it considers. Lower values make outputs more predictable; higher values allow for more varied and creative responses. top\_k integerminimum: 1maximum: 50Limits the AI to choose from the top 'k' most probable words. Lower values make responses more focused; higher values introduce more variety and potential surprises. seed integerminimum: 1maximum: 9999999999Random seed for reproducibility of the generation. repetition\_penalty numberminimum: 0maximum: 2Penalty for repeated tokens; higher values discourage repetition. frequency\_penalty numberminimum: -2maximum: 2Decreases the likelihood of the model repeating the same lines verbatim. presence\_penalty numberminimum: -2maximum: 2Increases the likelihood of the model introducing new topics. type string contentType text/event-stream format binary
Batch — Send multiple requests in a single API call ▶requests\[] array response stringThe generated text response from the model ▶usage{} objectUsage statistics for the inference request ▶tool\_calls\[] arrayAn array of tool calls requests made during the response generation
## API Schemas (Raw) SynchronousInput [Open](https://developers.cloudflare.com/workers-ai/models/llama-3.3-70b-instruct-fp8-fast/sync-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/llama-3.3-70b-instruct-fp8-fast/sync-input.json) SynchronousOutput [Open](https://developers.cloudflare.com/workers-ai/models/llama-3.3-70b-instruct-fp8-fast/sync-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/llama-3.3-70b-instruct-fp8-fast/sync-output.json) StreamingInput [Open](https://developers.cloudflare.com/workers-ai/models/llama-3.3-70b-instruct-fp8-fast/streaming-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/llama-3.3-70b-instruct-fp8-fast/streaming-input.json) StreamingOutput [Open](https://developers.cloudflare.com/workers-ai/models/llama-3.3-70b-instruct-fp8-fast/streaming-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/llama-3.3-70b-instruct-fp8-fast/streaming-output.json) BatchInput [Open](https://developers.cloudflare.com/workers-ai/models/llama-3.3-70b-instruct-fp8-fast/batch-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/llama-3.3-70b-instruct-fp8-fast/batch-input.json) BatchOutput [Open](https://developers.cloudflare.com/workers-ai/models/llama-3.3-70b-instruct-fp8-fast/batch-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/llama-3.3-70b-instruct-fp8-fast/batch-output.json) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/models/llama-3.3-70b-instruct-fp8-fast/#page","headline":"llama-3.3-70b-instruct-fp8-fast","description":"Llama 3.3 70B quantized to fp8 precision, optimized to be faster.","url":"https://developers.cloudflare.com/workers-ai/models/llama-3.3-70b-instruct-fp8-fast/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Meta's Llama 4 Scout is a 17 billion parameter model with 16 experts that is natively multimodal. These models leverage a mixture-of-experts architecture to offer industry-leading performance in text and image understanding. title: llama-4-scout-17b-16e-instruct image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/llms.txt > Use this file to discover all available pages before exploring further. ![Meta logo](https://developers.cloudflare.com/_astro/meta.CTzB_ysm.svg) # llama-4-scout-17b-16e-instruct Text Generation • Meta Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/models/llama-4-scout-17b-16e-instruct/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Model ID `@cf/meta/llama-4-scout-17b-16e-instruct` Copy - Cloudflare-hosted - Batch - Function calling - Vision - Zero data retention Meta's Llama 4 Scout is a 17 billion parameter model with 16 experts that is natively multimodal. These models leverage a mixture-of-experts architecture to offer industry-leading performance in text and image understanding. | Model Info | | | --- | --- | | Context Window [ ↗](https://developers.cloudflare.com/workers-ai/platform/glossary/) | 131,000 tokens | | Terms and License | [link ↗](https://github.com/meta-llama/llama-models/blob/main/models/llama4/LICENSE) | | Function calling [↗](https://developers.cloudflare.com/workers-ai/function-calling/) | Yes | | Vision | Yes | | Zero data retention | Yes | | Batch | Yes | | Unit Pricing | $0.27 per M input tokens, $0.85 per M output tokens | ## Playground Try out this model with Workers AI LLM Playground. It does not require any setup or authentication and is an instant way to preview and test a model directly in the browser. [Launch the LLM Playground](https://playground.ai.cloudflare.com/?model=@cf/meta/llama-4-scout-17b-16e-instruct) ## Usage ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const messages = [ { role: "system", content: "You are a friendly assistant" }, { role: "user", content: "What is the origin of the phrase Hello, World", }, ]; const stream = await env.AI.run("@cf/meta/llama-4-scout-17b-16e-instruct", { messages, stream: true, }); return new Response(stream, { headers: { "content-type": "text/event-stream" }, }); }, } satisfies ExportedHandler; ``` ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const messages = [ { role: "system", content: "You are a friendly assistant" }, { role: "user", content: "What is the origin of the phrase Hello, World", }, ]; const response = await env.AI.run("@cf/meta/llama-4-scout-17b-16e-instruct", { messages }); return Response.json(response); }, } satisfies ExportedHandler; ``` ```py import os import requests ACCOUNT_ID = "your-account-id" AUTH_TOKEN = os.environ.get("CLOUDFLARE_AUTH_TOKEN") prompt = "Tell me all about PEP-8" response = requests.post( f"https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai/run/@cf/meta/llama-4-scout-17b-16e-instruct", headers={"Authorization": f"Bearer {AUTH_TOKEN}"}, json={ "messages": [ {"role": "system", "content": "You are a friendly assistant"}, {"role": "user", "content": prompt} ] } ) result = response.json() print(result) ``` ```sh curl https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/run/@cf/meta/llama-4-scout-17b-16e-instruct \ -X POST \ -H "Authorization: Bearer $CLOUDFLARE_AUTH_TOKEN" \ -d '{ "messages": [{ "role": "system", "content": "You are a friendly assistant" }, { "role": "user", "content": "Why is pizza so good" }]}' ``` OpenAI compatible endpoints Workers AI also supports OpenAI compatible API endpoints for `/v1/chat/completions` and `/v1/embeddings`. For more details, refer to [Configurations](https://developers.cloudflare.com/workers-ai/configuration/open-ai-compatibility/) . ## Parameters
Synchronous — Send a request and receive a complete response prompt stringrequiredminLength: 1The input text prompt for the model to generate a response. guided\_json{} objectJSON schema that should be fulfilled for the response. ▶response\_format{} object raw booleandefault: falseIf true, a chat template is not applied and you must adhere to the specific model's expected formatting. stream booleandefault: falseIf true, the response will be streamed back incrementally using SSE, Server Sent Events. max\_tokens integerdefault: 256The maximum number of tokens to generate in the response. temperature numberdefault: 0.15minimum: 0maximum: 5Controls the randomness of the output; higher values produce more random results. top\_p numberminimum: 0maximum: 2Adjusts the creativity of the AI's responses by controlling how many possible words it considers. Lower values make outputs more predictable; higher values allow for more varied and creative responses. top\_k integerminimum: 1maximum: 50Limits the AI to choose from the top 'k' most probable words. Lower values make responses more focused; higher values introduce more variety and potential surprises. seed integerminimum: 1maximum: 9999999999Random seed for reproducibility of the generation. repetition\_penalty numberminimum: 0maximum: 2Penalty for repeated tokens; higher values discourage repetition. frequency\_penalty numberminimum: 0maximum: 2Decreases the likelihood of the model repeating the same lines verbatim. presence\_penalty numberminimum: 0maximum: 2Increases the likelihood of the model introducing new topics. response stringThe generated text response from the model ▶usage{} objectUsage statistics for the inference request ▶tool\_calls\[] arrayAn array of tool calls requests made during the response generation
Streaming — Send a request with `stream: true` and receive server-sent events prompt stringrequiredminLength: 1The input text prompt for the model to generate a response. guided\_json{} objectJSON schema that should be fulfilled for the response. ▶response\_format{} object raw booleandefault: falseIf true, a chat template is not applied and you must adhere to the specific model's expected formatting. stream booleandefault: falseIf true, the response will be streamed back incrementally using SSE, Server Sent Events. max\_tokens integerdefault: 256The maximum number of tokens to generate in the response. temperature numberdefault: 0.15minimum: 0maximum: 5Controls the randomness of the output; higher values produce more random results. top\_p numberminimum: 0maximum: 2Adjusts the creativity of the AI's responses by controlling how many possible words it considers. Lower values make outputs more predictable; higher values allow for more varied and creative responses. top\_k integerminimum: 1maximum: 50Limits the AI to choose from the top 'k' most probable words. Lower values make responses more focused; higher values introduce more variety and potential surprises. seed integerminimum: 1maximum: 9999999999Random seed for reproducibility of the generation. repetition\_penalty numberminimum: 0maximum: 2Penalty for repeated tokens; higher values discourage repetition. frequency\_penalty numberminimum: 0maximum: 2Decreases the likelihood of the model repeating the same lines verbatim. presence\_penalty numberminimum: 0maximum: 2Increases the likelihood of the model introducing new topics. type string contentType text/event-stream format binary
Batch — Send multiple requests in a single API call ▶requests\[] arrayrequired response stringThe generated text response from the model ▶usage{} objectUsage statistics for the inference request ▶tool\_calls\[] arrayAn array of tool calls requests made during the response generation
## API Schemas (Raw) SynchronousInput [Open](https://developers.cloudflare.com/workers-ai/models/llama-4-scout-17b-16e-instruct/sync-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/llama-4-scout-17b-16e-instruct/sync-input.json) SynchronousOutput [Open](https://developers.cloudflare.com/workers-ai/models/llama-4-scout-17b-16e-instruct/sync-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/llama-4-scout-17b-16e-instruct/sync-output.json) StreamingInput [Open](https://developers.cloudflare.com/workers-ai/models/llama-4-scout-17b-16e-instruct/streaming-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/llama-4-scout-17b-16e-instruct/streaming-input.json) StreamingOutput [Open](https://developers.cloudflare.com/workers-ai/models/llama-4-scout-17b-16e-instruct/streaming-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/llama-4-scout-17b-16e-instruct/streaming-output.json) BatchInput [Open](https://developers.cloudflare.com/workers-ai/models/llama-4-scout-17b-16e-instruct/batch-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/llama-4-scout-17b-16e-instruct/batch-input.json) BatchOutput [Open](https://developers.cloudflare.com/workers-ai/models/llama-4-scout-17b-16e-instruct/batch-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/llama-4-scout-17b-16e-instruct/batch-output.json) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/models/llama-4-scout-17b-16e-instruct/#page","headline":"llama-4-scout-17b-16e-instruct","description":"Meta's Llama 4 Scout is a 17 billion parameter model with 16 experts that is natively multimodal. These models leverage a mixture-of-experts architecture to offer industry-leading performance in text and image understanding.","url":"https://developers.cloudflare.com/workers-ai/models/llama-4-scout-17b-16e-instruct/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Llama Guard 3 is a Llama-3.1-8B pretrained model, fine-tuned for content safety classification. Similar to previous versions, it can be used to classify content in both LLM inputs (prompt classification) and in LLM responses (response classification). It acts as an LLM – it generates text in its output that indicates whether a given prompt or response is safe or unsafe, and if unsafe, it also lists the content categories violated. title: llama-guard-3-8b image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/llms.txt > Use this file to discover all available pages before exploring further. ![Meta logo](https://developers.cloudflare.com/_astro/meta.CTzB_ysm.svg) # llama-guard-3-8b Text Generation • Meta Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/models/llama-guard-3-8b/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Model ID `@cf/meta/llama-guard-3-8b` Copy - Cloudflare-hosted - LoRA - Zero data retention Llama Guard 3 is a Llama-3.1-8B pretrained model, fine-tuned for content safety classification. Similar to previous versions, it can be used to classify content in both LLM inputs (prompt classification) and in LLM responses (response classification). It acts as an LLM – it generates text in its output that indicates whether a given prompt or response is safe or unsafe, and if unsafe, it also lists the content categories violated. | Model Info | | | --- | --- | | Context Window [ ↗](https://developers.cloudflare.com/workers-ai/platform/glossary/) | 131,072 tokens | | Zero data retention | Yes | | LoRA | Yes | | Unit Pricing | $0.484 per M input tokens, $0.03 per M output tokens | ## Playground Try out this model with Workers AI LLM Playground. It does not require any setup or authentication and is an instant way to preview and test a model directly in the browser. [Launch the LLM Playground](https://playground.ai.cloudflare.com/?model=@cf/meta/llama-guard-3-8b) ## Usage ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const messages = [ { role: "system", content: "You are a friendly assistant" }, { role: "user", content: "What is the origin of the phrase Hello, World", }, ]; const stream = await env.AI.run("@cf/meta/llama-guard-3-8b", { messages, stream: true, }); return new Response(stream, { headers: { "content-type": "text/event-stream" }, }); }, } satisfies ExportedHandler; ``` ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const messages = [ { role: "system", content: "You are a friendly assistant" }, { role: "user", content: "What is the origin of the phrase Hello, World", }, ]; const response = await env.AI.run("@cf/meta/llama-guard-3-8b", { messages }); return Response.json(response); }, } satisfies ExportedHandler; ``` ```py import os import requests ACCOUNT_ID = "your-account-id" AUTH_TOKEN = os.environ.get("CLOUDFLARE_AUTH_TOKEN") prompt = "Tell me all about PEP-8" response = requests.post( f"https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai/run/@cf/meta/llama-guard-3-8b", headers={"Authorization": f"Bearer {AUTH_TOKEN}"}, json={ "messages": [ {"role": "system", "content": "You are a friendly assistant"}, {"role": "user", "content": prompt} ] } ) result = response.json() print(result) ``` ```sh curl https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/run/@cf/meta/llama-guard-3-8b \ -X POST \ -H "Authorization: Bearer $CLOUDFLARE_AUTH_TOKEN" \ -d '{ "messages": [{ "role": "system", "content": "You are a friendly assistant" }, { "role": "user", "content": "Why is pizza so good" }]}' ``` OpenAI compatible endpoints Workers AI also supports OpenAI compatible API endpoints for `/v1/chat/completions` and `/v1/embeddings`. For more details, refer to [Configurations](https://developers.cloudflare.com/workers-ai/configuration/open-ai-compatibility/) . ## Parameters ▶messages\[] `array`requiredAn array of message objects representing the conversation history. max\_tokens `integer`default: 256The maximum number of tokens to generate in the response. temperature `number`default: 0.6minimum: 0maximum: 5Controls the randomness of the output; higher values produce more random results. ▶response\_format{} `object`Dictate the output format of the generated response. ▶response `one of` ▶usage{} `object`Usage statistics for the inference request ## API Schemas (Raw) Input [Open](https://developers.cloudflare.com/workers-ai/models/llama-guard-3-8b/schema-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/llama-guard-3-8b/schema-input.json) Output [Open](https://developers.cloudflare.com/workers-ai/models/llama-guard-3-8b/schema-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/llama-guard-3-8b/schema-output.json) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/models/llama-guard-3-8b/#page","headline":"llama-guard-3-8b","description":"Llama Guard 3 is a Llama-3.1-8B pretrained model, fine-tuned for content safety classification. Similar to previous versions, it can be used to classify content in both LLM inputs (prompt classification) and in LLM responses (response classification). It acts as an LLM – it generates text in its output that indicates whether a given prompt or response is safe or unsafe, and if unsafe, it also lists the content categories violated.","url":"https://developers.cloudflare.com/workers-ai/models/llama-guard-3-8b/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Multilingual encoder-decoder (seq-to-seq) model trained for Many-to-Many multilingual translation title: m2m100-1.2b image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/llms.txt > Use this file to discover all available pages before exploring further. ![Meta logo](https://developers.cloudflare.com/_astro/meta.CTzB_ysm.svg) # m2m100-1.2b Translation • Meta Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/models/m2m100-1.2b/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Model ID `@cf/meta/m2m100-1.2b` Copy - Cloudflare-hosted - Batch - Zero data retention Multilingual encoder-decoder (seq-to-seq) model trained for Many-to-Many multilingual translation | Model Info | | | --- | --- | | Terms and License | [link ↗](https://github.com/facebookresearch/fairseq/blob/main/LICENSE) | | More information | [link ↗](https://github.com/facebookresearch/fairseq/tree/main/examples/m2m_100) | | Zero data retention | Yes | | Batch | Yes | | Unit Pricing | $0.342 per M input tokens, $0.342 per M output tokens | ## Usage ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const response = await env.AI.run( "@cf/meta/m2m100-1.2b", { text: "I'll have an order of the moule frites", source_lang: "english", // defaults to english target_lang: "french", } ); return new Response(JSON.stringify(response)); }, } satisfies ExportedHandler; ``` ```py import requests API_BASE_URL = "https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai/run/" headers = {"Authorization": "Bearer {API_TOKEN}"} def run(model, input): response = requests.post(f"{API_BASE_URL}{model}", headers=headers, json=input) return response.json() output = run('@cf/meta/m2m100-1.2b', { "text": "I'll have an order of the moule frites", "source_lang": "english", "target_lang": "french" }) print(output) ``` ```sh curl https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/run/@cf/meta/m2m100-1.2b \ -X POST \ -H "Authorization: Bearer $CLOUDFLARE_API_TOKEN" \ -d '{ "text": "Ill have an order of the moule frites", "source_lang": "english", "target_lang": "french" }' ``` ## Parameters
Synchronous — Send a request and receive a complete response text stringrequiredminLength: 1The text to be translated source\_lang stringdefault: enThe language code of the source text (e.g., 'en' for English). Defaults to 'en' if not specified target\_lang stringrequiredThe language code to translate the text into (e.g., 'es' for Spanish) request\_id stringThe async request id that can be used to obtain the results.
Batch — Send multiple requests in a single API call ▶requests\[] arrayrequiredBatch of the embeddings requests to run using async-queue request\_id stringThe async request id that can be used to obtain the results.
## API Schemas (Raw) SynchronousInput [Open](https://developers.cloudflare.com/workers-ai/models/m2m100-1.2b/sync-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/m2m100-1.2b/sync-input.json) SynchronousOutput [Open](https://developers.cloudflare.com/workers-ai/models/m2m100-1.2b/sync-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/m2m100-1.2b/sync-output.json) BatchInput [Open](https://developers.cloudflare.com/workers-ai/models/m2m100-1.2b/batch-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/m2m100-1.2b/batch-input.json) BatchOutput [Open](https://developers.cloudflare.com/workers-ai/models/m2m100-1.2b/batch-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/m2m100-1.2b/batch-output.json) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/models/m2m100-1.2b/#page","headline":"m2m100-1.2b","description":"Multilingual encoder-decoder (seq-to-seq) model trained for Many-to-Many multilingual translation","url":"https://developers.cloudflare.com/workers-ai/models/m2m100-1.2b/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: 50 layers deep image classification CNN trained on more than 1M images from ImageNet title: resnet-50 image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/llms.txt > Use this file to discover all available pages before exploring further. ![Microsoft logo](https://developers.cloudflare.com/_astro/microsoft.LujcDJ--.svg) # resnet-50 Image Classification • Microsoft Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/models/resnet-50/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Model ID `@cf/microsoft/resnet-50` Copy - Cloudflare-hosted - Zero data retention 50 layers deep image classification CNN trained on more than 1M images from ImageNet | Model Info | | | --- | --- | | More information | [link ↗](https://www.microsoft.com/en-us/research/blog/microsoft-vision-model-resnet-50-combines-web-scale-data-and-multi-task-learning-to-achieve-state-of-the-art/) | | Zero data retention | Yes | | Unit Pricing | $0.00000251 per inference request | ## Parameters Option 1 stringformat: binary The image to classify ▶Option 2{} object type `array` contentType `application/json` items `[object Object]` ## API Schemas (Raw) Input [Open](https://developers.cloudflare.com/workers-ai/models/resnet-50/schema-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/resnet-50/schema-input.json) Output [Open](https://developers.cloudflare.com/workers-ai/models/resnet-50/schema-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/resnet-50/schema-output.json) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/models/resnet-50/#page","headline":"resnet-50","description":"50 layers deep image classification CNN trained on more than 1M images from ImageNet","url":"https://developers.cloudflare.com/workers-ai/models/resnet-50/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: The Mistral-7B-Instruct-v0.2 Large Language Model (LLM) is an instruct fine-tuned version of the Mistral-7B-v0.2. title: mistral-7b-instruct-v0.2-lora image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/llms.txt > Use this file to discover all available pages before exploring further. ![MistralAI logo](https://developers.cloudflare.com/_astro/mistralai.Bn9UMUMu.svg) # mistral-7b-instruct-v0.2-lora Text Generation • MistralAI Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/models/mistral-7b-instruct-v0.2-lora/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Model ID `@cf/mistral/mistral-7b-instruct-v0.2-lora` Copy - Cloudflare-hosted - LoRA - Zero data retention The Mistral-7B-Instruct-v0.2 Large Language Model (LLM) is an instruct fine-tuned version of the Mistral-7B-v0.2. | Model Info | | | --- | --- | | Context Window [ ↗](https://developers.cloudflare.com/workers-ai/platform/glossary/) | 15,000 tokens | | Zero data retention | Yes | | LoRA | Yes | ## Playground Try out this model with Workers AI LLM Playground. It does not require any setup or authentication and is an instant way to preview and test a model directly in the browser. [Launch the LLM Playground](https://playground.ai.cloudflare.com/?model=@cf/mistral/mistral-7b-instruct-v0.2-lora) ## Usage ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const messages = [ { role: "system", content: "You are a friendly assistant" }, { role: "user", content: "What is the origin of the phrase Hello, World", }, ]; const stream = await env.AI.run("@cf/mistral/mistral-7b-instruct-v0.2-lora", { messages, stream: true, }); return new Response(stream, { headers: { "content-type": "text/event-stream" }, }); }, } satisfies ExportedHandler; ``` ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const messages = [ { role: "system", content: "You are a friendly assistant" }, { role: "user", content: "What is the origin of the phrase Hello, World", }, ]; const response = await env.AI.run("@cf/mistral/mistral-7b-instruct-v0.2-lora", { messages }); return Response.json(response); }, } satisfies ExportedHandler; ``` ```py import os import requests ACCOUNT_ID = "your-account-id" AUTH_TOKEN = os.environ.get("CLOUDFLARE_AUTH_TOKEN") prompt = "Tell me all about PEP-8" response = requests.post( f"https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai/run/@cf/mistral/mistral-7b-instruct-v0.2-lora", headers={"Authorization": f"Bearer {AUTH_TOKEN}"}, json={ "messages": [ {"role": "system", "content": "You are a friendly assistant"}, {"role": "user", "content": prompt} ] } ) result = response.json() print(result) ``` ```sh curl https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/run/@cf/mistral/mistral-7b-instruct-v0.2-lora \ -X POST \ -H "Authorization: Bearer $CLOUDFLARE_AUTH_TOKEN" \ -d '{ "messages": [{ "role": "system", "content": "You are a friendly assistant" }, { "role": "user", "content": "Why is pizza so good" }]}' ``` OpenAI compatible endpoints Workers AI also supports OpenAI compatible API endpoints for `/v1/chat/completions` and `/v1/embeddings`. For more details, refer to [Configurations](https://developers.cloudflare.com/workers-ai/configuration/open-ai-compatibility/) . ## Parameters ### Input prompt `string`requiredminLength: 1The input text prompt for the model to generate a response. lora `string`Name of the LoRA (Low-Rank Adaptation) model to fine-tune the base model. ▶response\_format{} `object` raw `boolean`default: falseIf true, a chat template is not applied and you must adhere to the specific model's expected formatting. stream `boolean`default: falseIf true, the response will be streamed back incrementally using SSE, Server Sent Events. max\_tokens `integer`default: 256The maximum number of tokens to generate in the response. temperature `number`default: 0.6minimum: 0maximum: 5Controls the randomness of the output; higher values produce more random results. top\_p `number`minimum: 0.001maximum: 1Adjusts the creativity of the AI's responses by controlling how many possible words it considers. Lower values make outputs more predictable; higher values allow for more varied and creative responses. top\_k `integer`minimum: 1maximum: 50Limits the AI to choose from the top 'k' most probable words. Lower values make responses more focused; higher values introduce more variety and potential surprises. seed `integer`minimum: 1maximum: 9999999999Random seed for reproducibility of the generation. repetition\_penalty `number`minimum: 0maximum: 2Penalty for repeated tokens; higher values discourage repetition. frequency\_penalty `number`minimum: -2maximum: 2Decreases the likelihood of the model repeating the same lines verbatim. presence\_penalty `number`minimum: -2maximum: 2Increases the likelihood of the model introducing new topics. ### Output
Synchronous — Send a request and receive a complete response response stringThe generated text response from the model ▶usage{} objectUsage statistics for the inference request ▶tool\_calls\[] arrayAn array of tool calls requests made during the response generation
Streaming — Send a request with `stream: true` and receive server-sent events type string format binary
## API Schemas (Raw) SynchronousInput [Open](https://developers.cloudflare.com/workers-ai/models/mistral-7b-instruct-v0.2-lora/sync-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/mistral-7b-instruct-v0.2-lora/sync-input.json) SynchronousOutput [Open](https://developers.cloudflare.com/workers-ai/models/mistral-7b-instruct-v0.2-lora/sync-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/mistral-7b-instruct-v0.2-lora/sync-output.json) StreamingInput [Open](https://developers.cloudflare.com/workers-ai/models/mistral-7b-instruct-v0.2-lora/streaming-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/mistral-7b-instruct-v0.2-lora/streaming-input.json) StreamingOutput [Open](https://developers.cloudflare.com/workers-ai/models/mistral-7b-instruct-v0.2-lora/streaming-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/mistral-7b-instruct-v0.2-lora/streaming-output.json) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/models/mistral-7b-instruct-v0.2-lora/#page","headline":"mistral-7b-instruct-v0.2-lora","description":"The Mistral-7B-Instruct-v0.2 Large Language Model (LLM) is an instruct fine-tuned version of the Mistral-7B-v0.2.","url":"https://developers.cloudflare.com/workers-ai/models/mistral-7b-instruct-v0.2-lora/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Building upon Mistral Small 3 (2501), Mistral Small 3.1 (2503) adds state-of-the-art vision understanding and enhances long context capabilities up to 128k tokens without compromising text performance. With 24 billion parameters, this model achieves top-tier capabilities in both text and vision tasks. title: mistral-small-3.1-24b-instruct image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/llms.txt > Use this file to discover all available pages before exploring further. ![MistralAI logo](https://developers.cloudflare.com/_astro/mistralai.Bn9UMUMu.svg) # mistral-small-3.1-24b-instruct Text Generation • MistralAI Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/models/mistral-small-3.1-24b-instruct/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Model ID `@cf/mistralai/mistral-small-3.1-24b-instruct` Copy - Cloudflare-hosted - Function calling - Zero data retention Building upon Mistral Small 3 (2501), Mistral Small 3.1 (2503) adds state-of-the-art vision understanding and enhances long context capabilities up to 128k tokens without compromising text performance. With 24 billion parameters, this model achieves top-tier capabilities in both text and vision tasks. | Model Info | | | --- | --- | | Context Window [ ↗](https://developers.cloudflare.com/workers-ai/platform/glossary/) | 128,000 tokens | | Function calling [↗](https://developers.cloudflare.com/workers-ai/function-calling/) | Yes | | Zero data retention | Yes | | Unit Pricing | $0.351 per M input tokens, $0.555 per M output tokens | ## Playground Try out this model with Workers AI LLM Playground. It does not require any setup or authentication and is an instant way to preview and test a model directly in the browser. [Launch the LLM Playground](https://playground.ai.cloudflare.com/?model=@cf/mistralai/mistral-small-3.1-24b-instruct) ## Usage ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const messages = [ { role: "system", content: "You are a friendly assistant" }, { role: "user", content: "What is the origin of the phrase Hello, World", }, ]; const stream = await env.AI.run("@cf/mistralai/mistral-small-3.1-24b-instruct", { messages, stream: true, }); return new Response(stream, { headers: { "content-type": "text/event-stream" }, }); }, } satisfies ExportedHandler; ``` ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const messages = [ { role: "system", content: "You are a friendly assistant" }, { role: "user", content: "What is the origin of the phrase Hello, World", }, ]; const response = await env.AI.run("@cf/mistralai/mistral-small-3.1-24b-instruct", { messages }); return Response.json(response); }, } satisfies ExportedHandler; ``` ```py import os import requests ACCOUNT_ID = "your-account-id" AUTH_TOKEN = os.environ.get("CLOUDFLARE_AUTH_TOKEN") prompt = "Tell me all about PEP-8" response = requests.post( f"https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai/run/@cf/mistralai/mistral-small-3.1-24b-instruct", headers={"Authorization": f"Bearer {AUTH_TOKEN}"}, json={ "messages": [ {"role": "system", "content": "You are a friendly assistant"}, {"role": "user", "content": prompt} ] } ) result = response.json() print(result) ``` ```sh curl https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/run/@cf/mistralai/mistral-small-3.1-24b-instruct \ -X POST \ -H "Authorization: Bearer $CLOUDFLARE_AUTH_TOKEN" \ -d '{ "messages": [{ "role": "system", "content": "You are a friendly assistant" }, { "role": "user", "content": "Why is pizza so good" }]}' ``` OpenAI compatible endpoints Workers AI also supports OpenAI compatible API endpoints for `/v1/chat/completions` and `/v1/embeddings`. For more details, refer to [Configurations](https://developers.cloudflare.com/workers-ai/configuration/open-ai-compatibility/) . ## Parameters ### Input prompt `string`requiredminLength: 1The input text prompt for the model to generate a response. guided\_json{} `object`JSON schema that should be fulfilled for the response. raw `boolean`default: falseIf true, a chat template is not applied and you must adhere to the specific model's expected formatting. stream `boolean`default: falseIf true, the response will be streamed back incrementally using SSE, Server Sent Events. max\_tokens `integer`default: 256The maximum number of tokens to generate in the response. temperature `number`default: 0.15minimum: 0maximum: 5Controls the randomness of the output; higher values produce more random results. top\_p `number`minimum: 0maximum: 2Adjusts the creativity of the AI's responses by controlling how many possible words it considers. Lower values make outputs more predictable; higher values allow for more varied and creative responses. top\_k `integer`minimum: 1maximum: 50Limits the AI to choose from the top 'k' most probable words. Lower values make responses more focused; higher values introduce more variety and potential surprises. seed `integer`minimum: 1maximum: 9999999999Random seed for reproducibility of the generation. repetition\_penalty `number`minimum: 0maximum: 2Penalty for repeated tokens; higher values discourage repetition. frequency\_penalty `number`minimum: 0maximum: 2Decreases the likelihood of the model repeating the same lines verbatim. presence\_penalty `number`minimum: 0maximum: 2Increases the likelihood of the model introducing new topics. ### Output
Synchronous — Send a request and receive a complete response response stringThe generated text response from the model ▶usage{} objectUsage statistics for the inference request ▶tool\_calls\[] arrayAn array of tool calls requests made during the response generation
Streaming — Send a request with `stream: true` and receive server-sent events type string contentType text/event-stream format binary
## API Schemas (Raw) SynchronousInput [Open](https://developers.cloudflare.com/workers-ai/models/mistral-small-3.1-24b-instruct/sync-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/mistral-small-3.1-24b-instruct/sync-input.json) SynchronousOutput [Open](https://developers.cloudflare.com/workers-ai/models/mistral-small-3.1-24b-instruct/sync-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/mistral-small-3.1-24b-instruct/sync-output.json) StreamingInput [Open](https://developers.cloudflare.com/workers-ai/models/mistral-small-3.1-24b-instruct/streaming-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/mistral-small-3.1-24b-instruct/streaming-input.json) StreamingOutput [Open](https://developers.cloudflare.com/workers-ai/models/mistral-small-3.1-24b-instruct/streaming-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/mistral-small-3.1-24b-instruct/streaming-output.json) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/models/mistral-small-3.1-24b-instruct/#page","headline":"mistral-small-3.1-24b-instruct","description":"Building upon Mistral Small 3 (2501), Mistral Small 3.1 (2503) adds state-of-the-art vision understanding and enhances long context capabilities up to 128k tokens without compromising text performance. With 24 billion parameters, this model achieves top-tier capabilities in both text and vision tasks.","url":"https://developers.cloudflare.com/workers-ai/models/mistral-small-3.1-24b-instruct/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Moondream 3 is a fast, efficient 9B mixture-of-experts vision language model (2B active parameters) that delivers frontier-level visual reasoning for tasks like object detection, pointing, OCR, and structured output. title: moondream3.1-9B-A2B image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/llms.txt > Use this file to discover all available pages before exploring further. m # moondream3.1-9B-A2B Image-to-Text • moondream Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/models/moondream3.1-9B-A2B/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Model ID `@cf/moondream/moondream3.1-9B-A2B` Copy - Cloudflare-hosted - Vision - Zero data retention Moondream 3 is a fast, efficient 9B mixture-of-experts vision language model (2B active parameters) that delivers frontier-level visual reasoning for tasks like object detection, pointing, OCR, and structured output. | Model Info | | | --- | --- | | Terms and License | [link ↗](https://moondream.ai/licenses/model/1.0) | | Vision | Yes | | Zero data retention | Yes | | Unit Pricing | $0.30 per M input tokens, $1.00 per M output tokens | ## Parameters task `string`default: queryenum: query, caption, point, detectWhich Moondream skill to run. image `string`Input image as a public HTTPS URL or base64 data URI. Optional for \`query\`; required for \`caption\`, \`point\`, and \`detect\`. question `string`default: What's in this image?Question for the \`query\` task. caption\_length `string`default: normalenum: short, normal, longCaption length for the \`caption\` task. target `string`default: personObject phrase to locate for \`point\` and \`detect\` tasks (e.g. 'person wearing a red shirt'). reasoning `boolean`default: trueEnable reasoning trace for the \`query\` task. temperature `number`default: 0.2minimum: 0maximum: 2Sampling temperature. top\_p `number`default: 0.9minimum: 0maximum: 1Top-p (nucleus) sampling. max\_tokens `integer`default: 8192minimum: 1maximum: 28672Max tokens to generate for \`query\` and \`caption\`. max\_objects `integer`default: 150minimum: 1maximum: 500Max objects to return for \`point\` and \`detect\`. stream `boolean`default: falseReturn incremental tokens for \`query\` and \`caption\`. \`point\` and \`detect\` do not support streaming. finish\_reason `string`Reason the generation finished. ▶metrics{} `object` answer `string`Answer text for the \`query\` task. Null for other tasks. caption `string`Caption text for the \`caption\` task. Null for other tasks. ▶points\[] `array`Located points for the \`point\` task. Null for other tasks. ▶objects\[] `array`Detected bounding boxes for the \`detect\` task. Null for other tasks. ▶reasoning{} `object`Reasoning trace for the \`query\` task when reasoning=true. Null otherwise. ## API Schemas (Raw) Input [Open](https://developers.cloudflare.com/workers-ai/models/moondream3.1-9B-A2B/schema-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/moondream3.1-9B-A2B/schema-input.json) Output [Open](https://developers.cloudflare.com/workers-ai/models/moondream3.1-9B-A2B/schema-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/moondream3.1-9B-A2B/schema-output.json) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/models/moondream3.1-9B-A2B/#page","headline":"moondream3.1-9B-A2B","description":"Moondream 3 is a fast, efficient 9B mixture-of-experts vision language model (2B active parameters) that delivers frontier-level visual reasoning for tasks like object detection, pointing, OCR, and structured output.","url":"https://developers.cloudflare.com/workers-ai/models/moondream3.1-9B-A2B/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Kimi K2.6 is a frontier-scale open-source 1T parameter model with a 262.1k context window, multi-turn tool calling, vision inputs, and structured outputs for agentic workloads. title: kimi-k2.6 image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/llms.txt > Use this file to discover all available pages before exploring further. ![Moonshot AI logo](https://developers.cloudflare.com/_astro/moonshotai.DjWMkXUS.svg) # kimi-k2.6 Text Generation • Moonshot AI Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/models/kimi-k2.6/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Model ID `@cf/moonshotai/kimi-k2.6` Copy - Cloudflare-hosted - Batch - Function calling - Reasoning - Vision - Planned deprecation - Zero data retention Kimi K2.6 is a frontier-scale open-source 1T parameter model with a 262.1k context window, multi-turn tool calling, vision inputs, and structured outputs for agentic workloads. Paid access required This model is not available through standard Workers Free billing. To use it, upgrade to the [Workers Paid plan](https://developers.cloudflare.com/workers/platform/pricing/#workers) or use prepaid [AI Gateway credits](https://developers.cloudflare.com/ai-gateway/features/unified-billing/). | Model Info | | | --- | --- | | Planned Deprecation | 11/20/2026 | | Context Window [ ↗](https://developers.cloudflare.com/workers-ai/platform/glossary/) | 262,144 tokens | | Terms and License | [link ↗](https://huggingface.co/moonshotai/Kimi-K2.6/blob/main/LICENSE) | | Function calling [↗](https://developers.cloudflare.com/workers-ai/function-calling/) | Yes | | Reasoning | `none``high` (default) | | Vision | Yes | | Zero data retention | Yes | | Batch | Yes | | Unit Pricing | $0.95 per M input tokens, $4.00 per M output tokens, $0.16 per M cached input tokens | ## Playground Try out this model with Workers AI LLM Playground. It does not require any setup or authentication and is an instant way to preview and test a model directly in the browser. [Launch the LLM Playground](https://playground.ai.cloudflare.com/?model=@cf/moonshotai/kimi-k2.6) ## Usage ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const messages = [ { role: "system", content: "You are a friendly assistant" }, { role: "user", content: "What is the origin of the phrase Hello, World", }, ]; const stream = await env.AI.run("@cf/moonshotai/kimi-k2.6", { messages, stream: true, }); return new Response(stream, { headers: { "content-type": "text/event-stream" }, }); }, } satisfies ExportedHandler; ``` ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const messages = [ { role: "system", content: "You are a friendly assistant" }, { role: "user", content: "What is the origin of the phrase Hello, World", }, ]; const response = await env.AI.run("@cf/moonshotai/kimi-k2.6", { messages }); return Response.json(response); }, } satisfies ExportedHandler; ``` ```py import os import requests ACCOUNT_ID = "your-account-id" AUTH_TOKEN = os.environ.get("CLOUDFLARE_AUTH_TOKEN") prompt = "Tell me all about PEP-8" response = requests.post( f"https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai/run/@cf/moonshotai/kimi-k2.6", headers={"Authorization": f"Bearer {AUTH_TOKEN}"}, json={ "messages": [ {"role": "system", "content": "You are a friendly assistant"}, {"role": "user", "content": prompt} ] } ) result = response.json() print(result) ``` ```sh curl https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/run/@cf/moonshotai/kimi-k2.6 \ -X POST \ -H "Authorization: Bearer $CLOUDFLARE_AUTH_TOKEN" \ -d '{ "messages": [{ "role": "system", "content": "You are a friendly assistant" }, { "role": "user", "content": "Why is pizza so good" }]}' ``` OpenAI compatible endpoints Workers AI also supports OpenAI compatible API endpoints for `/v1/chat/completions` and `/v1/embeddings`. For more details, refer to [Configurations](https://developers.cloudflare.com/workers-ai/configuration/open-ai-compatibility/) . ## Parameters
Synchronous — Send a request and receive a complete response Input format Prompt Simple text input for single-turn interactionsMessages Structured conversation format with roles (user, assistant, system) prompt stringrequiredminLength: 1The input text prompt for the model to generate a response. model stringID of the model to use (e.g. '@cf/zai-org/glm-4.7-flash, etc'). ▶audio{} objectParameters for audio output. Required when modalities includes 'audio'. frequency\_penalty number | nullPenalizes new tokens based on their existing frequency in the text so far. logit\_bias object | nullModify the likelihood of specified tokens appearing in the completion. Maps token IDs to bias values from -100 to 100. logprobs boolean | nullWhether to return log probabilities of the output tokens. top\_logprobs integer | nullHow many top log probabilities to return at each token position (0-20). Requires logprobs=true. max\_tokens integer | nullDeprecated in favor of max_completion_tokens. The maximum number of tokens to generate. max\_completion\_tokens integer | nullAn upper bound for the number of tokens that can be generated for a completion. metadata object | nullSet of 16 key-value pairs that can be attached to the object. modalities array | nullOutput types requested from the model (e.g. ['text'] or ['text', 'audio']). n integer | nullHow many chat completion choices to generate for each input message. parallel\_tool\_calls booleandefault: trueWhether to enable parallel function calling during tool use. ▶prediction{} object presence\_penalty number | nullPenalizes new tokens based on whether they appear in the text so far. reasoning\_effort string | nullenum: high, noneReasoning effort. Supported levels: high, none. Compatibility aliases: low maps to high; medium maps to high; max maps to high. ▶chat\_template\_kwargs{} object ▶response\_format one ofSpecifies the format the model must output. seed integer | nullIf specified, the system will make a best effort to sample deterministically. ▶stop one of store boolean | nullWhether to store the output for model distillation / evals. stream boolean | nullIf true, partial message deltas will be sent as server-sent events. ▶stream\_options{} object temperature number | nullSampling temperature between 0 and 2. ▶tool\_choice one ofControls which (if any) tool is called by the model. 'none' = no tools, 'auto' = model decides, 'required' = must call a tool. ▶tools\[] arrayA list of tools the model may call. top\_p number | nullNucleus sampling: considers the results of the tokens with top_p probability mass. user stringA unique identifier representing your end-user, for abuse monitoring. ▶web\_search\_options{} objectOptions for the web search tool (when using built-in web search). ▶function\_call one of ▶functions\[] arrayminItems: 1maxItems: 128 id stringA unique identifier for the chat completion. object string created integerUnix timestamp (seconds) of when the completion was created. model stringThe model used for the chat completion. ▶choices\[] arrayminItems: 1 ▶usage{} object system\_fingerprint string | null
Streaming — Send a request with `stream: true` and receive server-sent events Input format Prompt Simple text input for single-turn interactionsMessages Structured conversation format with roles (user, assistant, system) prompt stringrequiredminLength: 1The input text prompt for the model to generate a response. model stringID of the model to use (e.g. '@cf/zai-org/glm-4.7-flash, etc'). ▶audio{} objectParameters for audio output. Required when modalities includes 'audio'. frequency\_penalty number | nullPenalizes new tokens based on their existing frequency in the text so far. logit\_bias object | nullModify the likelihood of specified tokens appearing in the completion. Maps token IDs to bias values from -100 to 100. logprobs boolean | nullWhether to return log probabilities of the output tokens. top\_logprobs integer | nullHow many top log probabilities to return at each token position (0-20). Requires logprobs=true. max\_tokens integer | nullDeprecated in favor of max_completion_tokens. The maximum number of tokens to generate. max\_completion\_tokens integer | nullAn upper bound for the number of tokens that can be generated for a completion. metadata object | nullSet of 16 key-value pairs that can be attached to the object. modalities array | nullOutput types requested from the model (e.g. ['text'] or ['text', 'audio']). n integer | nullHow many chat completion choices to generate for each input message. parallel\_tool\_calls booleandefault: trueWhether to enable parallel function calling during tool use. ▶prediction{} object presence\_penalty number | nullPenalizes new tokens based on whether they appear in the text so far. reasoning\_effort string | nullenum: high, noneReasoning effort. Supported levels: high, none. Compatibility aliases: low maps to high; medium maps to high; max maps to high. ▶chat\_template\_kwargs{} object ▶response\_format one ofSpecifies the format the model must output. seed integer | nullIf specified, the system will make a best effort to sample deterministically. ▶stop one of store boolean | nullWhether to store the output for model distillation / evals. stream boolean | nullIf true, partial message deltas will be sent as server-sent events. ▶stream\_options{} object temperature number | nullSampling temperature between 0 and 2. ▶tool\_choice one ofControls which (if any) tool is called by the model. 'none' = no tools, 'auto' = model decides, 'required' = must call a tool. ▶tools\[] arrayA list of tools the model may call. top\_p number | nullNucleus sampling: considers the results of the tokens with top_p probability mass. user stringA unique identifier representing your end-user, for abuse monitoring. ▶web\_search\_options{} objectOptions for the web search tool (when using built-in web search). ▶function\_call one of ▶functions\[] arrayminItems: 1maxItems: 128 type string contentType text/event-stream format binary
Batch — Send multiple requests in a single API call ▶requests\[] array id stringA unique identifier for the chat completion. object string created integerUnix timestamp (seconds) of when the completion was created. model stringThe model used for the chat completion. ▶choices\[] arrayminItems: 1 ▶usage{} object system\_fingerprint string | null
## API Schemas (Raw) SynchronousInput [Open](https://developers.cloudflare.com/workers-ai/models/kimi-k2.6/sync-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/kimi-k2.6/sync-input.json) SynchronousOutput [Open](https://developers.cloudflare.com/workers-ai/models/kimi-k2.6/sync-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/kimi-k2.6/sync-output.json) StreamingInput [Open](https://developers.cloudflare.com/workers-ai/models/kimi-k2.6/streaming-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/kimi-k2.6/streaming-input.json) StreamingOutput [Open](https://developers.cloudflare.com/workers-ai/models/kimi-k2.6/streaming-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/kimi-k2.6/streaming-output.json) BatchInput [Open](https://developers.cloudflare.com/workers-ai/models/kimi-k2.6/batch-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/kimi-k2.6/batch-input.json) BatchOutput [Open](https://developers.cloudflare.com/workers-ai/models/kimi-k2.6/batch-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/kimi-k2.6/batch-output.json) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/models/kimi-k2.6/#page","headline":"kimi-k2.6","description":"Kimi K2.6 is a frontier-scale open-source 1T parameter model with a 262.1k context window, multi-turn tool calling, vision inputs, and structured outputs for agentic workloads.","url":"https://developers.cloudflare.com/workers-ai/models/kimi-k2.6/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Kimi K2.7 is a frontier-scale open-source 1T parameter model with a 262.1k context window, multi-turn tool calling, vision inputs, and structured outputs for agentic workloads. title: kimi-k2.7-code image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/llms.txt > Use this file to discover all available pages before exploring further. ![Moonshot AI logo](https://developers.cloudflare.com/_astro/moonshotai.DjWMkXUS.svg) # kimi-k2.7-code Text Generation • Moonshot AI Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/models/kimi-k2.7-code/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Model ID `@cf/moonshotai/kimi-k2.7-code` Copy - Cloudflare-hosted - Function calling - Reasoning - Vision - Planned deprecation - Zero data retention Kimi K2.7 is a frontier-scale open-source 1T parameter model with a 262.1k context window, multi-turn tool calling, vision inputs, and structured outputs for agentic workloads. Paid access required This model is not available through standard Workers Free billing. To use it, upgrade to the [Workers Paid plan](https://developers.cloudflare.com/workers/platform/pricing/#workers) or use prepaid [AI Gateway credits](https://developers.cloudflare.com/ai-gateway/features/unified-billing/). | Model Info | | | --- | --- | | Planned Deprecation | 11/20/2026 | | Context Window [ ↗](https://developers.cloudflare.com/workers-ai/platform/glossary/) | 262,144 tokens | | Terms and License | [link ↗](https://huggingface.co/moonshotai/Kimi-K2.7-Code/blob/main/LICENSE) | | Function calling [↗](https://developers.cloudflare.com/workers-ai/function-calling/) | Yes | | Reasoning | Always on | | Vision | Yes | | Zero data retention | Yes | | Unit Pricing | $0.95 per M input tokens, $4.00 per M output tokens, $0.19 per M cached input tokens | ## Playground Try out this model with Workers AI LLM Playground. It does not require any setup or authentication and is an instant way to preview and test a model directly in the browser. [Launch the LLM Playground](https://playground.ai.cloudflare.com/?model=@cf/moonshotai/kimi-k2.7-code) ## Usage ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const messages = [ { role: "system", content: "You are a friendly assistant" }, { role: "user", content: "What is the origin of the phrase Hello, World", }, ]; const stream = await env.AI.run("@cf/moonshotai/kimi-k2.7-code", { messages, stream: true, }); return new Response(stream, { headers: { "content-type": "text/event-stream" }, }); }, } satisfies ExportedHandler; ``` ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const messages = [ { role: "system", content: "You are a friendly assistant" }, { role: "user", content: "What is the origin of the phrase Hello, World", }, ]; const response = await env.AI.run("@cf/moonshotai/kimi-k2.7-code", { messages }); return Response.json(response); }, } satisfies ExportedHandler; ``` ```py import os import requests ACCOUNT_ID = "your-account-id" AUTH_TOKEN = os.environ.get("CLOUDFLARE_AUTH_TOKEN") prompt = "Tell me all about PEP-8" response = requests.post( f"https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai/run/@cf/moonshotai/kimi-k2.7-code", headers={"Authorization": f"Bearer {AUTH_TOKEN}"}, json={ "messages": [ {"role": "system", "content": "You are a friendly assistant"}, {"role": "user", "content": prompt} ] } ) result = response.json() print(result) ``` ```sh curl https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/run/@cf/moonshotai/kimi-k2.7-code \ -X POST \ -H "Authorization: Bearer $CLOUDFLARE_AUTH_TOKEN" \ -d '{ "messages": [{ "role": "system", "content": "You are a friendly assistant" }, { "role": "user", "content": "Why is pizza so good" }]}' ``` OpenAI compatible endpoints Workers AI also supports OpenAI compatible API endpoints for `/v1/chat/completions` and `/v1/embeddings`. For more details, refer to [Configurations](https://developers.cloudflare.com/workers-ai/configuration/open-ai-compatibility/) . ## Parameters
Synchronous — Send a request and receive a complete response Input format Prompt Simple text input for single-turn interactionsMessages Structured conversation format with roles (user, assistant, system) prompt stringrequiredminLength: 1The input text prompt for the model to generate a response. model stringID of the model to use (e.g. '@cf/zai-org/glm-4.7-flash, etc'). ▶audio{} objectParameters for audio output. Required when modalities includes 'audio'. frequency\_penalty number | nullPenalizes new tokens based on their existing frequency in the text so far. logit\_bias object | nullModify the likelihood of specified tokens appearing in the completion. Maps token IDs to bias values from -100 to 100. logprobs boolean | nullWhether to return log probabilities of the output tokens. top\_logprobs integer | nullHow many top log probabilities to return at each token position (0-20). Requires logprobs=true. max\_tokens integer | nullDeprecated in favor of max_completion_tokens. The maximum number of tokens to generate. max\_completion\_tokens integer | nullAn upper bound for the number of tokens that can be generated for a completion. metadata object | nullSet of 16 key-value pairs that can be attached to the object. modalities array | nullOutput types requested from the model (e.g. ['text'] or ['text', 'audio']). n integer | nullHow many chat completion choices to generate for each input message. parallel\_tool\_calls booleandefault: trueWhether to enable parallel function calling during tool use. ▶prediction{} object presence\_penalty number | nullPenalizes new tokens based on whether they appear in the text so far. ▶chat\_template\_kwargs{} object ▶response\_format one ofSpecifies the format the model must output. seed integer | nullIf specified, the system will make a best effort to sample deterministically. ▶stop one of store boolean | nullWhether to store the output for model distillation / evals. stream boolean | nullIf true, partial message deltas will be sent as server-sent events. ▶stream\_options{} object temperature number | nullSampling temperature between 0 and 2. ▶tool\_choice one ofControls which (if any) tool is called by the model. 'none' = no tools, 'auto' = model decides, 'required' = must call a tool. ▶tools\[] arrayA list of tools the model may call. top\_p number | nullNucleus sampling: considers the results of the tokens with top_p probability mass. user stringA unique identifier representing your end-user, for abuse monitoring. ▶web\_search\_options{} objectOptions for the web search tool (when using built-in web search). ▶function\_call one of ▶functions\[] arrayminItems: 1maxItems: 128 id stringA unique identifier for the chat completion. object string created integerUnix timestamp (seconds) of when the completion was created. model stringThe model used for the chat completion. ▶choices\[] arrayminItems: 1 ▶usage{} object system\_fingerprint string | null
Streaming — Send a request with `stream: true` and receive server-sent events Input format Prompt Simple text input for single-turn interactionsMessages Structured conversation format with roles (user, assistant, system) prompt stringrequiredminLength: 1The input text prompt for the model to generate a response. model stringID of the model to use (e.g. '@cf/zai-org/glm-4.7-flash, etc'). ▶audio{} objectParameters for audio output. Required when modalities includes 'audio'. frequency\_penalty number | nullPenalizes new tokens based on their existing frequency in the text so far. logit\_bias object | nullModify the likelihood of specified tokens appearing in the completion. Maps token IDs to bias values from -100 to 100. logprobs boolean | nullWhether to return log probabilities of the output tokens. top\_logprobs integer | nullHow many top log probabilities to return at each token position (0-20). Requires logprobs=true. max\_tokens integer | nullDeprecated in favor of max_completion_tokens. The maximum number of tokens to generate. max\_completion\_tokens integer | nullAn upper bound for the number of tokens that can be generated for a completion. metadata object | nullSet of 16 key-value pairs that can be attached to the object. modalities array | nullOutput types requested from the model (e.g. ['text'] or ['text', 'audio']). n integer | nullHow many chat completion choices to generate for each input message. parallel\_tool\_calls booleandefault: trueWhether to enable parallel function calling during tool use. ▶prediction{} object presence\_penalty number | nullPenalizes new tokens based on whether they appear in the text so far. ▶chat\_template\_kwargs{} object ▶response\_format one ofSpecifies the format the model must output. seed integer | nullIf specified, the system will make a best effort to sample deterministically. ▶stop one of store boolean | nullWhether to store the output for model distillation / evals. stream boolean | nullIf true, partial message deltas will be sent as server-sent events. ▶stream\_options{} object temperature number | nullSampling temperature between 0 and 2. ▶tool\_choice one ofControls which (if any) tool is called by the model. 'none' = no tools, 'auto' = model decides, 'required' = must call a tool. ▶tools\[] arrayA list of tools the model may call. top\_p number | nullNucleus sampling: considers the results of the tokens with top_p probability mass. user stringA unique identifier representing your end-user, for abuse monitoring. ▶web\_search\_options{} objectOptions for the web search tool (when using built-in web search). ▶function\_call one of ▶functions\[] arrayminItems: 1maxItems: 128 type string contentType text/event-stream format binary
Batch — Send multiple requests in a single API call ▶requests\[] array id stringA unique identifier for the chat completion. object string created integerUnix timestamp (seconds) of when the completion was created. model stringThe model used for the chat completion. ▶choices\[] arrayminItems: 1 ▶usage{} object system\_fingerprint string | null
## API Schemas (Raw) SynchronousInput [Open](https://developers.cloudflare.com/workers-ai/models/kimi-k2.7-code/sync-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/kimi-k2.7-code/sync-input.json) SynchronousOutput [Open](https://developers.cloudflare.com/workers-ai/models/kimi-k2.7-code/sync-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/kimi-k2.7-code/sync-output.json) StreamingInput [Open](https://developers.cloudflare.com/workers-ai/models/kimi-k2.7-code/streaming-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/kimi-k2.7-code/streaming-input.json) StreamingOutput [Open](https://developers.cloudflare.com/workers-ai/models/kimi-k2.7-code/streaming-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/kimi-k2.7-code/streaming-output.json) BatchInput [Open](https://developers.cloudflare.com/workers-ai/models/kimi-k2.7-code/batch-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/kimi-k2.7-code/batch-input.json) BatchOutput [Open](https://developers.cloudflare.com/workers-ai/models/kimi-k2.7-code/batch-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/kimi-k2.7-code/batch-output.json) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/models/kimi-k2.7-code/#page","headline":"kimi-k2.7-code","description":"Kimi K2.7 is a frontier-scale open-source 1T parameter model with a 262.1k context window, multi-turn tool calling, vision inputs, and structured outputs for agentic workloads.","url":"https://developers.cloudflare.com/workers-ai/models/kimi-k2.7-code/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: MeloTTS is a high-quality multi-lingual text-to-speech library by MyShell.ai. title: melotts image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/llms.txt > Use this file to discover all available pages before exploring further. ![MyShell logo](https://developers.cloudflare.com/_astro/myshell.6ROagMV2.svg) # melotts Text-to-Speech • MyShell Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/models/melotts/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Model ID `@cf/myshell-ai/melotts` Copy - Cloudflare-hosted - Planned deprecation - Zero data retention MeloTTS is a high-quality multi-lingual text-to-speech library by MyShell.ai. | Model Info | | | --- | --- | | Planned Deprecation | 11/20/2026 | | Zero data retention | Yes | | Unit Pricing | $0.000205 per audio minute | ## Parameters prompt `string`requiredminLength: 1A text description of the audio you want to generate lang `string`default: enThe speech language (e.g., 'en' for English, 'fr' for French). Defaults to 'en' if not specified ▶Option 1{} objectcontentType: application/json Option 2 stringcontentType: audio/mpegformat: binary The generated audio in MP3 format ## API Schemas (Raw) Input [Open](https://developers.cloudflare.com/workers-ai/models/melotts/schema-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/melotts/schema-input.json) Output [Open](https://developers.cloudflare.com/workers-ai/models/melotts/schema-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/melotts/schema-output.json) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/models/melotts/#page","headline":"melotts","description":"MeloTTS is a high-quality multi-lingual text-to-speech library by MyShell.ai.","url":"https://developers.cloudflare.com/workers-ai/models/melotts/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: NVIDIA Nemotron 3 Super is a hybrid MoE model with leading accuracy for multi-agent applications and specialized agentic AI systems. title: nemotron-3-120b-a12b image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/llms.txt > Use this file to discover all available pages before exploring further. ![NVIDIA logo](https://developers.cloudflare.com/_astro/nvidia.DI1bb8hH.svg) # nemotron-3-120b-a12b Text Generation • NVIDIA Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/models/nemotron-3-120b-a12b/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Model ID `@cf/nvidia/nemotron-3-120b-a12b` Copy - Cloudflare-hosted - Function calling - Reasoning - Planned deprecation - Zero data retention NVIDIA Nemotron 3 Super is a hybrid MoE model with leading accuracy for multi-agent applications and specialized agentic AI systems. | Model Info | | | --- | --- | | Planned Deprecation | 11/20/2026 | | Context Window [ ↗](https://developers.cloudflare.com/workers-ai/platform/glossary/) | 256,000 tokens | | Terms and License | [link ↗](https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-nemotron-open-model-license/) | | Function calling [↗](https://developers.cloudflare.com/workers-ai/function-calling/) | Yes | | Reasoning | Yes | | Zero data retention | Yes | | Unit Pricing | $0.50 per M input tokens, $1.50 per M output tokens | ## Reasoning effort Nemotron supports normal reasoning, low reasoning, and reasoning turned off. It does not currently select these modes from the top-level`reasoning_effort` field. Pass the corresponding options through `chat_template_kwargs` instead. To use low reasoning, set both `enable_thinking` and`low_effort`: ```python response = client.chat.completions.create( model="@cf/nvidia/nemotron-3-120b-a12b", messages=[{"role": "user", "content": "What is the capital of Japan?"}], max_tokens=16000, temperature=1.0, top_p=0.95, extra_body={ "chat_template_kwargs": { "enable_thinking": True, "low_effort": True, } }, ) ``` The low-effort option appends `{reasoning effort: low}` to the latest user message. To turn reasoning off, set`enable_thinking` to `False`. For coding agents, set`force_nonempty_content` to `True` in the same`chat_template_kwargs` object. For more information, refer to [NVIDIA's Nemotron API client example](https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-FP8#api-client). ## Playground Try out this model with Workers AI LLM Playground. It does not require any setup or authentication and is an instant way to preview and test a model directly in the browser. [Launch the LLM Playground](https://playground.ai.cloudflare.com/?model=@cf/nvidia/nemotron-3-120b-a12b) ## Usage ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const messages = [ { role: "system", content: "You are a friendly assistant" }, { role: "user", content: "What is the origin of the phrase Hello, World", }, ]; const stream = await env.AI.run("@cf/nvidia/nemotron-3-120b-a12b", { messages, stream: true, }); return new Response(stream, { headers: { "content-type": "text/event-stream" }, }); }, } satisfies ExportedHandler; ``` ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const messages = [ { role: "system", content: "You are a friendly assistant" }, { role: "user", content: "What is the origin of the phrase Hello, World", }, ]; const response = await env.AI.run("@cf/nvidia/nemotron-3-120b-a12b", { messages }); return Response.json(response); }, } satisfies ExportedHandler; ``` ```py import os import requests ACCOUNT_ID = "your-account-id" AUTH_TOKEN = os.environ.get("CLOUDFLARE_AUTH_TOKEN") prompt = "Tell me all about PEP-8" response = requests.post( f"https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai/run/@cf/nvidia/nemotron-3-120b-a12b", headers={"Authorization": f"Bearer {AUTH_TOKEN}"}, json={ "messages": [ {"role": "system", "content": "You are a friendly assistant"}, {"role": "user", "content": prompt} ] } ) result = response.json() print(result) ``` ```sh curl https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/run/@cf/nvidia/nemotron-3-120b-a12b \ -X POST \ -H "Authorization: Bearer $CLOUDFLARE_AUTH_TOKEN" \ -d '{ "messages": [{ "role": "system", "content": "You are a friendly assistant" }, { "role": "user", "content": "Why is pizza so good" }]}' ``` OpenAI compatible endpoints Workers AI also supports OpenAI compatible API endpoints for `/v1/chat/completions` and `/v1/embeddings`. For more details, refer to [Configurations](https://developers.cloudflare.com/workers-ai/configuration/open-ai-compatibility/) . ## Parameters ### Input model `string`ID of the model to use (for example, '@cf/nvidia/nemotron-3-120b-a12b'). ▶audio{} `object`Parameters for audio output. Required when modalities includes 'audio'. frequency\_penalty `number | null`Penalizes new tokens based on their existing frequency in the text so far. logit\_bias `object | null`Modify the likelihood of specified tokens appearing in the completion. Maps token IDs to bias values from -100 to 100. logprobs `boolean | null`Whether to return log probabilities of the output tokens. top\_logprobs `integer | null`How many top log probabilities to return at each token position (0-20). Requires logprobs=true. max\_tokens `integer | null`The maximum number of tokens to generate. max\_completion\_tokens `integer | null`An upper bound for the number of tokens that can be generated for a completion. metadata `object | null`Set of key-value pairs that can be attached to the object. modalities `array | null`Output types requested from the model. n `integer | null`How many chat completion choices to generate for each input message. parallel\_tool\_calls `boolean`default: trueWhether to enable parallel function calling during tool use. ▶prediction{} `object` presence\_penalty `number | null`Penalizes new tokens based on whether they appear in the text so far. ▶chat\_template\_kwargs{} `object`Nemotron chat-template controls for normal reasoning, low-effort reasoning, and non-reasoning responses. ▶response\_format `one of`Specifies the format the model must output. seed `integer | null`If specified, the system will make a best effort to sample deterministically. ▶stop `one of` store `boolean | null`Whether to store the output for model distillation or evaluation. stream `boolean | null`If true, partial message deltas will be sent as server-sent events. ▶stream\_options{} `object` temperature `number | null`Sampling temperature between 0 and 2. ▶tool\_choice `one of`Controls which (if any) tool is called by the model. 'none' = no tools, 'auto' = model decides, 'required' = must call a tool. ▶tools\[] `array`A list of tools the model may call. top\_p `number | null`Nucleus sampling: considers the results of the tokens with top\_p probability mass. user `string`A unique identifier representing your end-user, for abuse monitoring. ▶web\_search\_options{} `object`Options for the web search tool (when using built-in web search). ▶function\_call `one of` ▶functions\[] `array`minItems: 1maxItems: 128 prompt `string`requiredminLength: 1The input text prompt for the model to generate a response. ### Output
Synchronous — Send a request and receive a complete response id stringA unique identifier for the chat completion. object string created integerUnix timestamp (seconds) of when the completion was created. model stringThe model used for the chat completion. ▶choices\[] arrayminItems: 1 ▶usage{} object system\_fingerprint string | null
Streaming — Send a request with `stream: true` and receive server-sent events type string contentType text/event-stream format binary
## API Schemas (Raw) SynchronousInput [Open](https://developers.cloudflare.com/workers-ai/models/nemotron-3-120b-a12b/sync-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/nemotron-3-120b-a12b/sync-input.json) SynchronousOutput [Open](https://developers.cloudflare.com/workers-ai/models/nemotron-3-120b-a12b/sync-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/nemotron-3-120b-a12b/sync-output.json) StreamingInput [Open](https://developers.cloudflare.com/workers-ai/models/nemotron-3-120b-a12b/streaming-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/nemotron-3-120b-a12b/streaming-input.json) StreamingOutput [Open](https://developers.cloudflare.com/workers-ai/models/nemotron-3-120b-a12b/streaming-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/nemotron-3-120b-a12b/streaming-output.json) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/models/nemotron-3-120b-a12b/#page","headline":"nemotron-3-120b-a12b","description":"NVIDIA Nemotron 3 Super is a hybrid MoE model with leading accuracy for multi-agent applications and specialized agentic AI systems.","url":"https://developers.cloudflare.com/workers-ai/models/nemotron-3-120b-a12b/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: OpenAI’s open-weight models designed for powerful reasoning, agentic tasks, and versatile developer use cases – gpt-oss-120b is for production, general purpose, high reasoning use-cases. title: gpt-oss-120b image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/llms.txt > Use this file to discover all available pages before exploring further. ![OpenAI logo](https://developers.cloudflare.com/_astro/openai.BBwNKzBb.svg) # gpt-oss-120b Text Generation • OpenAI Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/models/gpt-oss-120b/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Model ID `@cf/openai/gpt-oss-120b` Copy - Cloudflare-hosted - Batch - Function calling - Reasoning - Zero data retention OpenAI’s open-weight models designed for powerful reasoning, agentic tasks, and versatile developer use cases – gpt-oss-120b is for production, general purpose, high reasoning use-cases. | Model Info | | | --- | --- | | Context Window [ ↗](https://developers.cloudflare.com/workers-ai/platform/glossary/) | 128,000 tokens | | Function calling [↗](https://developers.cloudflare.com/workers-ai/function-calling/) | Yes | | Reasoning | `low``medium` (default)`high` | | Zero data retention | Yes | | Batch | Yes | | Unit Pricing | $0.35 per M input tokens, $0.75 per M output tokens | ## Usage ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const messages = [ { role: "system", content: "You are a friendly assistant" }, { role: "user", content: "What is the origin of the phrase Hello, World", }, ]; const stream = await env.AI.run("@cf/openai/gpt-oss-120b", { messages, stream: true, }); return new Response(stream, { headers: { "content-type": "text/event-stream" }, }); }, } satisfies ExportedHandler; ``` ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const messages = [ { role: "system", content: "You are a friendly assistant" }, { role: "user", content: "What is the origin of the phrase Hello, World", }, ]; const response = await env.AI.run("@cf/openai/gpt-oss-120b", { messages }); return Response.json(response); }, } satisfies ExportedHandler; ``` ```py import os import requests ACCOUNT_ID = "your-account-id" AUTH_TOKEN = os.environ.get("CLOUDFLARE_AUTH_TOKEN") prompt = "Tell me all about PEP-8" response = requests.post( f"https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai/run/@cf/openai/gpt-oss-120b", headers={"Authorization": f"Bearer {AUTH_TOKEN}"}, json={ "messages": [ {"role": "system", "content": "You are a friendly assistant"}, {"role": "user", "content": prompt} ] } ) result = response.json() print(result) ``` ```sh curl https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/run/@cf/openai/gpt-oss-120b \ -X POST \ -H "Authorization: Bearer $CLOUDFLARE_AUTH_TOKEN" \ -d '{ "messages": [{ "role": "system", "content": "You are a friendly assistant" }, { "role": "user", "content": "Why is pizza so good" }]}' ``` OpenAI compatible endpoints Workers AI also supports OpenAI compatible API endpoints for `/v1/chat/completions` and `/v1/embeddings`. For more details, refer to [Configurations](https://developers.cloudflare.com/workers-ai/configuration/open-ai-compatibility/) . ## Parameters
Synchronous — Send a request and receive a complete response Input format Prompt Simple text input for single-turn interactionsMessages Structured conversation format with roles (user, assistant, system) prompt stringrequiredminLength: 1The input text prompt for the model to generate a response. lora stringName of the LoRA (Low-Rank Adaptation) model to fine-tune the base model. ▶response\_format{} object raw booleandefault: falseIf true, a chat template is not applied and you must adhere to the specific model's expected formatting. stream booleandefault: falseIf true, the response will be streamed back incrementally using SSE, Server Sent Events. max\_tokens integerdefault: 256The maximum number of tokens to generate in the response. temperature numberdefault: 0.6minimum: 0maximum: 5Controls the randomness of the output; higher values produce more random results. top\_p numberminimum: 0.001maximum: 1Adjusts the creativity of the AI's responses by controlling how many possible words it considers. Lower values make outputs more predictable; higher values allow for more varied and creative responses. top\_k integerminimum: 1maximum: 50Limits the AI to choose from the top 'k' most probable words. Lower values make responses more focused; higher values introduce more variety and potential surprises. seed integerminimum: 1maximum: 9999999999Random seed for reproducibility of the generation. repetition\_penalty numberminimum: 0maximum: 2Penalty for repeated tokens; higher values discourage repetition. frequency\_penalty numberminimum: -2maximum: 2Decreases the likelihood of the model repeating the same lines verbatim. presence\_penalty numberminimum: -2maximum: 2Increases the likelihood of the model introducing new topics. type object contentType application/json
Streaming — Send a request with `stream: true` and receive server-sent events Input format Prompt Simple text input for single-turn interactionsMessages Structured conversation format with roles (user, assistant, system) prompt stringrequiredminLength: 1The input text prompt for the model to generate a response. lora stringName of the LoRA (Low-Rank Adaptation) model to fine-tune the base model. ▶response\_format{} object raw booleandefault: falseIf true, a chat template is not applied and you must adhere to the specific model's expected formatting. stream booleandefault: falseIf true, the response will be streamed back incrementally using SSE, Server Sent Events. max\_tokens integerdefault: 256The maximum number of tokens to generate in the response. temperature numberdefault: 0.6minimum: 0maximum: 5Controls the randomness of the output; higher values produce more random results. top\_p numberminimum: 0.001maximum: 1Adjusts the creativity of the AI's responses by controlling how many possible words it considers. Lower values make outputs more predictable; higher values allow for more varied and creative responses. top\_k integerminimum: 1maximum: 50Limits the AI to choose from the top 'k' most probable words. Lower values make responses more focused; higher values introduce more variety and potential surprises. seed integerminimum: 1maximum: 9999999999Random seed for reproducibility of the generation. repetition\_penalty numberminimum: 0maximum: 2Penalty for repeated tokens; higher values discourage repetition. frequency\_penalty numberminimum: -2maximum: 2Decreases the likelihood of the model repeating the same lines verbatim. presence\_penalty numberminimum: -2maximum: 2Increases the likelihood of the model introducing new topics. type string contentType text/event-stream format binary
Batch — Send multiple requests in a single API call ▶requests\[] arrayrequired type object contentType application/json
## API Schemas (Raw) SynchronousInput [Open](https://developers.cloudflare.com/workers-ai/models/gpt-oss-120b/sync-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/gpt-oss-120b/sync-input.json) SynchronousOutput [Open](https://developers.cloudflare.com/workers-ai/models/gpt-oss-120b/sync-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/gpt-oss-120b/sync-output.json) StreamingInput [Open](https://developers.cloudflare.com/workers-ai/models/gpt-oss-120b/streaming-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/gpt-oss-120b/streaming-input.json) StreamingOutput [Open](https://developers.cloudflare.com/workers-ai/models/gpt-oss-120b/streaming-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/gpt-oss-120b/streaming-output.json) BatchInput [Open](https://developers.cloudflare.com/workers-ai/models/gpt-oss-120b/batch-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/gpt-oss-120b/batch-input.json) BatchOutput [Open](https://developers.cloudflare.com/workers-ai/models/gpt-oss-120b/batch-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/gpt-oss-120b/batch-output.json) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/models/gpt-oss-120b/#page","headline":"gpt-oss-120b","description":"OpenAI’s open-weight models designed for powerful reasoning, agentic tasks, and versatile developer use cases – gpt-oss-120b is for production, general purpose, high reasoning use-cases.","url":"https://developers.cloudflare.com/workers-ai/models/gpt-oss-120b/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: OpenAI’s open-weight models designed for powerful reasoning, agentic tasks, and versatile developer use cases – gpt-oss-20b is for lower latency, and local or specialized use-cases. title: gpt-oss-20b image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/llms.txt > Use this file to discover all available pages before exploring further. ![OpenAI logo](https://developers.cloudflare.com/_astro/openai.BBwNKzBb.svg) # gpt-oss-20b Text Generation • OpenAI Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/models/gpt-oss-20b/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Model ID `@cf/openai/gpt-oss-20b` Copy - Cloudflare-hosted - Batch - Function calling - Reasoning - Zero data retention OpenAI’s open-weight models designed for powerful reasoning, agentic tasks, and versatile developer use cases – gpt-oss-20b is for lower latency, and local or specialized use-cases. | Model Info | | | --- | --- | | Context Window [ ↗](https://developers.cloudflare.com/workers-ai/platform/glossary/) | 128,000 tokens | | Function calling [↗](https://developers.cloudflare.com/workers-ai/function-calling/) | Yes | | Reasoning | `low``medium` (default)`high` | | Zero data retention | Yes | | Batch | Yes | | Unit Pricing | $0.20 per M input tokens, $0.30 per M output tokens | ## Usage ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const messages = [ { role: "system", content: "You are a friendly assistant" }, { role: "user", content: "What is the origin of the phrase Hello, World", }, ]; const stream = await env.AI.run("@cf/openai/gpt-oss-20b", { messages, stream: true, }); return new Response(stream, { headers: { "content-type": "text/event-stream" }, }); }, } satisfies ExportedHandler; ``` ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const messages = [ { role: "system", content: "You are a friendly assistant" }, { role: "user", content: "What is the origin of the phrase Hello, World", }, ]; const response = await env.AI.run("@cf/openai/gpt-oss-20b", { messages }); return Response.json(response); }, } satisfies ExportedHandler; ``` ```py import os import requests ACCOUNT_ID = "your-account-id" AUTH_TOKEN = os.environ.get("CLOUDFLARE_AUTH_TOKEN") prompt = "Tell me all about PEP-8" response = requests.post( f"https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai/run/@cf/openai/gpt-oss-20b", headers={"Authorization": f"Bearer {AUTH_TOKEN}"}, json={ "messages": [ {"role": "system", "content": "You are a friendly assistant"}, {"role": "user", "content": prompt} ] } ) result = response.json() print(result) ``` ```sh curl https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/run/@cf/openai/gpt-oss-20b \ -X POST \ -H "Authorization: Bearer $CLOUDFLARE_AUTH_TOKEN" \ -d '{ "messages": [{ "role": "system", "content": "You are a friendly assistant" }, { "role": "user", "content": "Why is pizza so good" }]}' ``` OpenAI compatible endpoints Workers AI also supports OpenAI compatible API endpoints for `/v1/chat/completions` and `/v1/embeddings`. For more details, refer to [Configurations](https://developers.cloudflare.com/workers-ai/configuration/open-ai-compatibility/) . ## Parameters
Synchronous — Send a request and receive a complete response Input format Prompt Simple text input for single-turn interactionsMessages Structured conversation format with roles (user, assistant, system) prompt stringrequiredminLength: 1The input text prompt for the model to generate a response. lora stringName of the LoRA (Low-Rank Adaptation) model to fine-tune the base model. ▶response\_format{} object raw booleandefault: falseIf true, a chat template is not applied and you must adhere to the specific model's expected formatting. stream booleandefault: falseIf true, the response will be streamed back incrementally using SSE, Server Sent Events. max\_tokens integerdefault: 256The maximum number of tokens to generate in the response. temperature numberdefault: 0.6minimum: 0maximum: 5Controls the randomness of the output; higher values produce more random results. top\_p numberminimum: 0.001maximum: 1Adjusts the creativity of the AI's responses by controlling how many possible words it considers. Lower values make outputs more predictable; higher values allow for more varied and creative responses. top\_k integerminimum: 1maximum: 50Limits the AI to choose from the top 'k' most probable words. Lower values make responses more focused; higher values introduce more variety and potential surprises. seed integerminimum: 1maximum: 9999999999Random seed for reproducibility of the generation. repetition\_penalty numberminimum: 0maximum: 2Penalty for repeated tokens; higher values discourage repetition. frequency\_penalty numberminimum: -2maximum: 2Decreases the likelihood of the model repeating the same lines verbatim. presence\_penalty numberminimum: -2maximum: 2Increases the likelihood of the model introducing new topics. type object contentType application/json
Streaming — Send a request with `stream: true` and receive server-sent events Input format Prompt Simple text input for single-turn interactionsMessages Structured conversation format with roles (user, assistant, system) prompt stringrequiredminLength: 1The input text prompt for the model to generate a response. lora stringName of the LoRA (Low-Rank Adaptation) model to fine-tune the base model. ▶response\_format{} object raw booleandefault: falseIf true, a chat template is not applied and you must adhere to the specific model's expected formatting. stream booleandefault: falseIf true, the response will be streamed back incrementally using SSE, Server Sent Events. max\_tokens integerdefault: 256The maximum number of tokens to generate in the response. temperature numberdefault: 0.6minimum: 0maximum: 5Controls the randomness of the output; higher values produce more random results. top\_p numberminimum: 0.001maximum: 1Adjusts the creativity of the AI's responses by controlling how many possible words it considers. Lower values make outputs more predictable; higher values allow for more varied and creative responses. top\_k integerminimum: 1maximum: 50Limits the AI to choose from the top 'k' most probable words. Lower values make responses more focused; higher values introduce more variety and potential surprises. seed integerminimum: 1maximum: 9999999999Random seed for reproducibility of the generation. repetition\_penalty numberminimum: 0maximum: 2Penalty for repeated tokens; higher values discourage repetition. frequency\_penalty numberminimum: -2maximum: 2Decreases the likelihood of the model repeating the same lines verbatim. presence\_penalty numberminimum: -2maximum: 2Increases the likelihood of the model introducing new topics. type string contentType text/event-stream format binary
Batch — Send multiple requests in a single API call ▶requests\[] arrayrequired type object contentType application/json
## API Schemas (Raw) SynchronousInput [Open](https://developers.cloudflare.com/workers-ai/models/gpt-oss-20b/sync-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/gpt-oss-20b/sync-input.json) SynchronousOutput [Open](https://developers.cloudflare.com/workers-ai/models/gpt-oss-20b/sync-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/gpt-oss-20b/sync-output.json) StreamingInput [Open](https://developers.cloudflare.com/workers-ai/models/gpt-oss-20b/streaming-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/gpt-oss-20b/streaming-input.json) StreamingOutput [Open](https://developers.cloudflare.com/workers-ai/models/gpt-oss-20b/streaming-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/gpt-oss-20b/streaming-output.json) BatchInput [Open](https://developers.cloudflare.com/workers-ai/models/gpt-oss-20b/batch-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/gpt-oss-20b/batch-input.json) BatchOutput [Open](https://developers.cloudflare.com/workers-ai/models/gpt-oss-20b/batch-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/gpt-oss-20b/batch-output.json) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/models/gpt-oss-20b/#page","headline":"gpt-oss-20b","description":"OpenAI’s open-weight models designed for powerful reasoning, agentic tasks, and versatile developer use cases – gpt-oss-20b is for lower latency, and local or specialized use-cases.","url":"https://developers.cloudflare.com/workers-ai/models/gpt-oss-20b/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Whisper is a general-purpose speech recognition model. It is trained on a large dataset of diverse audio and is also a multitasking model that can perform multilingual speech recognition, speech translation, and language identification. title: whisper image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/llms.txt > Use this file to discover all available pages before exploring further. ![OpenAI logo](https://developers.cloudflare.com/_astro/openai.BBwNKzBb.svg) # whisper Automatic Speech Recognition • OpenAI Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/models/whisper/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Model ID `@cf/openai/whisper` Copy - Cloudflare-hosted - Zero data retention Whisper is a general-purpose speech recognition model. It is trained on a large dataset of diverse audio and is also a multitasking model that can perform multilingual speech recognition, speech translation, and language identification. | Model Info | | | --- | --- | | More information | [link ↗](https://openai.com/research/whisper) | | Zero data retention | Yes | | Unit Pricing | $0.000453 per audio minute | ## Parameters Option 1 stringformat: binary ▶Option 2{} object text `string`The transcription word\_count `number` ▶words\[] `array` vtt `string` ## API Schemas (Raw) Input [Open](https://developers.cloudflare.com/workers-ai/models/whisper/schema-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/whisper/schema-input.json) Output [Open](https://developers.cloudflare.com/workers-ai/models/whisper/schema-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/whisper/schema-output.json) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/models/whisper/#page","headline":"whisper","description":"Whisper is a general-purpose speech recognition model. It is trained on a large dataset of diverse audio and is also a multitasking model that can perform multilingual speech recognition, speech translation, and language identification.","url":"https://developers.cloudflare.com/workers-ai/models/whisper/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: "Whisper is a pre-trained model for automatic speech recognition (ASR) and speech translation. " title: whisper-large-v3-turbo image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/llms.txt > Use this file to discover all available pages before exploring further. ![OpenAI logo](https://developers.cloudflare.com/_astro/openai.BBwNKzBb.svg) # whisper-large-v3-turbo Automatic Speech Recognition • OpenAI Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/models/whisper-large-v3-turbo/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Model ID `@cf/openai/whisper-large-v3-turbo` Copy - Cloudflare-hosted - Batch - Zero data retention Whisper is a pre-trained model for automatic speech recognition (ASR) and speech translation. | Model Info | | | --- | --- | | Zero data retention | Yes | | Batch | Yes | | Unit Pricing | $0.000513 per audio minute | ## Parameters ▶audio `one of`required task `string`default: transcribeSupported tasks are 'translate' or 'transcribe'. language `string`The language of the audio being transcribed or translated. vad\_filter `boolean`default: falsePreprocess the audio with a voice activity detection model. initial\_prompt `string`A text prompt to help provide context to the model on the contents of the audio. prefix `string`The prefix appended to the beginning of the output of the transcription and can guide the transcription result. beam\_size `integer`default: 5The number of beams to use in beam search decoding. Higher values may improve accuracy at the cost of speed. condition\_on\_previous\_text `boolean`default: trueWhether to condition on previous text during transcription. Setting to false may help prevent hallucination loops. no\_speech\_threshold `number`default: 0.6Threshold for detecting no-speech segments. Segments with no-speech probability above this value are skipped. compression\_ratio\_threshold `number`default: 2.4Threshold for filtering out segments with high compression ratio, which often indicate repetitive or hallucinated text. log\_prob\_threshold `number`default: -1Threshold for filtering out segments with low average log probability, indicating low confidence. hallucination\_silence\_threshold `number`Optional threshold (in seconds) to skip silent periods that may cause hallucinations. ▶transcription\_info{} `object` text `string`The complete transcription of the audio. word\_count `number`The total number of words in the transcription. ▶segments\[] `array` vtt `string`The transcription in WebVTT format, which includes timing and text information for use in subtitles. ## API Schemas (Raw) Input [Open](https://developers.cloudflare.com/workers-ai/models/whisper-large-v3-turbo/schema-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/whisper-large-v3-turbo/schema-input.json) Output [Open](https://developers.cloudflare.com/workers-ai/models/whisper-large-v3-turbo/schema-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/whisper-large-v3-turbo/schema-output.json) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/models/whisper-large-v3-turbo/#page","headline":"whisper-large-v3-turbo","description":"Whisper is a pre-trained model for automatic speech recognition (ASR) and speech translation.","url":"https://developers.cloudflare.com/workers-ai/models/whisper-large-v3-turbo/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Whisper is a pre-trained model for automatic speech recognition (ASR) and speech translation. Trained on 680k hours of labelled data, Whisper models demonstrate a strong ability to generalize to many datasets and domains without the need for fine-tuning. This is the English-only version of the Whisper Tiny model which was trained on the task of speech recognition. title: whisper-tiny-en image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/llms.txt > Use this file to discover all available pages before exploring further. ![OpenAI logo](https://developers.cloudflare.com/_astro/openai.BBwNKzBb.svg) # whisper-tiny-en Beta Automatic Speech Recognition • OpenAI Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/models/whisper-tiny-en/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Model ID `@cf/openai/whisper-tiny-en` Copy - Cloudflare-hosted - Planned deprecation - Zero data retention Whisper is a pre-trained model for automatic speech recognition (ASR) and speech translation. Trained on 680k hours of labelled data, Whisper models demonstrate a strong ability to generalize to many datasets and domains without the need for fine-tuning. This is the English-only version of the Whisper Tiny model which was trained on the task of speech recognition. | Model Info | | | --- | --- | | Planned Deprecation | 11/20/2026 | | Zero data retention | Yes | | Beta | Yes | ## Parameters Option 1 stringformat: binary ▶Option 2{} object text `string`The transcription word\_count `number` ▶words\[] `array` vtt `string` ## API Schemas (Raw) Input [Open](https://developers.cloudflare.com/workers-ai/models/whisper-tiny-en/schema-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/whisper-tiny-en/schema-input.json) Output [Open](https://developers.cloudflare.com/workers-ai/models/whisper-tiny-en/schema-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/whisper-tiny-en/schema-output.json) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/models/whisper-tiny-en/#page","headline":"whisper-tiny-en","description":"Whisper is a pre-trained model for automatic speech recognition (ASR) and speech translation. Trained on 680k hours of labelled data, Whisper models demonstrate a strong ability to generalize to many datasets and domains without the need for fine-tuning. This is the English-only version of the Whisper Tiny model which was trained on the task of speech recognition.","url":"https://developers.cloudflare.com/workers-ai/models/whisper-tiny-en/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: "PLaMo-Embedding-1B is a Japanese text embedding model developed by Preferred Networks, Inc.\n\nIt can convert Japanese text input into numerical vectors and can be used for a wide range of applications, including information retrieval, text classification, and clustering." title: plamo-embedding-1b image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/llms.txt > Use this file to discover all available pages before exploring further. p # plamo-embedding-1b Text Embeddings • pfnet Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/models/plamo-embedding-1b/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Model ID `@cf/pfnet/plamo-embedding-1b` Copy - Cloudflare-hosted - Zero data retention PLaMo-Embedding-1B is a Japanese text embedding model developed by Preferred Networks, Inc. It can convert Japanese text input into numerical vectors and can be used for a wide range of applications, including information retrieval, text classification, and clustering. | Model Info | | | --- | --- | | Zero data retention | Yes | | Unit Pricing | $0.0186 per M input tokens | ## Usage ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { // Can be a string or array of strings] const stories = [ "This is a story about an orange cloud", "This is a story about a llama", "This is a story about a hugging emoji", ]; const embeddings = await env.AI.run( "@cf/pfnet/plamo-embedding-1b", { text: stories, } ); return Response.json(embeddings); }, } satisfies ExportedHandler; ``` ```py import os import requests ACCOUNT_ID = "your-account-id" AUTH_TOKEN = os.environ.get("CLOUDFLARE_AUTH_TOKEN") stories = [ 'This is a story about an orange cloud', 'This is a story about a llama', 'This is a story about a hugging emoji' ] response = requests.post( f"https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai/run/@cf/pfnet/plamo-embedding-1b", headers={"Authorization": f"Bearer {AUTH_TOKEN}"}, json={"text": stories} ) print(response.json()) ``` ```sh curl https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/run/@cf/pfnet/plamo-embedding-1b \ -X POST \ -H "Authorization: Bearer $CLOUDFLARE_API_TOKEN" \ -d '{ "text": ["This is a story about an orange cloud", "This is a story about a llama", "This is a story about a hugging emoji"] }' ``` OpenAI compatible endpoints Workers AI also supports OpenAI compatible API endpoints for `/v1/chat/completions` and `/v1/embeddings`. For more details, refer to [Configurations](https://developers.cloudflare.com/workers-ai/configuration/open-ai-compatibility/). ## Parameters ▶text `one of`required ▶data\[] `array`Embedding vectors, where each vector is a list of floats. ▶shape\[] `array`minItems: 2maxItems: 2Shape of the embedding data as \[number\_of\_embeddings, embedding\_dimension]. ## API Schemas (Raw) Input [Open](https://developers.cloudflare.com/workers-ai/models/plamo-embedding-1b/schema-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/plamo-embedding-1b/schema-input.json) Output [Open](https://developers.cloudflare.com/workers-ai/models/plamo-embedding-1b/schema-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/plamo-embedding-1b/schema-output.json) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/models/plamo-embedding-1b/#page","headline":"plamo-embedding-1b","description":"PLaMo-Embedding-1B is a Japanese text embedding model developed by Preferred Networks, Inc.\n\nIt can convert Japanese text input into numerical vectors and can be used for a wide range of applications, including information retrieval, text classification, and clustering.","url":"https://developers.cloudflare.com/workers-ai/models/plamo-embedding-1b/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: An open source, community-driven, native audio turn detection model in 2nd version title: smart-turn-v2 image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/llms.txt > Use this file to discover all available pages before exploring further. ![Pipecat logo](https://developers.cloudflare.com/_astro/pipecat.B-PNBdef.svg) # smart-turn-v2 Dumb Pipe • Pipecat Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/models/smart-turn-v2/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Model ID `@cf/pipecat-ai/smart-turn-v2` Copy - Cloudflare-hosted - Batch - Real-time - Zero data retention An open source, community-driven, native audio turn detection model in 2nd version | Model Info | | | --- | --- | | Zero data retention | Yes | | Batch | Yes | | Real-time | Yes | | Unit Pricing | $0.000338 per audio minute | ## Parameters ▶Option 1{} object ▶Option 2{} object is\_complete `boolean`if true, end-of-turn was detected probability `number`probability of the end-of-turn detection ## API Schemas (Raw) Input [Open](https://developers.cloudflare.com/workers-ai/models/smart-turn-v2/schema-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/smart-turn-v2/schema-input.json) Output [Open](https://developers.cloudflare.com/workers-ai/models/smart-turn-v2/schema-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/smart-turn-v2/schema-output.json) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/models/smart-turn-v2/#page","headline":"smart-turn-v2","description":"An open source, community-driven, native audio turn detection model in 2nd version","url":"https://developers.cloudflare.com/workers-ai/models/smart-turn-v2/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: "Qwen2.5-Coder is the latest series of Code-Specific Qwen large language models (formerly known as CodeQwen). As of now, Qwen2.5-Coder has covered six mainstream model sizes, 0.5, 1.5, 3, 7, 14, 32 billion parameters, to meet the needs of different developers. Qwen2.5-Coder brings the following improvements upon CodeQwen1.5:" title: qwen2.5-coder-32b-instruct image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/llms.txt > Use this file to discover all available pages before exploring further. ![Qwen logo](https://developers.cloudflare.com/_astro/qwen.ByCZjtXU.svg) # qwen2.5-coder-32b-instruct Text Generation • Qwen Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/models/qwen2.5-coder-32b-instruct/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Model ID `@cf/qwen/qwen2.5-coder-32b-instruct` Copy - Cloudflare-hosted - LoRA - Zero data retention Qwen2.5-Coder is the latest series of Code-Specific Qwen large language models (formerly known as CodeQwen). As of now, Qwen2.5-Coder has covered six mainstream model sizes, 0.5, 1.5, 3, 7, 14, 32 billion parameters, to meet the needs of different developers. Qwen2.5-Coder brings the following improvements upon CodeQwen1.5: | Model Info | | | --- | --- | | Context Window [ ↗](https://developers.cloudflare.com/workers-ai/platform/glossary/) | 32,768 tokens | | Zero data retention | Yes | | LoRA | Yes | | Unit Pricing | $0.66 per M input tokens, $1.00 per M output tokens | ## Playground Try out this model with Workers AI LLM Playground. It does not require any setup or authentication and is an instant way to preview and test a model directly in the browser. [Launch the LLM Playground](https://playground.ai.cloudflare.com/?model=@cf/qwen/qwen2.5-coder-32b-instruct) ## Usage ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const messages = [ { role: "system", content: "You are a friendly assistant" }, { role: "user", content: "What is the origin of the phrase Hello, World", }, ]; const stream = await env.AI.run("@cf/qwen/qwen2.5-coder-32b-instruct", { messages, stream: true, }); return new Response(stream, { headers: { "content-type": "text/event-stream" }, }); }, } satisfies ExportedHandler; ``` ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const messages = [ { role: "system", content: "You are a friendly assistant" }, { role: "user", content: "What is the origin of the phrase Hello, World", }, ]; const response = await env.AI.run("@cf/qwen/qwen2.5-coder-32b-instruct", { messages }); return Response.json(response); }, } satisfies ExportedHandler; ``` ```py import os import requests ACCOUNT_ID = "your-account-id" AUTH_TOKEN = os.environ.get("CLOUDFLARE_AUTH_TOKEN") prompt = "Tell me all about PEP-8" response = requests.post( f"https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai/run/@cf/qwen/qwen2.5-coder-32b-instruct", headers={"Authorization": f"Bearer {AUTH_TOKEN}"}, json={ "messages": [ {"role": "system", "content": "You are a friendly assistant"}, {"role": "user", "content": prompt} ] } ) result = response.json() print(result) ``` ```sh curl https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/run/@cf/qwen/qwen2.5-coder-32b-instruct \ -X POST \ -H "Authorization: Bearer $CLOUDFLARE_AUTH_TOKEN" \ -d '{ "messages": [{ "role": "system", "content": "You are a friendly assistant" }, { "role": "user", "content": "Why is pizza so good" }]}' ``` OpenAI compatible endpoints Workers AI also supports OpenAI compatible API endpoints for `/v1/chat/completions` and `/v1/embeddings`. For more details, refer to [Configurations](https://developers.cloudflare.com/workers-ai/configuration/open-ai-compatibility/) . ## Parameters ### Input prompt `string`requiredminLength: 1The input text prompt for the model to generate a response. lora `string`Name of the LoRA (Low-Rank Adaptation) model to fine-tune the base model. ▶response\_format{} `object` raw `boolean`default: falseIf true, a chat template is not applied and you must adhere to the specific model's expected formatting. stream `boolean`default: falseIf true, the response will be streamed back incrementally using SSE, Server Sent Events. max\_tokens `integer`default: 256The maximum number of tokens to generate in the response. temperature `number`default: 0.6minimum: 0maximum: 5Controls the randomness of the output; higher values produce more random results. top\_p `number`minimum: 0maximum: 2Adjusts the creativity of the AI's responses by controlling how many possible words it considers. Lower values make outputs more predictable; higher values allow for more varied and creative responses. top\_k `integer`minimum: 1maximum: 50Limits the AI to choose from the top 'k' most probable words. Lower values make responses more focused; higher values introduce more variety and potential surprises. seed `integer`minimum: 1maximum: 9999999999Random seed for reproducibility of the generation. repetition\_penalty `number`minimum: 0maximum: 2Penalty for repeated tokens; higher values discourage repetition. frequency\_penalty `number`minimum: 0maximum: 2Decreases the likelihood of the model repeating the same lines verbatim. presence\_penalty `number`minimum: 0maximum: 2Increases the likelihood of the model introducing new topics. ### Output
Synchronous — Send a request and receive a complete response response stringThe generated text response from the model ▶usage{} objectUsage statistics for the inference request ▶tool\_calls\[] arrayAn array of tool calls requests made during the response generation
Streaming — Send a request with `stream: true` and receive server-sent events type string format binary
## API Schemas (Raw) SynchronousInput [Open](https://developers.cloudflare.com/workers-ai/models/qwen2.5-coder-32b-instruct/sync-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/qwen2.5-coder-32b-instruct/sync-input.json) SynchronousOutput [Open](https://developers.cloudflare.com/workers-ai/models/qwen2.5-coder-32b-instruct/sync-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/qwen2.5-coder-32b-instruct/sync-output.json) StreamingInput [Open](https://developers.cloudflare.com/workers-ai/models/qwen2.5-coder-32b-instruct/streaming-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/qwen2.5-coder-32b-instruct/streaming-input.json) StreamingOutput [Open](https://developers.cloudflare.com/workers-ai/models/qwen2.5-coder-32b-instruct/streaming-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/qwen2.5-coder-32b-instruct/streaming-output.json) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/models/qwen2.5-coder-32b-instruct/#page","headline":"qwen2.5-coder-32b-instruct","description":"Qwen2.5-Coder is the latest series of Code-Specific Qwen large language models (formerly known as CodeQwen). As of now, Qwen2.5-Coder has covered six mainstream model sizes, 0.5, 1.5, 3, 7, 14, 32 billion parameters, to meet the needs of different developers. Qwen2.5-Coder brings the following improvements upon CodeQwen1.5:","url":"https://developers.cloudflare.com/workers-ai/models/qwen2.5-coder-32b-instruct/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Qwen3 is the latest generation of large language models in Qwen series, offering a comprehensive suite of dense and mixture-of-experts (MoE) models. Built upon extensive training, Qwen3 delivers groundbreaking advancements in reasoning, instruction-following, agent capabilities, and multilingual support. title: qwen3-30b-a3b-fp8 image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/llms.txt > Use this file to discover all available pages before exploring further. ![Qwen logo](https://developers.cloudflare.com/_astro/qwen.ByCZjtXU.svg) # qwen3-30b-a3b-fp8 Text Generation • Qwen Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/models/qwen3-30b-a3b-fp8/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Model ID `@cf/qwen/qwen3-30b-a3b-fp8` Copy - Cloudflare-hosted - Batch - Function calling - Reasoning - Planned deprecation - Zero data retention Qwen3 is the latest generation of large language models in Qwen series, offering a comprehensive suite of dense and mixture-of-experts (MoE) models. Built upon extensive training, Qwen3 delivers groundbreaking advancements in reasoning, instruction-following, agent capabilities, and multilingual support. | Model Info | | | --- | --- | | Planned Deprecation | 11/20/2026 | | Context Window [ ↗](https://developers.cloudflare.com/workers-ai/platform/glossary/) | 32,768 tokens | | Function calling [↗](https://developers.cloudflare.com/workers-ai/function-calling/) | Yes | | Reasoning | Yes | | Zero data retention | Yes | | Batch | Yes | | Unit Pricing | $0.0509 per M input tokens, $0.335 per M output tokens | ## Playground Try out this model with Workers AI LLM Playground. It does not require any setup or authentication and is an instant way to preview and test a model directly in the browser. [Launch the LLM Playground](https://playground.ai.cloudflare.com/?model=@cf/qwen/qwen3-30b-a3b-fp8) ## Usage ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const messages = [ { role: "system", content: "You are a friendly assistant" }, { role: "user", content: "What is the origin of the phrase Hello, World", }, ]; const stream = await env.AI.run("@cf/qwen/qwen3-30b-a3b-fp8", { messages, stream: true, }); return new Response(stream, { headers: { "content-type": "text/event-stream" }, }); }, } satisfies ExportedHandler; ``` ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const messages = [ { role: "system", content: "You are a friendly assistant" }, { role: "user", content: "What is the origin of the phrase Hello, World", }, ]; const response = await env.AI.run("@cf/qwen/qwen3-30b-a3b-fp8", { messages }); return Response.json(response); }, } satisfies ExportedHandler; ``` ```py import os import requests ACCOUNT_ID = "your-account-id" AUTH_TOKEN = os.environ.get("CLOUDFLARE_AUTH_TOKEN") prompt = "Tell me all about PEP-8" response = requests.post( f"https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai/run/@cf/qwen/qwen3-30b-a3b-fp8", headers={"Authorization": f"Bearer {AUTH_TOKEN}"}, json={ "messages": [ {"role": "system", "content": "You are a friendly assistant"}, {"role": "user", "content": prompt} ] } ) result = response.json() print(result) ``` ```sh curl https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/run/@cf/qwen/qwen3-30b-a3b-fp8 \ -X POST \ -H "Authorization: Bearer $CLOUDFLARE_AUTH_TOKEN" \ -d '{ "messages": [{ "role": "system", "content": "You are a friendly assistant" }, { "role": "user", "content": "Why is pizza so good" }]}' ``` OpenAI compatible endpoints Workers AI also supports OpenAI compatible API endpoints for `/v1/chat/completions` and `/v1/embeddings`. For more details, refer to [Configurations](https://developers.cloudflare.com/workers-ai/configuration/open-ai-compatibility/) . ## Parameters
Synchronous — Send a request and receive a complete response prompt stringrequiredminLength: 1The input text prompt for the model to generate a response. lora stringName of the LoRA (Low-Rank Adaptation) model to fine-tune the base model. ▶response\_format{} object raw booleandefault: falseIf true, a chat template is not applied and you must adhere to the specific model's expected formatting. stream booleandefault: falseIf true, the response will be streamed back incrementally using SSE, Server Sent Events. max\_tokens integerdefault: 2000The maximum number of tokens to generate in the response. temperature numberdefault: 0.6minimum: 0maximum: 5Controls the randomness of the output; higher values produce more random results. top\_p numberminimum: 0.001maximum: 1Adjusts the creativity of the AI's responses by controlling how many possible words it considers. Lower values make outputs more predictable; higher values allow for more varied and creative responses. top\_k integerminimum: 1maximum: 50Limits the AI to choose from the top 'k' most probable words. Lower values make responses more focused; higher values introduce more variety and potential surprises. seed integerminimum: 1maximum: 9999999999Random seed for reproducibility of the generation. repetition\_penalty numberminimum: 0maximum: 2Penalty for repeated tokens; higher values discourage repetition. frequency\_penalty numberminimum: -2maximum: 2Decreases the likelihood of the model repeating the same lines verbatim. presence\_penalty numberminimum: -2maximum: 2Increases the likelihood of the model introducing new topics. id stringUnique identifier for the completion object stringenum: chat.completionObject type identifier created numberUnix timestamp of when the completion was created model stringModel used for the completion ▶choices\[] arrayList of completion choices ▶usage{} objectUsage statistics for the inference request prompt\_logprobs{} objectLog probabilities for the prompt (if requested)
Streaming — Send a request with `stream: true` and receive server-sent events prompt stringrequiredminLength: 1The input text prompt for the model to generate a response. lora stringName of the LoRA (Low-Rank Adaptation) model to fine-tune the base model. ▶response\_format{} object raw booleandefault: falseIf true, a chat template is not applied and you must adhere to the specific model's expected formatting. stream booleandefault: falseIf true, the response will be streamed back incrementally using SSE, Server Sent Events. max\_tokens integerdefault: 2000The maximum number of tokens to generate in the response. temperature numberdefault: 0.6minimum: 0maximum: 5Controls the randomness of the output; higher values produce more random results. top\_p numberminimum: 0.001maximum: 1Adjusts the creativity of the AI's responses by controlling how many possible words it considers. Lower values make outputs more predictable; higher values allow for more varied and creative responses. top\_k integerminimum: 1maximum: 50Limits the AI to choose from the top 'k' most probable words. Lower values make responses more focused; higher values introduce more variety and potential surprises. seed integerminimum: 1maximum: 9999999999Random seed for reproducibility of the generation. repetition\_penalty numberminimum: 0maximum: 2Penalty for repeated tokens; higher values discourage repetition. frequency\_penalty numberminimum: -2maximum: 2Decreases the likelihood of the model repeating the same lines verbatim. presence\_penalty numberminimum: -2maximum: 2Increases the likelihood of the model introducing new topics. type string contentType text/event-stream format binary
Batch — Send multiple requests in a single API call ▶requests\[] arrayrequired id stringUnique identifier for the completion object stringenum: chat.completionObject type identifier created numberUnix timestamp of when the completion was created model stringModel used for the completion ▶choices\[] arrayList of completion choices ▶usage{} objectUsage statistics for the inference request prompt\_logprobs{} objectLog probabilities for the prompt (if requested)
## API Schemas (Raw) SynchronousInput [Open](https://developers.cloudflare.com/workers-ai/models/qwen3-30b-a3b-fp8/sync-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/qwen3-30b-a3b-fp8/sync-input.json) SynchronousOutput [Open](https://developers.cloudflare.com/workers-ai/models/qwen3-30b-a3b-fp8/sync-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/qwen3-30b-a3b-fp8/sync-output.json) StreamingInput [Open](https://developers.cloudflare.com/workers-ai/models/qwen3-30b-a3b-fp8/streaming-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/qwen3-30b-a3b-fp8/streaming-input.json) StreamingOutput [Open](https://developers.cloudflare.com/workers-ai/models/qwen3-30b-a3b-fp8/streaming-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/qwen3-30b-a3b-fp8/streaming-output.json) BatchInput [Open](https://developers.cloudflare.com/workers-ai/models/qwen3-30b-a3b-fp8/batch-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/qwen3-30b-a3b-fp8/batch-input.json) BatchOutput [Open](https://developers.cloudflare.com/workers-ai/models/qwen3-30b-a3b-fp8/batch-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/qwen3-30b-a3b-fp8/batch-output.json) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/models/qwen3-30b-a3b-fp8/#page","headline":"qwen3-30b-a3b-fp8","description":"Qwen3 is the latest generation of large language models in Qwen series, offering a comprehensive suite of dense and mixture-of-experts (MoE) models. Built upon extensive training, Qwen3 delivers groundbreaking advancements in reasoning, instruction-following, agent capabilities, and multilingual support.","url":"https://developers.cloudflare.com/workers-ai/models/qwen3-30b-a3b-fp8/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: "The Qwen3 Embedding model series is the latest proprietary model of the Qwen family, specifically designed for text embedding and ranking tasks. " title: qwen3-embedding-0.6b image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/llms.txt > Use this file to discover all available pages before exploring further. ![Qwen logo](https://developers.cloudflare.com/_astro/qwen.ByCZjtXU.svg) # qwen3-embedding-0.6b Text Embeddings • Qwen Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/models/qwen3-embedding-0.6b/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Model ID `@cf/qwen/qwen3-embedding-0.6b` Copy - Cloudflare-hosted - Zero data retention The Qwen3 Embedding model series is the latest proprietary model of the Qwen family, specifically designed for text embedding and ranking tasks. | Model Info | | | --- | --- | | Context Window [ ↗](https://developers.cloudflare.com/workers-ai/platform/glossary/) | 8,192 tokens | | Zero data retention | Yes | | Unit Pricing | $0.0118 per M input tokens | ## Usage ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { // Can be a string or array of strings] const stories = [ "This is a story about an orange cloud", "This is a story about a llama", "This is a story about a hugging emoji", ]; const embeddings = await env.AI.run( "@cf/qwen/qwen3-embedding-0.6b", { text: stories, } ); return Response.json(embeddings); }, } satisfies ExportedHandler; ``` ```py import os import requests ACCOUNT_ID = "your-account-id" AUTH_TOKEN = os.environ.get("CLOUDFLARE_AUTH_TOKEN") stories = [ 'This is a story about an orange cloud', 'This is a story about a llama', 'This is a story about a hugging emoji' ] response = requests.post( f"https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai/run/@cf/qwen/qwen3-embedding-0.6b", headers={"Authorization": f"Bearer {AUTH_TOKEN}"}, json={"text": stories} ) print(response.json()) ``` ```sh curl https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/run/@cf/qwen/qwen3-embedding-0.6b \ -X POST \ -H "Authorization: Bearer $CLOUDFLARE_API_TOKEN" \ -d '{ "text": ["This is a story about an orange cloud", "This is a story about a llama", "This is a story about a hugging emoji"] }' ``` OpenAI compatible endpoints Workers AI also supports OpenAI compatible API endpoints for `/v1/chat/completions` and `/v1/embeddings`. For more details, refer to [Configurations](https://developers.cloudflare.com/workers-ai/configuration/open-ai-compatibility/). ## Parameters ▶queries `one of` instruction `string`default: Given a web search query, retrieve relevant passages that answer the queryOptional instruction for the task ▶documents `one of` ▶text `one of` ▶data\[] `array` ▶shape\[] `array` ## API Schemas (Raw) Input [Open](https://developers.cloudflare.com/workers-ai/models/qwen3-embedding-0.6b/schema-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/qwen3-embedding-0.6b/schema-input.json) Output [Open](https://developers.cloudflare.com/workers-ai/models/qwen3-embedding-0.6b/schema-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/qwen3-embedding-0.6b/schema-output.json) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/models/qwen3-embedding-0.6b/#page","headline":"qwen3-embedding-0.6b","description":"The Qwen3 Embedding model series is the latest proprietary model of the Qwen family, specifically designed for text embedding and ranking tasks.","url":"https://developers.cloudflare.com/workers-ai/models/qwen3-embedding-0.6b/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Qwen 3.8 27B is a 27-billion-parameter instruction-tuned language model from Alibaba's Qwen family, designed for vision, efficient general-purpose text generation and agentic workloads. title: qwen3.8-27b image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/llms.txt > Use this file to discover all available pages before exploring further. ![Qwen logo](https://developers.cloudflare.com/_astro/qwen.ByCZjtXU.svg) # qwen3.8-27b Text Generation • Qwen Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/models/qwen3.8-27b/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Model ID `@cf/qwen/qwen3.8-27b` Copy - Cloudflare-hosted - Batch - Function calling - Reasoning - Vision - Zero data retention Qwen 3.8 27B is a 27-billion-parameter instruction-tuned language model from Alibaba's Qwen family, designed for vision, efficient general-purpose text generation and agentic workloads. | Model Info | | | --- | --- | | Context Window [ ↗](https://developers.cloudflare.com/workers-ai/platform/glossary/) | 262,144 tokens | | Function calling [↗](https://developers.cloudflare.com/workers-ai/function-calling/) | Yes | | Reasoning | `low``medium``xhigh` (default) | | Vision | Yes | | Zero data retention | Yes | | Batch | Yes | | Unit Pricing | $0.45 per M input tokens, $3.20 per M output tokens, $0.05 per M cached input tokens | ## Playground Try out this model with Workers AI LLM Playground. It does not require any setup or authentication and is an instant way to preview and test a model directly in the browser. [Launch the LLM Playground](https://playground.ai.cloudflare.com/?model=@cf/qwen/qwen3.8-27b) ## Usage ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const messages = [ { role: "system", content: "You are a friendly assistant" }, { role: "user", content: "What is the origin of the phrase Hello, World", }, ]; const stream = await env.AI.run("@cf/qwen/qwen3.8-27b", { messages, stream: true, }); return new Response(stream, { headers: { "content-type": "text/event-stream" }, }); }, } satisfies ExportedHandler; ``` ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const messages = [ { role: "system", content: "You are a friendly assistant" }, { role: "user", content: "What is the origin of the phrase Hello, World", }, ]; const response = await env.AI.run("@cf/qwen/qwen3.8-27b", { messages }); return Response.json(response); }, } satisfies ExportedHandler; ``` ```py import os import requests ACCOUNT_ID = "your-account-id" AUTH_TOKEN = os.environ.get("CLOUDFLARE_AUTH_TOKEN") prompt = "Tell me all about PEP-8" response = requests.post( f"https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai/run/@cf/qwen/qwen3.8-27b", headers={"Authorization": f"Bearer {AUTH_TOKEN}"}, json={ "messages": [ {"role": "system", "content": "You are a friendly assistant"}, {"role": "user", "content": prompt} ] } ) result = response.json() print(result) ``` ```sh curl https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/run/@cf/qwen/qwen3.8-27b \ -X POST \ -H "Authorization: Bearer $CLOUDFLARE_AUTH_TOKEN" \ -d '{ "messages": [{ "role": "system", "content": "You are a friendly assistant" }, { "role": "user", "content": "Why is pizza so good" }]}' ``` OpenAI compatible endpoints Workers AI also supports OpenAI compatible API endpoints for `/v1/chat/completions` and `/v1/embeddings`. For more details, refer to [Configurations](https://developers.cloudflare.com/workers-ai/configuration/open-ai-compatibility/) . ## Parameters
Synchronous — Send a request and receive a complete response Input format Prompt Simple text input for single-turn interactionsMessages Structured conversation format with roles (user, assistant, system) prompt stringrequiredminLength: 1The input text prompt for the model to generate a response. model stringID of the model to use (e.g. '@cf/zai-org/glm-4.7-flash, etc'). ▶audio{} objectParameters for audio output. Required when modalities includes 'audio'. frequency\_penalty number | nullPenalizes new tokens based on their existing frequency in the text so far. logit\_bias object | nullModify the likelihood of specified tokens appearing in the completion. Maps token IDs to bias values from -100 to 100. logprobs boolean | nullWhether to return log probabilities of the output tokens. top\_logprobs integer | nullHow many top log probabilities to return at each token position (0-20). Requires logprobs=true. max\_tokens integer | nullDeprecated in favor of max_completion_tokens. The maximum number of tokens to generate. max\_completion\_tokens integer | nullAn upper bound for the number of tokens that can be generated for a completion. metadata object | nullSet of 16 key-value pairs that can be attached to the object. modalities array | nullOutput types requested from the model (e.g. ['text'] or ['text', 'audio']). n integer | nullHow many chat completion choices to generate for each input message. parallel\_tool\_calls booleandefault: trueWhether to enable parallel function calling during tool use. ▶prediction{} object presence\_penalty number | nullPenalizes new tokens based on whether they appear in the text so far. reasoning\_effort string | nullenum: low, medium, xhighReasoning effort. Supported levels: low, medium, xhigh. ▶chat\_template\_kwargs{} object ▶response\_format one ofSpecifies the format the model must output. seed integer | nullIf specified, the system will make a best effort to sample deterministically. ▶stop one of store boolean | nullWhether to store the output for model distillation / evals. stream boolean | nullIf true, partial message deltas will be sent as server-sent events. ▶stream\_options{} object temperature number | nullSampling temperature between 0 and 2. ▶tool\_choice one ofControls which (if any) tool is called by the model. 'none' = no tools, 'auto' = model decides, 'required' = must call a tool. ▶tools\[] arrayA list of tools the model may call. top\_p number | nullNucleus sampling: considers the results of the tokens with top_p probability mass. user stringA unique identifier representing your end-user, for abuse monitoring. ▶web\_search\_options{} objectOptions for the web search tool (when using built-in web search). ▶function\_call one of ▶functions\[] arrayminItems: 1maxItems: 128 id stringA unique identifier for the chat completion. object string created integerUnix timestamp (seconds) of when the completion was created. model stringThe model used for the chat completion. ▶choices\[] arrayminItems: 1 ▶usage{} object system\_fingerprint string | null
Streaming — Send a request with `stream: true` and receive server-sent events Input format Prompt Simple text input for single-turn interactionsMessages Structured conversation format with roles (user, assistant, system) prompt stringrequiredminLength: 1The input text prompt for the model to generate a response. model stringID of the model to use (e.g. '@cf/zai-org/glm-4.7-flash, etc'). ▶audio{} objectParameters for audio output. Required when modalities includes 'audio'. frequency\_penalty number | nullPenalizes new tokens based on their existing frequency in the text so far. logit\_bias object | nullModify the likelihood of specified tokens appearing in the completion. Maps token IDs to bias values from -100 to 100. logprobs boolean | nullWhether to return log probabilities of the output tokens. top\_logprobs integer | nullHow many top log probabilities to return at each token position (0-20). Requires logprobs=true. max\_tokens integer | nullDeprecated in favor of max_completion_tokens. The maximum number of tokens to generate. max\_completion\_tokens integer | nullAn upper bound for the number of tokens that can be generated for a completion. metadata object | nullSet of 16 key-value pairs that can be attached to the object. modalities array | nullOutput types requested from the model (e.g. ['text'] or ['text', 'audio']). n integer | nullHow many chat completion choices to generate for each input message. parallel\_tool\_calls booleandefault: trueWhether to enable parallel function calling during tool use. ▶prediction{} object presence\_penalty number | nullPenalizes new tokens based on whether they appear in the text so far. reasoning\_effort string | nullenum: low, medium, xhighReasoning effort. Supported levels: low, medium, xhigh. ▶chat\_template\_kwargs{} object ▶response\_format one ofSpecifies the format the model must output. seed integer | nullIf specified, the system will make a best effort to sample deterministically. ▶stop one of store boolean | nullWhether to store the output for model distillation / evals. stream boolean | nullIf true, partial message deltas will be sent as server-sent events. ▶stream\_options{} object temperature number | nullSampling temperature between 0 and 2. ▶tool\_choice one ofControls which (if any) tool is called by the model. 'none' = no tools, 'auto' = model decides, 'required' = must call a tool. ▶tools\[] arrayA list of tools the model may call. top\_p number | nullNucleus sampling: considers the results of the tokens with top_p probability mass. user stringA unique identifier representing your end-user, for abuse monitoring. ▶web\_search\_options{} objectOptions for the web search tool (when using built-in web search). ▶function\_call one of ▶functions\[] arrayminItems: 1maxItems: 128 type string contentType text/event-stream format binary
Batch — Send multiple requests in a single API call ▶requests\[] array id stringA unique identifier for the chat completion. object string created integerUnix timestamp (seconds) of when the completion was created. model stringThe model used for the chat completion. ▶choices\[] arrayminItems: 1 ▶usage{} object system\_fingerprint string | null
## API Schemas (Raw) SynchronousInput [Open](https://developers.cloudflare.com/workers-ai/models/qwen3.8-27b/sync-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/qwen3.8-27b/sync-input.json) SynchronousOutput [Open](https://developers.cloudflare.com/workers-ai/models/qwen3.8-27b/sync-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/qwen3.8-27b/sync-output.json) StreamingInput [Open](https://developers.cloudflare.com/workers-ai/models/qwen3.8-27b/streaming-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/qwen3.8-27b/streaming-input.json) StreamingOutput [Open](https://developers.cloudflare.com/workers-ai/models/qwen3.8-27b/streaming-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/qwen3.8-27b/streaming-output.json) BatchInput [Open](https://developers.cloudflare.com/workers-ai/models/qwen3.8-27b/batch-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/qwen3.8-27b/batch-input.json) BatchOutput [Open](https://developers.cloudflare.com/workers-ai/models/qwen3.8-27b/batch-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/qwen3.8-27b/batch-output.json) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/models/qwen3.8-27b/#page","headline":"qwen3.8-27b","description":"Qwen 3.8 27B is a 27-billion-parameter instruction-tuned language model from Alibaba's Qwen family, designed for vision, efficient general-purpose text generation and agentic workloads.","url":"https://developers.cloudflare.com/workers-ai/models/qwen3.8-27b/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: QwQ is the reasoning model of the Qwen series. Compared with conventional instruction-tuned models, QwQ, which is capable of thinking and reasoning, can achieve significantly enhanced performance in downstream tasks, especially hard problems. QwQ-32B is the medium-sized reasoning model, which is capable of achieving competitive performance against state-of-the-art reasoning models, e.g., DeepSeek-R1, o1-mini. title: qwq-32b image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/llms.txt > Use this file to discover all available pages before exploring further. ![Qwen logo](https://developers.cloudflare.com/_astro/qwen.ByCZjtXU.svg) # qwq-32b Text Generation • Qwen Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/models/qwq-32b/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Model ID `@cf/qwen/qwq-32b` Copy - Cloudflare-hosted - LoRA - Reasoning - Planned deprecation - Zero data retention QwQ is the reasoning model of the Qwen series. Compared with conventional instruction-tuned models, QwQ, which is capable of thinking and reasoning, can achieve significantly enhanced performance in downstream tasks, especially hard problems. QwQ-32B is the medium-sized reasoning model, which is capable of achieving competitive performance against state-of-the-art reasoning models, e.g., DeepSeek-R1, o1-mini. | Model Info | | | --- | --- | | Planned Deprecation | 11/20/2026 | | Context Window [ ↗](https://developers.cloudflare.com/workers-ai/platform/glossary/) | 24,000 tokens | | Reasoning | Yes | | Zero data retention | Yes | | LoRA | Yes | | Unit Pricing | $0.66 per M input tokens, $1.00 per M output tokens | ## Playground Try out this model with Workers AI LLM Playground. It does not require any setup or authentication and is an instant way to preview and test a model directly in the browser. [Launch the LLM Playground](https://playground.ai.cloudflare.com/?model=@cf/qwen/qwq-32b) ## Usage ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const messages = [ { role: "system", content: "You are a friendly assistant" }, { role: "user", content: "What is the origin of the phrase Hello, World", }, ]; const stream = await env.AI.run("@cf/qwen/qwq-32b", { messages, stream: true, }); return new Response(stream, { headers: { "content-type": "text/event-stream" }, }); }, } satisfies ExportedHandler; ``` ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const messages = [ { role: "system", content: "You are a friendly assistant" }, { role: "user", content: "What is the origin of the phrase Hello, World", }, ]; const response = await env.AI.run("@cf/qwen/qwq-32b", { messages }); return Response.json(response); }, } satisfies ExportedHandler; ``` ```py import os import requests ACCOUNT_ID = "your-account-id" AUTH_TOKEN = os.environ.get("CLOUDFLARE_AUTH_TOKEN") prompt = "Tell me all about PEP-8" response = requests.post( f"https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai/run/@cf/qwen/qwq-32b", headers={"Authorization": f"Bearer {AUTH_TOKEN}"}, json={ "messages": [ {"role": "system", "content": "You are a friendly assistant"}, {"role": "user", "content": prompt} ] } ) result = response.json() print(result) ``` ```sh curl https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/run/@cf/qwen/qwq-32b \ -X POST \ -H "Authorization: Bearer $CLOUDFLARE_AUTH_TOKEN" \ -d '{ "messages": [{ "role": "system", "content": "You are a friendly assistant" }, { "role": "user", "content": "Why is pizza so good" }]}' ``` OpenAI compatible endpoints Workers AI also supports OpenAI compatible API endpoints for `/v1/chat/completions` and `/v1/embeddings`. For more details, refer to [Configurations](https://developers.cloudflare.com/workers-ai/configuration/open-ai-compatibility/) . ## Parameters ### Input prompt `string`requiredminLength: 1The input text prompt for the model to generate a response. guided\_json{} `object`JSON schema that should be fulfilled for the response. raw `boolean`default: falseIf true, a chat template is not applied and you must adhere to the specific model's expected formatting. stream `boolean`default: falseIf true, the response will be streamed back incrementally using SSE, Server Sent Events. max\_tokens `integer`default: 256The maximum number of tokens to generate in the response. temperature `number`default: 0.15minimum: 0maximum: 5Controls the randomness of the output; higher values produce more random results. top\_p `number`minimum: 0maximum: 2Adjusts the creativity of the AI's responses by controlling how many possible words it considers. Lower values make outputs more predictable; higher values allow for more varied and creative responses. top\_k `integer`minimum: 1maximum: 50Limits the AI to choose from the top 'k' most probable words. Lower values make responses more focused; higher values introduce more variety and potential surprises. seed `integer`minimum: 1maximum: 9999999999Random seed for reproducibility of the generation. repetition\_penalty `number`minimum: 0maximum: 2Penalty for repeated tokens; higher values discourage repetition. frequency\_penalty `number`minimum: 0maximum: 2Decreases the likelihood of the model repeating the same lines verbatim. presence\_penalty `number`minimum: 0maximum: 2Increases the likelihood of the model introducing new topics. ### Output
Synchronous — Send a request and receive a complete response response stringThe generated text response from the model ▶usage{} objectUsage statistics for the inference request ▶tool\_calls\[] arrayAn array of tool calls requests made during the response generation
Streaming — Send a request with `stream: true` and receive server-sent events type string contentType text/event-stream format binary
## API Schemas (Raw) SynchronousInput [Open](https://developers.cloudflare.com/workers-ai/models/qwq-32b/sync-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/qwq-32b/sync-input.json) SynchronousOutput [Open](https://developers.cloudflare.com/workers-ai/models/qwq-32b/sync-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/qwq-32b/sync-output.json) StreamingInput [Open](https://developers.cloudflare.com/workers-ai/models/qwq-32b/streaming-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/qwq-32b/streaming-input.json) StreamingOutput [Open](https://developers.cloudflare.com/workers-ai/models/qwq-32b/streaming-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/qwq-32b/streaming-output.json) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/models/qwq-32b/#page","headline":"qwq-32b","description":"QwQ is the reasoning model of the Qwen series. Compared with conventional instruction-tuned models, QwQ, which is capable of thinking and reasoning, can achieve significantly enhanced performance in downstream tasks, especially hard problems. QwQ-32B is the medium-sized reasoning model, which is capable of achieving competitive performance against state-of-the-art reasoning models, e.g., DeepSeek-R1, o1-mini.","url":"https://developers.cloudflare.com/workers-ai/models/qwq-32b/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Stable Diffusion Inpainting is a latent text-to-image diffusion model capable of generating photo-realistic images given any text input, with the extra capability of inpainting the pictures by using a mask. title: stable-diffusion-v1-5-inpainting image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/llms.txt > Use this file to discover all available pages before exploring further. ![RunwayML logo](https://developers.cloudflare.com/_astro/runway.Cq8Cjov4.svg) # stable-diffusion-v1-5-inpainting Beta Text-to-Image • RunwayML Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/models/stable-diffusion-v1-5-inpainting/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Model ID `@cf/runwayml/stable-diffusion-v1-5-inpainting` Copy - Cloudflare-hosted - Planned deprecation - Zero data retention Stable Diffusion Inpainting is a latent text-to-image diffusion model capable of generating photo-realistic images given any text input, with the extra capability of inpainting the pictures by using a mask. | Model Info | | | --- | --- | | Planned Deprecation | 11/20/2026 | | Terms and License | [link ↗](https://github.com/runwayml/stable-diffusion/blob/main/LICENSE) | | More information | [link ↗](https://huggingface.co/runwayml/stable-diffusion-inpainting) | | Zero data retention | Yes | | Beta | Yes | | Unit Pricing | $0.00 per step | ## Parameters prompt `string`requiredminLength: 1A text description of the image you want to generate negative\_prompt `string`Text describing elements to avoid in the generated image height `integer`minimum: 256maximum: 2048The height of the generated image in pixels width `integer`minimum: 256maximum: 2048The width of the generated image in pixels ▶image\[] `array`For use with img2img tasks. An array of integers that represent the image data constrained to 8-bit unsigned integer values image\_b64 `string`For use with img2img tasks. A base64-encoded string of the input image ▶mask\[] `array`An array representing An array of integers that represent mask image data for inpainting constrained to 8-bit unsigned integer values num\_steps `integer`default: 20maximum: 20The number of diffusion steps; higher values can improve quality but take longer strength `number`default: 1A value between 0 and 1 indicating how strongly to apply the transformation during img2img tasks; lower values make the output closer to the input image guidance `number`default: 7.5Controls how closely the generated image should adhere to the prompt; higher values make the image more aligned with the prompt seed `integer`Random seed for reproducibility of the image generation The binding returns a `ReadableStream` with the output (check the model's output schema). ## API Schemas (Raw) Input [Open](https://developers.cloudflare.com/workers-ai/models/stable-diffusion-v1-5-inpainting/schema-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/stable-diffusion-v1-5-inpainting/schema-input.json) Output [Open](https://developers.cloudflare.com/workers-ai/models/stable-diffusion-v1-5-inpainting/schema-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/stable-diffusion-v1-5-inpainting/schema-output.json) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/models/stable-diffusion-v1-5-inpainting/#page","headline":"stable-diffusion-v1-5-inpainting","description":"Stable Diffusion Inpainting is a latent text-to-image diffusion model capable of generating photo-realistic images given any text input, with the extra capability of inpainting the pictures by using a mask.","url":"https://developers.cloudflare.com/workers-ai/models/stable-diffusion-v1-5-inpainting/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Diffusion-based text-to-image generative model by Stability AI. Generates and modify images based on text prompts. title: stable-diffusion-xl-base-1.0 image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/llms.txt > Use this file to discover all available pages before exploring further. ![Stability.ai logo](https://developers.cloudflare.com/_astro/stabilityai.VsBx3CKv.svg) # stable-diffusion-xl-base-1.0 Beta Text-to-Image • Stability.ai Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/models/stable-diffusion-xl-base-1.0/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Model ID `@cf/stabilityai/stable-diffusion-xl-base-1.0` Copy - Cloudflare-hosted - Planned deprecation - Zero data retention Diffusion-based text-to-image generative model by Stability AI. Generates and modify images based on text prompts. | Model Info | | | --- | --- | | Planned Deprecation | 11/20/2026 | | Terms and License | [link ↗](https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0/blob/main/LICENSE.md) | | More information | [link ↗](https://stability.ai/stable-diffusion) | | Zero data retention | Yes | | Beta | Yes | | Unit Pricing | $0.00 per step | ## Parameters prompt `string`requiredminLength: 1A text description of the image you want to generate negative\_prompt `string`Text describing elements to avoid in the generated image height `integer`minimum: 256maximum: 2048The height of the generated image in pixels width `integer`minimum: 256maximum: 2048The width of the generated image in pixels ▶image\[] `array`For use with img2img tasks. An array of integers that represent the image data constrained to 8-bit unsigned integer values image\_b64 `string`For use with img2img tasks. A base64-encoded string of the input image ▶mask\[] `array`An array representing An array of integers that represent mask image data for inpainting constrained to 8-bit unsigned integer values num\_steps `integer`default: 20maximum: 20The number of diffusion steps; higher values can improve quality but take longer strength `number`default: 1A value between 0 and 1 indicating how strongly to apply the transformation during img2img tasks; lower values make the output closer to the input image guidance `number`default: 7.5Controls how closely the generated image should adhere to the prompt; higher values make the image more aligned with the prompt seed `integer`Random seed for reproducibility of the image generation The binding returns a `ReadableStream` with the output (check the model's output schema). ## API Schemas (Raw) Input [Open](https://developers.cloudflare.com/workers-ai/models/stable-diffusion-xl-base-1.0/schema-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/stable-diffusion-xl-base-1.0/schema-input.json) Output [Open](https://developers.cloudflare.com/workers-ai/models/stable-diffusion-xl-base-1.0/schema-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/stable-diffusion-xl-base-1.0/schema-output.json) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/models/stable-diffusion-xl-base-1.0/#page","headline":"stable-diffusion-xl-base-1.0","description":"Diffusion-based text-to-image generative model by Stability AI. Generates and modify images based on text prompts.","url":"https://developers.cloudflare.com/workers-ai/models/stable-diffusion-xl-base-1.0/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: "Apertus 1.5 is an 8B parameter language model designed to advance the state of multilingual, multimodal, fully open, and transparent AI. The models support a wide range of languages, handle contexts of up to 262,144 tokens, and it uses only fully open training data whilst delivering performance comparable to other models of similar size. For access, please fill out this form: https://forms.gle/kgvmkr6ucHN3xNuH6" title: apertus-v1.5-8b image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/llms.txt > Use this file to discover all available pages before exploring further. s # apertus-v1.5-8b Text Generation • swiss-ai Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/models/apertus-v1.5-8b/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Model ID `@cf/swiss-ai/apertus-v1.5-8b` Copy - Cloudflare-hosted - Function calling - Vision - Zero data retention Apertus 1.5 is an 8B parameter language model designed to advance the state of multilingual, multimodal, fully open, and transparent AI. The models support a wide range of languages, handle contexts of up to 262,144 tokens, and it uses only fully open training data whilst delivering performance comparable to other models of similar size. For access, please fill out this form: https://forms.gle/kgvmkr6ucHN3xNuH6 | Model Info | | | --- | --- | | Context Window [ ↗](https://developers.cloudflare.com/workers-ai/platform/glossary/) | 262,144 tokens | | Function calling [↗](https://developers.cloudflare.com/workers-ai/function-calling/) | Yes | | Vision | Yes | | Zero data retention | Yes | ## Playground Try out this model with Workers AI LLM Playground. It does not require any setup or authentication and is an instant way to preview and test a model directly in the browser. [Launch the LLM Playground](https://playground.ai.cloudflare.com/?model=@cf/swiss-ai/apertus-v1.5-8b) ## Usage ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const messages = [ { role: "system", content: "You are a friendly assistant" }, { role: "user", content: "What is the origin of the phrase Hello, World", }, ]; const stream = await env.AI.run("@cf/swiss-ai/apertus-v1.5-8b", { messages, stream: true, }); return new Response(stream, { headers: { "content-type": "text/event-stream" }, }); }, } satisfies ExportedHandler; ``` ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const messages = [ { role: "system", content: "You are a friendly assistant" }, { role: "user", content: "What is the origin of the phrase Hello, World", }, ]; const response = await env.AI.run("@cf/swiss-ai/apertus-v1.5-8b", { messages }); return Response.json(response); }, } satisfies ExportedHandler; ``` ```py import os import requests ACCOUNT_ID = "your-account-id" AUTH_TOKEN = os.environ.get("CLOUDFLARE_AUTH_TOKEN") prompt = "Tell me all about PEP-8" response = requests.post( f"https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai/run/@cf/swiss-ai/apertus-v1.5-8b", headers={"Authorization": f"Bearer {AUTH_TOKEN}"}, json={ "messages": [ {"role": "system", "content": "You are a friendly assistant"}, {"role": "user", "content": prompt} ] } ) result = response.json() print(result) ``` ```sh curl https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/run/@cf/swiss-ai/apertus-v1.5-8b \ -X POST \ -H "Authorization: Bearer $CLOUDFLARE_AUTH_TOKEN" \ -d '{ "messages": [{ "role": "system", "content": "You are a friendly assistant" }, { "role": "user", "content": "Why is pizza so good" }]}' ``` OpenAI compatible endpoints Workers AI also supports OpenAI compatible API endpoints for `/v1/chat/completions` and `/v1/embeddings`. For more details, refer to [Configurations](https://developers.cloudflare.com/workers-ai/configuration/open-ai-compatibility/) . Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/models/apertus-v1.5-8b/#page","headline":"apertus-v1.5-8b","description":"Apertus 1.5 is an 8B parameter language model designed to advance the state of multilingual, multimodal, fully open, and transparent AI. The models support a wide range of languages, handle contexts of up to 262,144 tokens, and it uses only fully open training data whilst delivering performance comparable to other models of similar size. For access, please fill out this form: https://forms.gle/kgvmkr6ucHN3xNuH6","url":"https://developers.cloudflare.com/workers-ai/models/apertus-v1.5-8b/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: "EuroLLM-9B is a 9B parameter model trained on 4 trillion tokens divided across the considered languages and several data sources: Web data, parallel data (en-xx and xx-en), and high-quality datasets. For access, please fill out this form: https://forms.gle/kgvmkr6ucHN3xNuH6" title: eurollm-9b-it image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/llms.txt > Use this file to discover all available pages before exploring further. u # eurollm-9b-it Text Generation • utter-project Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/models/eurollm-9b-it/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Model ID `@cf/utter-project/eurollm-9b-it` Copy - Cloudflare-hosted - Function calling - Zero data retention EuroLLM-9B is a 9B parameter model trained on 4 trillion tokens divided across the considered languages and several data sources: Web data, parallel data (en-xx and xx-en), and high-quality datasets. For access, please fill out this form: https://forms.gle/kgvmkr6ucHN3xNuH6 | Model Info | | | --- | --- | | Context Window [ ↗](https://developers.cloudflare.com/workers-ai/platform/glossary/) | 32,000 tokens | | Function calling [↗](https://developers.cloudflare.com/workers-ai/function-calling/) | Yes | | Zero data retention | Yes | ## Playground Try out this model with Workers AI LLM Playground. It does not require any setup or authentication and is an instant way to preview and test a model directly in the browser. [Launch the LLM Playground](https://playground.ai.cloudflare.com/?model=@cf/utter-project/eurollm-9b-it) ## Usage ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const messages = [ { role: "system", content: "You are a friendly assistant" }, { role: "user", content: "What is the origin of the phrase Hello, World", }, ]; const stream = await env.AI.run("@cf/utter-project/eurollm-9b-it", { messages, stream: true, }); return new Response(stream, { headers: { "content-type": "text/event-stream" }, }); }, } satisfies ExportedHandler; ``` ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const messages = [ { role: "system", content: "You are a friendly assistant" }, { role: "user", content: "What is the origin of the phrase Hello, World", }, ]; const response = await env.AI.run("@cf/utter-project/eurollm-9b-it", { messages }); return Response.json(response); }, } satisfies ExportedHandler; ``` ```py import os import requests ACCOUNT_ID = "your-account-id" AUTH_TOKEN = os.environ.get("CLOUDFLARE_AUTH_TOKEN") prompt = "Tell me all about PEP-8" response = requests.post( f"https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai/run/@cf/utter-project/eurollm-9b-it", headers={"Authorization": f"Bearer {AUTH_TOKEN}"}, json={ "messages": [ {"role": "system", "content": "You are a friendly assistant"}, {"role": "user", "content": prompt} ] } ) result = response.json() print(result) ``` ```sh curl https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/run/@cf/utter-project/eurollm-9b-it \ -X POST \ -H "Authorization: Bearer $CLOUDFLARE_AUTH_TOKEN" \ -d '{ "messages": [{ "role": "system", "content": "You are a friendly assistant" }, { "role": "user", "content": "Why is pizza so good" }]}' ``` OpenAI compatible endpoints Workers AI also supports OpenAI compatible API endpoints for `/v1/chat/completions` and `/v1/embeddings`. For more details, refer to [Configurations](https://developers.cloudflare.com/workers-ai/configuration/open-ai-compatibility/) . Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/models/eurollm-9b-it/#page","headline":"eurollm-9b-it","description":"EuroLLM-9B is a 9B parameter model trained on 4 trillion tokens divided across the considered languages and several data sources: Web data, parallel data (en-xx and xx-en), and high-quality datasets. For access, please fill out this form: https://forms.gle/kgvmkr6ucHN3xNuH6","url":"https://developers.cloudflare.com/workers-ai/models/eurollm-9b-it/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: GLM-4.7-Flash is a fast and efficient multilingual text generation model with a 131,072 token context window. Optimized for dialogue, instruction-following, and multi-turn tool calling across 100+ languages. title: glm-4.7-flash image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/llms.txt > Use this file to discover all available pages before exploring further. ![Zhipu AI logo](https://developers.cloudflare.com/_astro/zai-org.Dj2vcayE.svg) # glm-4.7-flash Text Generation • Zhipu AI Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/models/glm-4.7-flash/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Model ID `@cf/zai-org/glm-4.7-flash` Copy - Cloudflare-hosted - Function calling - Reasoning - Zero data retention GLM-4.7-Flash is a fast and efficient multilingual text generation model with a 131,072 token context window. Optimized for dialogue, instruction-following, and multi-turn tool calling across 100+ languages. | Model Info | | | --- | --- | | Context Window [ ↗](https://developers.cloudflare.com/workers-ai/platform/glossary/) | 131,072 tokens | | Function calling [↗](https://developers.cloudflare.com/workers-ai/function-calling/) | Yes | | Reasoning | Yes | | Zero data retention | Yes | | Unit Pricing | $0.0605 per M input tokens, $0.40 per M output tokens | ## Playground Try out this model with Workers AI LLM Playground. It does not require any setup or authentication and is an instant way to preview and test a model directly in the browser. [Launch the LLM Playground](https://playground.ai.cloudflare.com/?model=@cf/zai-org/glm-4.7-flash) ## Usage ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const messages = [ { role: "system", content: "You are a friendly assistant" }, { role: "user", content: "What is the origin of the phrase Hello, World", }, ]; const stream = await env.AI.run("@cf/zai-org/glm-4.7-flash", { messages, stream: true, }); return new Response(stream, { headers: { "content-type": "text/event-stream" }, }); }, } satisfies ExportedHandler; ``` ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const messages = [ { role: "system", content: "You are a friendly assistant" }, { role: "user", content: "What is the origin of the phrase Hello, World", }, ]; const response = await env.AI.run("@cf/zai-org/glm-4.7-flash", { messages }); return Response.json(response); }, } satisfies ExportedHandler; ``` ```py import os import requests ACCOUNT_ID = "your-account-id" AUTH_TOKEN = os.environ.get("CLOUDFLARE_AUTH_TOKEN") prompt = "Tell me all about PEP-8" response = requests.post( f"https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai/run/@cf/zai-org/glm-4.7-flash", headers={"Authorization": f"Bearer {AUTH_TOKEN}"}, json={ "messages": [ {"role": "system", "content": "You are a friendly assistant"}, {"role": "user", "content": prompt} ] } ) result = response.json() print(result) ``` ```sh curl https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/run/@cf/zai-org/glm-4.7-flash \ -X POST \ -H "Authorization: Bearer $CLOUDFLARE_AUTH_TOKEN" \ -d '{ "messages": [{ "role": "system", "content": "You are a friendly assistant" }, { "role": "user", "content": "Why is pizza so good" }]}' ``` OpenAI compatible endpoints Workers AI also supports OpenAI compatible API endpoints for `/v1/chat/completions` and `/v1/embeddings`. For more details, refer to [Configurations](https://developers.cloudflare.com/workers-ai/configuration/open-ai-compatibility/) . ## Parameters ### Input prompt `string`requiredminLength: 1The input text prompt for the model to generate a response. model `string`ID of the model to use (e.g. '@cf/zai-org/glm-4.7-flash, etc'). ▶audio{} `object`Parameters for audio output. Required when modalities includes 'audio'. frequency\_penalty `number | null`Penalizes new tokens based on their existing frequency in the text so far. logit\_bias `object | null`Modify the likelihood of specified tokens appearing in the completion. Maps token IDs to bias values from -100 to 100. logprobs `boolean | null`Whether to return log probabilities of the output tokens. top\_logprobs `integer | null`How many top log probabilities to return at each token position (0-20). Requires logprobs=true. max\_tokens `integer | null`Deprecated in favor of max\_completion\_tokens. The maximum number of tokens to generate. max\_completion\_tokens `integer | null`An upper bound for the number of tokens that can be generated for a completion. metadata `object | null`Set of 16 key-value pairs that can be attached to the object. modalities `array | null`Output types requested from the model (e.g. \['text'] or \['text', 'audio']). n `integer | null`How many chat completion choices to generate for each input message. parallel\_tool\_calls `boolean`default: trueWhether to enable parallel function calling during tool use. ▶prediction{} `object` presence\_penalty `number | null`Penalizes new tokens based on whether they appear in the text so far. ▶chat\_template\_kwargs{} `object` ▶response\_format `one of`Specifies the format the model must output. seed `integer | null`If specified, the system will make a best effort to sample deterministically. ▶stop `one of` store `boolean | null`Whether to store the output for model distillation / evals. stream `boolean | null`If true, partial message deltas will be sent as server-sent events. ▶stream\_options{} `object` temperature `number | null`Sampling temperature between 0 and 2. ▶tool\_choice `one of`Controls which (if any) tool is called by the model. 'none' = no tools, 'auto' = model decides, 'required' = must call a tool. ▶tools\[] `array`A list of tools the model may call. top\_p `number | null`Nucleus sampling: considers the results of the tokens with top\_p probability mass. user `string`A unique identifier representing your end-user, for abuse monitoring. ▶web\_search\_options{} `object`Options for the web search tool (when using built-in web search). ▶function\_call `one of` ▶functions\[] `array`minItems: 1maxItems: 128 ### Output
Synchronous — Send a request and receive a complete response id stringA unique identifier for the chat completion. object string created integerUnix timestamp (seconds) of when the completion was created. model stringThe model used for the chat completion. ▶choices\[] arrayminItems: 1 ▶usage{} object system\_fingerprint string | null
Streaming — Send a request with `stream: true` and receive server-sent events type string contentType text/event-stream format binary
## API Schemas (Raw) SynchronousInput [Open](https://developers.cloudflare.com/workers-ai/models/glm-4.7-flash/sync-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/glm-4.7-flash/sync-input.json) SynchronousOutput [Open](https://developers.cloudflare.com/workers-ai/models/glm-4.7-flash/sync-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/glm-4.7-flash/sync-output.json) StreamingInput [Open](https://developers.cloudflare.com/workers-ai/models/glm-4.7-flash/streaming-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/glm-4.7-flash/streaming-input.json) StreamingOutput [Open](https://developers.cloudflare.com/workers-ai/models/glm-4.7-flash/streaming-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/glm-4.7-flash/streaming-output.json) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/models/glm-4.7-flash/#page","headline":"glm-4.7-flash","description":"GLM-4.7-Flash is a fast and efficient multilingual text generation model with a 131,072 token context window. Optimized for dialogue, instruction-following, and multi-turn tool calling across 100+ languages.","url":"https://developers.cloudflare.com/workers-ai/models/glm-4.7-flash/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Z.ai's flagship agentic coding model title: glm-5.2 image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/llms.txt > Use this file to discover all available pages before exploring further. ![Zhipu AI logo](https://developers.cloudflare.com/_astro/zai-org.Dj2vcayE.svg) # glm-5.2 Text Generation • Zhipu AI Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/models/glm-5.2/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Model ID `@cf/zai-org/glm-5.2` Copy - Cloudflare-hosted - Function calling - Reasoning - Planned deprecation - Zero data retention Z.ai's flagship agentic coding model Paid access required This model is not available through standard Workers Free billing. To use it, upgrade to the [Workers Paid plan](https://developers.cloudflare.com/workers/platform/pricing/#workers) or use prepaid [AI Gateway credits](https://developers.cloudflare.com/ai-gateway/features/unified-billing/). | Model Info | | | --- | --- | | Planned Deprecation | 11/20/2026 | | Context Window [ ↗](https://developers.cloudflare.com/workers-ai/platform/glossary/) | 262,144 tokens | | Function calling [↗](https://developers.cloudflare.com/workers-ai/function-calling/) | Yes | | Reasoning | `none``high``max` (default) | | Zero data retention | Yes | | Unit Pricing | $1.40 per M input tokens, $4.40 per M output tokens, $0.26 per M cached input tokens | ## Reasoning effort GLM-5.2 supports `high` and `max` reasoning effort, or you can turn reasoning off. For compatibility with Chat Completions and other protocols in the ecosystem, other reasoning efforts will be mapped to these modes: - `none` and `minimal` turn reasoning off. - `low` and `medium` map to `high`. - `xhigh` maps to `max`. This is the same mapping used by Z.ai's API, and [in their `reasoning_effort` documentation](https://docs.z.ai/api-reference/llm/chat-completion#body-one-of-0-reasoning-effort). ## Playground Try out this model with Workers AI LLM Playground. It does not require any setup or authentication and is an instant way to preview and test a model directly in the browser. [Launch the LLM Playground](https://playground.ai.cloudflare.com/?model=@cf/zai-org/glm-5.2) ## Usage ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const messages = [ { role: "system", content: "You are a friendly assistant" }, { role: "user", content: "What is the origin of the phrase Hello, World", }, ]; const stream = await env.AI.run("@cf/zai-org/glm-5.2", { messages, stream: true, }); return new Response(stream, { headers: { "content-type": "text/event-stream" }, }); }, } satisfies ExportedHandler; ``` ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const messages = [ { role: "system", content: "You are a friendly assistant" }, { role: "user", content: "What is the origin of the phrase Hello, World", }, ]; const response = await env.AI.run("@cf/zai-org/glm-5.2", { messages }); return Response.json(response); }, } satisfies ExportedHandler; ``` ```py import os import requests ACCOUNT_ID = "your-account-id" AUTH_TOKEN = os.environ.get("CLOUDFLARE_AUTH_TOKEN") prompt = "Tell me all about PEP-8" response = requests.post( f"https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai/run/@cf/zai-org/glm-5.2", headers={"Authorization": f"Bearer {AUTH_TOKEN}"}, json={ "messages": [ {"role": "system", "content": "You are a friendly assistant"}, {"role": "user", "content": prompt} ] } ) result = response.json() print(result) ``` ```sh curl https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/run/@cf/zai-org/glm-5.2 \ -X POST \ -H "Authorization: Bearer $CLOUDFLARE_AUTH_TOKEN" \ -d '{ "messages": [{ "role": "system", "content": "You are a friendly assistant" }, { "role": "user", "content": "Why is pizza so good" }]}' ``` OpenAI compatible endpoints Workers AI also supports OpenAI compatible API endpoints for `/v1/chat/completions` and `/v1/embeddings`. For more details, refer to [Configurations](https://developers.cloudflare.com/workers-ai/configuration/open-ai-compatibility/) . ## Parameters
Synchronous — Send a request and receive a complete response Input format Prompt Simple text input for single-turn interactionsMessages Structured conversation format with roles (user, assistant, system) prompt stringrequiredminLength: 1The input text prompt for the model to generate a response. model stringID of the model to use (e.g. '@cf/zai-org/glm-4.7-flash, etc'). ▶audio{} objectParameters for audio output. Required when modalities includes 'audio'. frequency\_penalty number | nullPenalizes new tokens based on their existing frequency in the text so far. logit\_bias object | nullModify the likelihood of specified tokens appearing in the completion. Maps token IDs to bias values from -100 to 100. logprobs boolean | nullWhether to return log probabilities of the output tokens. top\_logprobs integer | nullHow many top log probabilities to return at each token position (0-20). Requires logprobs=true. max\_tokens integer | nullDeprecated in favor of max_completion_tokens. The maximum number of tokens to generate. max\_completion\_tokens integer | nullAn upper bound for the number of tokens that can be generated for a completion. metadata object | nullSet of 16 key-value pairs that can be attached to the object. modalities array | nullOutput types requested from the model (e.g. ['text'] or ['text', 'audio']). n integer | nullHow many chat completion choices to generate for each input message. parallel\_tool\_calls booleandefault: trueWhether to enable parallel function calling during tool use. ▶prediction{} object presence\_penalty number | nullPenalizes new tokens based on whether they appear in the text so far. reasoning\_effort string | nullenum: max, high, noneReasoning effort. Supported levels: max, high, none. Compatibility aliases: low maps to high; medium maps to high; xhigh maps to max; minimal maps to none. ▶chat\_template\_kwargs{} object ▶response\_format one ofSpecifies the format the model must output. seed integer | nullIf specified, the system will make a best effort to sample deterministically. ▶stop one of store boolean | nullWhether to store the output for model distillation / evals. stream boolean | nullIf true, partial message deltas will be sent as server-sent events. ▶stream\_options{} object temperature number | nullSampling temperature between 0 and 2. ▶tool\_choice one ofControls which (if any) tool is called by the model. 'none' = no tools, 'auto' = model decides, 'required' = must call a tool. ▶tools\[] arrayA list of tools the model may call. top\_p number | nullNucleus sampling: considers the results of the tokens with top_p probability mass. user stringA unique identifier representing your end-user, for abuse monitoring. ▶web\_search\_options{} objectOptions for the web search tool (when using built-in web search). ▶function\_call one of ▶functions\[] arrayminItems: 1maxItems: 128 id stringA unique identifier for the chat completion. object string created integerUnix timestamp (seconds) of when the completion was created. model stringThe model used for the chat completion. ▶choices\[] arrayminItems: 1 ▶usage{} object system\_fingerprint string | null
Streaming — Send a request with `stream: true` and receive server-sent events Input format Prompt Simple text input for single-turn interactionsMessages Structured conversation format with roles (user, assistant, system) prompt stringrequiredminLength: 1The input text prompt for the model to generate a response. model stringID of the model to use (e.g. '@cf/zai-org/glm-4.7-flash, etc'). ▶audio{} objectParameters for audio output. Required when modalities includes 'audio'. frequency\_penalty number | nullPenalizes new tokens based on their existing frequency in the text so far. logit\_bias object | nullModify the likelihood of specified tokens appearing in the completion. Maps token IDs to bias values from -100 to 100. logprobs boolean | nullWhether to return log probabilities of the output tokens. top\_logprobs integer | nullHow many top log probabilities to return at each token position (0-20). Requires logprobs=true. max\_tokens integer | nullDeprecated in favor of max_completion_tokens. The maximum number of tokens to generate. max\_completion\_tokens integer | nullAn upper bound for the number of tokens that can be generated for a completion. metadata object | nullSet of 16 key-value pairs that can be attached to the object. modalities array | nullOutput types requested from the model (e.g. ['text'] or ['text', 'audio']). n integer | nullHow many chat completion choices to generate for each input message. parallel\_tool\_calls booleandefault: trueWhether to enable parallel function calling during tool use. ▶prediction{} object presence\_penalty number | nullPenalizes new tokens based on whether they appear in the text so far. reasoning\_effort string | nullenum: max, high, noneReasoning effort. Supported levels: max, high, none. Compatibility aliases: low maps to high; medium maps to high; xhigh maps to max; minimal maps to none. ▶chat\_template\_kwargs{} object ▶response\_format one ofSpecifies the format the model must output. seed integer | nullIf specified, the system will make a best effort to sample deterministically. ▶stop one of store boolean | nullWhether to store the output for model distillation / evals. stream boolean | nullIf true, partial message deltas will be sent as server-sent events. ▶stream\_options{} object temperature number | nullSampling temperature between 0 and 2. ▶tool\_choice one ofControls which (if any) tool is called by the model. 'none' = no tools, 'auto' = model decides, 'required' = must call a tool. ▶tools\[] arrayA list of tools the model may call. top\_p number | nullNucleus sampling: considers the results of the tokens with top_p probability mass. user stringA unique identifier representing your end-user, for abuse monitoring. ▶web\_search\_options{} objectOptions for the web search tool (when using built-in web search). ▶function\_call one of ▶functions\[] arrayminItems: 1maxItems: 128 type string contentType text/event-stream format binary
Batch — Send multiple requests in a single API call ▶requests\[] array id stringA unique identifier for the chat completion. object string created integerUnix timestamp (seconds) of when the completion was created. model stringThe model used for the chat completion. ▶choices\[] arrayminItems: 1 ▶usage{} object system\_fingerprint string | null
## API Schemas (Raw) SynchronousInput [Open](https://developers.cloudflare.com/workers-ai/models/glm-5.2/sync-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/glm-5.2/sync-input.json) SynchronousOutput [Open](https://developers.cloudflare.com/workers-ai/models/glm-5.2/sync-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/glm-5.2/sync-output.json) StreamingInput [Open](https://developers.cloudflare.com/workers-ai/models/glm-5.2/streaming-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/glm-5.2/streaming-input.json) StreamingOutput [Open](https://developers.cloudflare.com/workers-ai/models/glm-5.2/streaming-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/glm-5.2/streaming-output.json) BatchInput [Open](https://developers.cloudflare.com/workers-ai/models/glm-5.2/batch-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/glm-5.2/batch-input.json) BatchOutput [Open](https://developers.cloudflare.com/workers-ai/models/glm-5.2/batch-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/glm-5.2/batch-output.json) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/models/glm-5.2/#page","headline":"glm-5.2","description":"Z.ai's flagship agentic coding model","url":"https://developers.cloudflare.com/workers-ai/models/glm-5.2/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: GLM-5.3 is Z.ai's flagship agentic coding model, pairing a 1M-token context window with reasoning, function calling, and structured outputs to power multi-step, tool-driven development workflows. title: glm-5.3 image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/llms.txt > Use this file to discover all available pages before exploring further. ![Zhipu AI logo](https://developers.cloudflare.com/_astro/zai-org.Dj2vcayE.svg) # glm-5.3 Text Generation • Zhipu AI Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/models/glm-5.3/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Model ID `@cf/zai-org/glm-5.3` Copy - Cloudflare-hosted - Function calling - Reasoning - Zero data retention GLM-5.3 is Z.ai's flagship agentic coding model, pairing a 1M-token context window with reasoning, function calling, and structured outputs to power multi-step, tool-driven development workflows. Paid access required This model is not available through standard Workers Free billing. To use it, upgrade to the [Workers Paid plan](https://developers.cloudflare.com/workers/platform/pricing/#workers) or use prepaid [AI Gateway credits](https://developers.cloudflare.com/ai-gateway/features/unified-billing/). | Model Info | | | --- | --- | | Context Window [ ↗](https://developers.cloudflare.com/workers-ai/platform/glossary/) | 1,048,576 tokens | | Function calling [↗](https://developers.cloudflare.com/workers-ai/function-calling/) | Yes | | Reasoning | `low``high``max` (default) | | Zero data retention | Yes | | Unit Pricing | $1.40 per M input tokens, $4.40 per M output tokens, $0.26 per M cached input tokens | ## Playground Try out this model with Workers AI LLM Playground. It does not require any setup or authentication and is an instant way to preview and test a model directly in the browser. [Launch the LLM Playground](https://playground.ai.cloudflare.com/?model=@cf/zai-org/glm-5.3) ## Usage ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const messages = [ { role: "system", content: "You are a friendly assistant" }, { role: "user", content: "What is the origin of the phrase Hello, World", }, ]; const stream = await env.AI.run("@cf/zai-org/glm-5.3", { messages, stream: true, }); return new Response(stream, { headers: { "content-type": "text/event-stream" }, }); }, } satisfies ExportedHandler; ``` ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const messages = [ { role: "system", content: "You are a friendly assistant" }, { role: "user", content: "What is the origin of the phrase Hello, World", }, ]; const response = await env.AI.run("@cf/zai-org/glm-5.3", { messages }); return Response.json(response); }, } satisfies ExportedHandler; ``` ```py import os import requests ACCOUNT_ID = "your-account-id" AUTH_TOKEN = os.environ.get("CLOUDFLARE_AUTH_TOKEN") prompt = "Tell me all about PEP-8" response = requests.post( f"https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai/run/@cf/zai-org/glm-5.3", headers={"Authorization": f"Bearer {AUTH_TOKEN}"}, json={ "messages": [ {"role": "system", "content": "You are a friendly assistant"}, {"role": "user", "content": prompt} ] } ) result = response.json() print(result) ``` ```sh curl https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/run/@cf/zai-org/glm-5.3 \ -X POST \ -H "Authorization: Bearer $CLOUDFLARE_AUTH_TOKEN" \ -d '{ "messages": [{ "role": "system", "content": "You are a friendly assistant" }, { "role": "user", "content": "Why is pizza so good" }]}' ``` OpenAI compatible endpoints Workers AI also supports OpenAI compatible API endpoints for `/v1/chat/completions` and `/v1/embeddings`. For more details, refer to [Configurations](https://developers.cloudflare.com/workers-ai/configuration/open-ai-compatibility/) . ## Parameters
Synchronous — Send a request and receive a complete response Input format Prompt Simple text input for single-turn interactionsMessages Structured conversation format with roles (user, assistant, system) prompt stringrequiredminLength: 1The input text prompt for the model to generate a response. model stringID of the model to use (e.g. '@cf/zai-org/glm-4.7-flash, etc'). ▶audio{} objectParameters for audio output. Required when modalities includes 'audio'. frequency\_penalty number | nullPenalizes new tokens based on their existing frequency in the text so far. logit\_bias object | nullModify the likelihood of specified tokens appearing in the completion. Maps token IDs to bias values from -100 to 100. logprobs boolean | nullWhether to return log probabilities of the output tokens. top\_logprobs integer | nullHow many top log probabilities to return at each token position (0-20). Requires logprobs=true. max\_tokens integer | nullDeprecated in favor of max_completion_tokens. The maximum number of tokens to generate. max\_completion\_tokens integer | nullAn upper bound for the number of tokens that can be generated for a completion. metadata object | nullSet of 16 key-value pairs that can be attached to the object. modalities array | nullOutput types requested from the model (e.g. ['text'] or ['text', 'audio']). n integer | nullHow many chat completion choices to generate for each input message. parallel\_tool\_calls booleandefault: trueWhether to enable parallel function calling during tool use. ▶prediction{} object presence\_penalty number | nullPenalizes new tokens based on whether they appear in the text so far. reasoning\_effort string | nullenum: max, high, lowReasoning effort. Supported levels: max, high, low. Reasoning cannot be disabled. Compatibility aliases: none maps to max; minimal maps to max; medium maps to max; xhigh maps to max. ▶chat\_template\_kwargs{} object ▶response\_format one ofSpecifies the format the model must output. seed integer | nullIf specified, the system will make a best effort to sample deterministically. ▶stop one of store boolean | nullWhether to store the output for model distillation / evals. stream boolean | nullIf true, partial message deltas will be sent as server-sent events. ▶stream\_options{} object temperature number | nullSampling temperature between 0 and 2. ▶tool\_choice one ofControls which (if any) tool is called by the model. 'none' = no tools, 'auto' = model decides, 'required' = must call a tool. ▶tools\[] arrayA list of tools the model may call. top\_p number | nullNucleus sampling: considers the results of the tokens with top_p probability mass. user stringA unique identifier representing your end-user, for abuse monitoring. ▶web\_search\_options{} objectOptions for the web search tool (when using built-in web search). ▶function\_call one of ▶functions\[] arrayminItems: 1maxItems: 128 id stringA unique identifier for the chat completion. object string created integerUnix timestamp (seconds) of when the completion was created. model stringThe model used for the chat completion. ▶choices\[] arrayminItems: 1 ▶usage{} object system\_fingerprint string | null
Streaming — Send a request with `stream: true` and receive server-sent events Input format Prompt Simple text input for single-turn interactionsMessages Structured conversation format with roles (user, assistant, system) prompt stringrequiredminLength: 1The input text prompt for the model to generate a response. model stringID of the model to use (e.g. '@cf/zai-org/glm-4.7-flash, etc'). ▶audio{} objectParameters for audio output. Required when modalities includes 'audio'. frequency\_penalty number | nullPenalizes new tokens based on their existing frequency in the text so far. logit\_bias object | nullModify the likelihood of specified tokens appearing in the completion. Maps token IDs to bias values from -100 to 100. logprobs boolean | nullWhether to return log probabilities of the output tokens. top\_logprobs integer | nullHow many top log probabilities to return at each token position (0-20). Requires logprobs=true. max\_tokens integer | nullDeprecated in favor of max_completion_tokens. The maximum number of tokens to generate. max\_completion\_tokens integer | nullAn upper bound for the number of tokens that can be generated for a completion. metadata object | nullSet of 16 key-value pairs that can be attached to the object. modalities array | nullOutput types requested from the model (e.g. ['text'] or ['text', 'audio']). n integer | nullHow many chat completion choices to generate for each input message. parallel\_tool\_calls booleandefault: trueWhether to enable parallel function calling during tool use. ▶prediction{} object presence\_penalty number | nullPenalizes new tokens based on whether they appear in the text so far. reasoning\_effort string | nullenum: max, high, lowReasoning effort. Supported levels: max, high, low. Reasoning cannot be disabled. Compatibility aliases: none maps to max; minimal maps to max; medium maps to max; xhigh maps to max. ▶chat\_template\_kwargs{} object ▶response\_format one ofSpecifies the format the model must output. seed integer | nullIf specified, the system will make a best effort to sample deterministically. ▶stop one of store boolean | nullWhether to store the output for model distillation / evals. stream boolean | nullIf true, partial message deltas will be sent as server-sent events. ▶stream\_options{} object temperature number | nullSampling temperature between 0 and 2. ▶tool\_choice one ofControls which (if any) tool is called by the model. 'none' = no tools, 'auto' = model decides, 'required' = must call a tool. ▶tools\[] arrayA list of tools the model may call. top\_p number | nullNucleus sampling: considers the results of the tokens with top_p probability mass. user stringA unique identifier representing your end-user, for abuse monitoring. ▶web\_search\_options{} objectOptions for the web search tool (when using built-in web search). ▶function\_call one of ▶functions\[] arrayminItems: 1maxItems: 128 type string contentType text/event-stream format binary
Batch — Send multiple requests in a single API call ▶requests\[] array id stringA unique identifier for the chat completion. object string created integerUnix timestamp (seconds) of when the completion was created. model stringThe model used for the chat completion. ▶choices\[] arrayminItems: 1 ▶usage{} object system\_fingerprint string | null
## API Schemas (Raw) SynchronousInput [Open](https://developers.cloudflare.com/workers-ai/models/glm-5.3/sync-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/glm-5.3/sync-input.json) SynchronousOutput [Open](https://developers.cloudflare.com/workers-ai/models/glm-5.3/sync-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/glm-5.3/sync-output.json) StreamingInput [Open](https://developers.cloudflare.com/workers-ai/models/glm-5.3/streaming-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/glm-5.3/streaming-input.json) StreamingOutput [Open](https://developers.cloudflare.com/workers-ai/models/glm-5.3/streaming-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/glm-5.3/streaming-output.json) BatchInput [Open](https://developers.cloudflare.com/workers-ai/models/glm-5.3/batch-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/glm-5.3/batch-input.json) BatchOutput [Open](https://developers.cloudflare.com/workers-ai/models/glm-5.3/batch-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/glm-5.3/batch-output.json) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/models/glm-5.3/#page","headline":"glm-5.3","description":"GLM-5.3 is Z.ai's flagship agentic coding model, pairing a 1M-token context window with reasoning, function calling, and structured outputs to power multi-step, tool-driven development workflows.","url":"https://developers.cloudflare.com/workers-ai/models/glm-5.3/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: The first natively multimodal model in the GLM-5 series. With 320B total parameters and just 18B active parameters, it outperforms GLM-5.2 across benchmarks and real-world workloads at one-tenth the price, while approaching Claude Opus 4.8 on coding and agentic benchmarks. title: glm-5.3-flash image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/llms.txt > Use this file to discover all available pages before exploring further. ![Zhipu AI logo](https://developers.cloudflare.com/_astro/zai-org.Dj2vcayE.svg) # glm-5.3-flash Text Generation • Zhipu AI Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/models/glm-5.3-flash/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Model ID `@cf/zai-org/glm-5.3-flash` Copy - Cloudflare-hosted - Function calling - Reasoning - Vision - Planned deprecation - Zero data retention The first natively multimodal model in the GLM-5 series. With 320B total parameters and just 18B active parameters, it outperforms GLM-5.2 across benchmarks and real-world workloads at one-tenth the price, while approaching Claude Opus 4.8 on coding and agentic benchmarks. Paid access required This model is not available through standard Workers Free billing. To use it, upgrade to the [Workers Paid plan](https://developers.cloudflare.com/workers/platform/pricing/#workers) or use prepaid [AI Gateway credits](https://developers.cloudflare.com/ai-gateway/features/unified-billing/). | Model Info | | | --- | --- | | Planned Deprecation | 11/20/2026 | | Context Window [ ↗](https://developers.cloudflare.com/workers-ai/platform/glossary/) | 1,048,576 tokens | | Terms and License | [link ↗](https://huggingface.co/zai-org/GLM-5.3-Flash/blob/main/LICENSE) | | Function calling [↗](https://developers.cloudflare.com/workers-ai/function-calling/) | Yes | | Reasoning | `low``high``max` (default) | | Vision | Yes | | Zero data retention | Yes | | Unit Pricing | $0.15 per M input tokens, $0.50 per M output tokens, $0.03 per M cached input tokens | ## Playground Try out this model with Workers AI LLM Playground. It does not require any setup or authentication and is an instant way to preview and test a model directly in the browser. [Launch the LLM Playground](https://playground.ai.cloudflare.com/?model=@cf/zai-org/glm-5.3-flash) ## Usage ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const messages = [ { role: "system", content: "You are a friendly assistant" }, { role: "user", content: "What is the origin of the phrase Hello, World", }, ]; const stream = await env.AI.run("@cf/zai-org/glm-5.3-flash", { messages, stream: true, }); return new Response(stream, { headers: { "content-type": "text/event-stream" }, }); }, } satisfies ExportedHandler; ``` ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const messages = [ { role: "system", content: "You are a friendly assistant" }, { role: "user", content: "What is the origin of the phrase Hello, World", }, ]; const response = await env.AI.run("@cf/zai-org/glm-5.3-flash", { messages }); return Response.json(response); }, } satisfies ExportedHandler; ``` ```py import os import requests ACCOUNT_ID = "your-account-id" AUTH_TOKEN = os.environ.get("CLOUDFLARE_AUTH_TOKEN") prompt = "Tell me all about PEP-8" response = requests.post( f"https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai/run/@cf/zai-org/glm-5.3-flash", headers={"Authorization": f"Bearer {AUTH_TOKEN}"}, json={ "messages": [ {"role": "system", "content": "You are a friendly assistant"}, {"role": "user", "content": prompt} ] } ) result = response.json() print(result) ``` ```sh curl https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/run/@cf/zai-org/glm-5.3-flash \ -X POST \ -H "Authorization: Bearer $CLOUDFLARE_AUTH_TOKEN" \ -d '{ "messages": [{ "role": "system", "content": "You are a friendly assistant" }, { "role": "user", "content": "Why is pizza so good" }]}' ``` OpenAI compatible endpoints Workers AI also supports OpenAI compatible API endpoints for `/v1/chat/completions` and `/v1/embeddings`. For more details, refer to [Configurations](https://developers.cloudflare.com/workers-ai/configuration/open-ai-compatibility/) . ## Parameters
Synchronous — Send a request and receive a complete response Input format Prompt Simple text input for single-turn interactionsMessages Structured conversation format with roles (user, assistant, system) prompt stringrequiredminLength: 1The input text prompt for the model to generate a response. model stringID of the model to use (e.g. '@cf/zai-org/glm-4.7-flash, etc'). ▶audio{} objectParameters for audio output. Required when modalities includes 'audio'. frequency\_penalty number | nullPenalizes new tokens based on their existing frequency in the text so far. logit\_bias object | nullModify the likelihood of specified tokens appearing in the completion. Maps token IDs to bias values from -100 to 100. logprobs boolean | nullWhether to return log probabilities of the output tokens. top\_logprobs integer | nullHow many top log probabilities to return at each token position (0-20). Requires logprobs=true. max\_tokens integer | nullDeprecated in favor of max_completion_tokens. The maximum number of tokens to generate. max\_completion\_tokens integer | nullAn upper bound for the number of tokens that can be generated for a completion. metadata object | nullSet of 16 key-value pairs that can be attached to the object. modalities array | nullOutput types requested from the model (e.g. ['text'] or ['text', 'audio']). n integer | nullHow many chat completion choices to generate for each input message. parallel\_tool\_calls booleandefault: trueWhether to enable parallel function calling during tool use. ▶prediction{} object presence\_penalty number | nullPenalizes new tokens based on whether they appear in the text so far. reasoning\_effort string | nullenum: max, high, lowReasoning effort. Supported levels: max, high, low. Reasoning cannot be disabled. Compatibility aliases: none maps to max; minimal maps to max; medium maps to max; xhigh maps to max. ▶chat\_template\_kwargs{} object ▶response\_format one ofSpecifies the format the model must output. seed integer | nullIf specified, the system will make a best effort to sample deterministically. ▶stop one of store boolean | nullWhether to store the output for model distillation / evals. stream boolean | nullIf true, partial message deltas will be sent as server-sent events. ▶stream\_options{} object temperature number | nullSampling temperature between 0 and 2. ▶tool\_choice one ofControls which (if any) tool is called by the model. 'none' = no tools, 'auto' = model decides, 'required' = must call a tool. ▶tools\[] arrayA list of tools the model may call. top\_p number | nullNucleus sampling: considers the results of the tokens with top_p probability mass. user stringA unique identifier representing your end-user, for abuse monitoring. ▶web\_search\_options{} objectOptions for the web search tool (when using built-in web search). ▶function\_call one of ▶functions\[] arrayminItems: 1maxItems: 128 id stringA unique identifier for the chat completion. object string created integerUnix timestamp (seconds) of when the completion was created. model stringThe model used for the chat completion. ▶choices\[] arrayminItems: 1 ▶usage{} object system\_fingerprint string | null
Streaming — Send a request with `stream: true` and receive server-sent events Input format Prompt Simple text input for single-turn interactionsMessages Structured conversation format with roles (user, assistant, system) prompt stringrequiredminLength: 1The input text prompt for the model to generate a response. model stringID of the model to use (e.g. '@cf/zai-org/glm-4.7-flash, etc'). ▶audio{} objectParameters for audio output. Required when modalities includes 'audio'. frequency\_penalty number | nullPenalizes new tokens based on their existing frequency in the text so far. logit\_bias object | nullModify the likelihood of specified tokens appearing in the completion. Maps token IDs to bias values from -100 to 100. logprobs boolean | nullWhether to return log probabilities of the output tokens. top\_logprobs integer | nullHow many top log probabilities to return at each token position (0-20). Requires logprobs=true. max\_tokens integer | nullDeprecated in favor of max_completion_tokens. The maximum number of tokens to generate. max\_completion\_tokens integer | nullAn upper bound for the number of tokens that can be generated for a completion. metadata object | nullSet of 16 key-value pairs that can be attached to the object. modalities array | nullOutput types requested from the model (e.g. ['text'] or ['text', 'audio']). n integer | nullHow many chat completion choices to generate for each input message. parallel\_tool\_calls booleandefault: trueWhether to enable parallel function calling during tool use. ▶prediction{} object presence\_penalty number | nullPenalizes new tokens based on whether they appear in the text so far. reasoning\_effort string | nullenum: max, high, lowReasoning effort. Supported levels: max, high, low. Reasoning cannot be disabled. Compatibility aliases: none maps to max; minimal maps to max; medium maps to max; xhigh maps to max. ▶chat\_template\_kwargs{} object ▶response\_format one ofSpecifies the format the model must output. seed integer | nullIf specified, the system will make a best effort to sample deterministically. ▶stop one of store boolean | nullWhether to store the output for model distillation / evals. stream boolean | nullIf true, partial message deltas will be sent as server-sent events. ▶stream\_options{} object temperature number | nullSampling temperature between 0 and 2. ▶tool\_choice one ofControls which (if any) tool is called by the model. 'none' = no tools, 'auto' = model decides, 'required' = must call a tool. ▶tools\[] arrayA list of tools the model may call. top\_p number | nullNucleus sampling: considers the results of the tokens with top_p probability mass. user stringA unique identifier representing your end-user, for abuse monitoring. ▶web\_search\_options{} objectOptions for the web search tool (when using built-in web search). ▶function\_call one of ▶functions\[] arrayminItems: 1maxItems: 128 type string contentType text/event-stream format binary
Batch — Send multiple requests in a single API call ▶requests\[] array id stringA unique identifier for the chat completion. object string created integerUnix timestamp (seconds) of when the completion was created. model stringThe model used for the chat completion. ▶choices\[] arrayminItems: 1 ▶usage{} object system\_fingerprint string | null
## API Schemas (Raw) SynchronousInput [Open](https://developers.cloudflare.com/workers-ai/models/glm-5.3-flash/sync-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/glm-5.3-flash/sync-input.json) SynchronousOutput [Open](https://developers.cloudflare.com/workers-ai/models/glm-5.3-flash/sync-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/glm-5.3-flash/sync-output.json) StreamingInput [Open](https://developers.cloudflare.com/workers-ai/models/glm-5.3-flash/streaming-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/glm-5.3-flash/streaming-input.json) StreamingOutput [Open](https://developers.cloudflare.com/workers-ai/models/glm-5.3-flash/streaming-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/glm-5.3-flash/streaming-output.json) BatchInput [Open](https://developers.cloudflare.com/workers-ai/models/glm-5.3-flash/batch-input.json) [Download](https://developers.cloudflare.com/workers-ai/models/glm-5.3-flash/batch-input.json) BatchOutput [Open](https://developers.cloudflare.com/workers-ai/models/glm-5.3-flash/batch-output.json) [Download](https://developers.cloudflare.com/workers-ai/models/glm-5.3-flash/batch-output.json) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/models/glm-5.3-flash/#page","headline":"glm-5.3-flash","description":"The first natively multimodal model in the GLM-5 series. With 320B total parameters and just 18B active parameters, it outperforms GLM-5.2 across benchmarks and real-world workloads at one-tenth the price, while approaching Claude Opus 4.8 on coding and agentic benchmarks.","url":"https://developers.cloudflare.com/workers-ai/models/glm-5.3-flash/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Use Workers AI models with the AI SDK through createAI from agents/models/ai-sdk. title: Vercel AI SDK image: https://developers.cloudflare.com/workers-ai/configuration/ai-sdk/og.png?v=6e60b4e5c89fbf07 --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/workers-ai/llms.txt > Use this file to discover all available pages before exploring further. # Vercel AI SDK Last updated Oct 21, 2026|Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/configuration/ai-sdk/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) To use Workers AI with the [AI SDK ↗︎](https://ai-sdk.dev/), use `createAI` from `agents/models/ai-sdk`. This page covers Workers AI models. For AI Gateway catalog models, fallback, gateway options, and the full option reference, refer to [AI SDK](https://developers.cloudflare.com/agents/models/ai-sdk/). The `workers-ai-provider` package is deprecated. Use `createAI` instead. ## Setup Install the Agents SDK and the AI SDK. `createAI` needs `ai@^7` and `@ai-sdk/provider@^4`: npmyarnpnpmbun ``` npm i agents ai @ai-sdk/provider ``` ``` yarn add agents ai @ai-sdk/provider ``` ``` pnpm add agents ai @ai-sdk/provider ``` ``` bun add agents ai @ai-sdk/provider ``` Add the `AI` binding to your Wrangler configuration: ```jsonc { "ai": { "binding": "AI", }, } ``` ```toml [ai] binding = "AI" ``` ## Choose a model Pass the id of any [Workers AI model](https://developers.cloudflare.com/workers-ai/models/). Model ids start with `@cf/` and autocomplete from `@cloudflare/workers-types`: ```ts import { createAI } from "agents/models/ai-sdk"; const ai = createAI({ binding: env.AI }); const model = ai("@cf/zai-org/glm-4.7-flash"); ``` ## Generate text ```js import { createAI } from "agents/models/ai-sdk"; import { generateText } from "ai"; export default { async fetch(request, env) { const ai = createAI({ binding: env.AI }); const { text } = await generateText({ model: ai("@cf/zai-org/glm-4.7-flash"), prompt: "Write a 50-word essay about hello world.", }); return new Response(text); }, }; ``` ```ts import { createAI } from "agents/models/ai-sdk"; import { generateText } from "ai"; export default { async fetch(request: Request, env: Env) { const ai = createAI({ binding: env.AI }); const { text } = await generateText({ model: ai("@cf/zai-org/glm-4.7-flash"), prompt: "Write a 50-word essay about hello world.", }); return new Response(text); }, }; ``` ## Stream text For longer responses, stream the output as the model generates it: ```js import { createAI } from "agents/models/ai-sdk"; import { streamText } from "ai"; export default { async fetch(request, env) { const ai = createAI({ binding: env.AI }); const result = streamText({ model: ai("@cf/zai-org/glm-4.7-flash"), prompt: "Write a 50-word essay about hello world.", }); return result.toTextStreamResponse(); }, }; ``` ```ts import { createAI } from "agents/models/ai-sdk"; import { streamText } from "ai"; export default { async fetch(request: Request, env: Env) { const ai = createAI({ binding: env.AI }); const result = streamText({ model: ai("@cf/zai-org/glm-4.7-flash"), prompt: "Write a 50-word essay about hello world.", }); return result.toTextStreamResponse(); }, }; ``` ## Generate structured output Pass a Zod schema to get a typed JSON response: ```js import { createAI } from "agents/models/ai-sdk"; import { generateText, Output } from "ai"; import { z } from "zod"; export default { async fetch(request, env) { const ai = createAI({ binding: env.AI }); const { output } = await generateText({ model: ai("@cf/zai-org/glm-4.7-flash"), prompt: "Generate a lasagna recipe.", output: Output.object({ schema: z.object({ recipe: z.object({ ingredients: z.array(z.string()), description: z.string(), }), }), }), }); return Response.json(output); }, }; ``` ```ts import { createAI } from "agents/models/ai-sdk"; import { generateText, Output } from "ai"; import { z } from "zod"; export default { async fetch(request: Request, env: Env) { const ai = createAI({ binding: env.AI }); const { output } = await generateText({ model: ai("@cf/zai-org/glm-4.7-flash"), prompt: "Generate a lasagna recipe.", output: Output.object({ schema: z.object({ recipe: z.object({ ingredients: z.array(z.string()), description: z.string(), }), }), }), }); return Response.json(output); }, }; ``` ## Use other model types Embeddings, images, speech, transcription, and reranking come from the same provider: ```ts import { embedMany, generateImage, generateSpeech, rerank, transcribe, } from "ai"; const { embeddings } = await embedMany({ model: ai.embedding("@cf/baai/bge-base-en-v1.5"), values: ["hello", "world"], }); const { image } = await generateImage({ model: ai.image("@cf/black-forest-labs/flux-1-schnell"), prompt: "A cyberpunk lizard", }); const { audio } = await generateSpeech({ model: ai.speech("@cf/deepgram/aura-1"), text: "Hello from Cloudflare.", }); const { text } = await transcribe({ model: ai.transcription("@cf/openai/whisper-large-v3-turbo"), audio: audio.uint8Array, }); const { rerankedDocuments } = await rerank({ model: ai.reranking("@cf/baai/bge-reranker-base"), documents: ["a cyberpunk lizard", "a rainy Tuesday"], query: "Which one is cooler?", }); ``` For model-specific settings, refer to [Use other modalities](https://developers.cloudflare.com/agents/models/ai-sdk/#use-other-modalities). ## Use AI Gateway Requests go through [AI Gateway](https://developers.cloudflare.com/ai-gateway/). The gateway defaults to `default`. To use another gateway, set its `id`: ```ts const ai = createAI({ binding: env.AI, id: "my-gateway" }); ``` The same provider also calls models from the [AI Gateway catalog](https://developers.cloudflare.com/ai/models/), such as `anthropic/claude-opus-5.5`. Refer to [AI SDK](https://developers.cloudflare.com/agents/models/ai-sdk/) for catalog models, caching, fallback, and the other gateway options. ## Related resources - [AI SDK provider reference](https://developers.cloudflare.com/agents/models/ai-sdk/) - [Workers AI models](https://developers.cloudflare.com/workers-ai/models/) - [AI SDK documentation ↗︎](https://ai-sdk.dev/docs/introduction) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/configuration/ai-sdk/#page","headline":"Vercel AI SDK","description":"Use Workers AI models with the AI SDK through createAI from agents/models/ai-sdk.","url":"https://developers.cloudflare.com/workers-ai/configuration/ai-sdk/","inLanguage":"en","image":"https://developers.cloudflare.com/workers-ai/configuration/ai-sdk/og.png?v=6e60b4e5c89fbf07","dateModified":"2026-10-21","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Create an AI binding to connect your Cloudflare Worker to Workers AI. title: Workers Bindings image: https://developers.cloudflare.com/workers-ai/configuration/bindings/og.png?v=afa2e8cca2301ad1 --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/workers-ai/llms.txt > Use this file to discover all available pages before exploring further. # Workers Bindings Last updated Apr 21, 2026|Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/configuration/bindings/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) ## Workers [Workers](https://developers.cloudflare.com/workers/) provides a serverless execution environment that allows you to create new applications or augment existing ones. To use Workers AI with Workers, you must create a Workers AI [binding](https://developers.cloudflare.com/workers/runtime-apis/bindings/). Bindings allow your Workers to interact with resources, like Workers AI, on the Cloudflare Developer Platform. You create bindings on the Cloudflare dashboard or by updating your [Wrangler file](https://developers.cloudflare.com/workers/wrangler/configuration/). To bind Workers AI to your Worker, add the following to the end of your Wrangler file: ```jsonc { "ai": { "binding": "AI" // i.e. available in your Worker on env.AI } } ``` ```toml [ai] binding = "AI" ``` ## Pages Functions [Pages Functions](https://developers.cloudflare.com/pages/functions/) allow you to build full-stack applications with Cloudflare Pages by executing code on the Cloudflare network. Functions are Workers under the hood. To configure a Workers AI binding in your Pages Function, you must use the Cloudflare dashboard. Refer to [Workers AI bindings](https://developers.cloudflare.com/pages/functions/bindings/#workers-ai) for instructions. ## Methods ### async env.AI.run() `async env.AI.run()` runs a model. Takes a model as the first parameter, and an object as the second parameter. ```javascript const answer = await env.AI.run('@cf/meta/llama-3.1-8b-instruct', { prompt: "What is the origin of the phrase 'Hello, World'" }); ``` **Parameters** - `model` `string` required - The model to run. **Supported options** - `stream` `boolean` optional - Returns a stream of results as they are available. ```javascript const answer = await env.AI.run('@cf/meta/llama-3.1-8b-instruct', { prompt: "What is the origin of the phrase 'Hello, World'", stream: true }); return new Response(answer, { headers: { "content-type": "text/event-stream" } }); ``` Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/configuration/bindings/#page","headline":"Workers Bindings","description":"Create an AI binding to connect your Cloudflare Worker to Workers AI.","url":"https://developers.cloudflare.com/workers-ai/configuration/bindings/","inLanguage":"en","image":"https://developers.cloudflare.com/workers-ai/configuration/bindings/og.png?v=afa2e8cca2301ad1","dateModified":"2026-04-21","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Connect Workers AI models to Hugging Face's open-source Chat UI interface. title: Hugging Face Chat UI image: https://developers.cloudflare.com/workers-ai/configuration/hugging-face-chat-ui/og.png?v=e9b5c6da7ed0c500 --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/workers-ai/llms.txt > Use this file to discover all available pages before exploring further. # Hugging Face Chat UI Last updated Apr 21, 2026|Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/configuration/hugging-face-chat-ui/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Use Workers AI with [Chat UI ↗︎](https://github.com/huggingface/chat-ui?tab=readme-ov-file#text-embedding-models), an open-source chat interface offered by Hugging Face. ## Prerequisites You will need the following: - A [Cloudflare account ↗︎](https://dash.cloudflare.com) - Your [Account ID](https://developers.cloudflare.com/fundamentals/account/find-account-and-zone-ids/) - An [API token](https://developers.cloudflare.com/workers-ai/get-started/rest-api/#1-get-api-token-and-account-id) for Workers AI ## Setup First, decide how to reference your Account ID and API token (either directly in your `.env.local` using the `CLOUDFLARE_ACCOUNT_ID` and `CLOUDFLARE_API_TOKEN` variables or in the endpoint configuration). Then, follow the rest of the setup instructions in the [Chat UI GitHub repository ↗︎](https://github.com/huggingface/chat-ui?tab=readme-ov-file#text-embedding-models). When setting up your models, specify the `cloudflare` endpoint. ```json { "name" : "nousresearch/hermes-2-pro-mistral-7b", "tokenizer": "nousresearch/hermes-2-pro-mistral-7b", "parameters": { "stop": ["<|im_end|>"] }, "endpoints" : [ { "type": "cloudflare", // optionally specify these if not included in .env.local "accountId": "your-account-id", "apiToken": "your-api-token" // } ] } ``` ## Supported models This template works with any [text generation models](https://developers.cloudflare.com/workers-ai/models/) that begin with the `@hf` parameter. Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"WebPage","@id":"https://developers.cloudflare.com/workers-ai/configuration/hugging-face-chat-ui/#page","headline":"Hugging Face Chat UI","description":"Connect Workers AI models to Hugging Face's open-source Chat UI interface.","url":"https://developers.cloudflare.com/workers-ai/configuration/hugging-face-chat-ui/","inLanguage":"en","image":"https://developers.cloudflare.com/workers-ai/configuration/hugging-face-chat-ui/og.png?v=e9b5c6da7ed0c500","dateModified":"2026-04-21","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Use the OpenAI SDK with Workers AI Chat Completions, embeddings, and supported GPT-OSS Responses requests. title: OpenAI compatible API endpoints image: https://developers.cloudflare.com/workers-ai/configuration/open-ai-compatibility/og.png?v=43d7808c7c4959b5 --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/workers-ai/llms.txt > Use this file to discover all available pages before exploring further. # OpenAI compatible API endpoints Last updated Sep 18, 2026|Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/configuration/open-ai-compatibility/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Workers AI provides OpenAI-compatible endpoints for [text generation](https://developers.cloudflare.com/workers-ai/models/) through Chat Completions (`/v1/chat/completions`) and for [text embedding models](https://developers.cloudflare.com/workers-ai/models/) (`/v1/embeddings`). The Responses API (`/v1/responses`) is available only for GPT-OSS models. Easily call Workers AI by swapping the `baseURL` in the standard OpenAI SDK. ## Usage ### Chat Completions and embeddings Most Workers AI text generation models support the OpenAI Chat Completions API. Embedding models support the OpenAI Embeddings API. Use the [OpenAI JavaScript SDK ↗︎](https://github.com/openai/openai-node) by setting the Workers AI base URL, API token, and model name. *OpenAI SDK examplejs* ```js import OpenAI from "openai"; const openai = new OpenAI({ apiKey: env.CLOUDFLARE_API_KEY, baseURL: `https://api.cloudflare.com/client/v4/accounts/${env.CLOUDFLARE_ACCOUNT_ID}/ai/v1`, }); const chatCompletion = await openai.chat.completions.create({ messages: [{ role: "user", content: "Make some robot noises" }], model: "@cf/meta/llama-3.1-8b-instruct", }); const embeddings = await openai.embeddings.create({ model: "@cf/baai/bge-large-en-v1.5", input: "I love matcha", }); ``` *cURL examplebash* ```bash curl --request POST \ --url https://api.cloudflare.com/client/v4/accounts/{account_id}/ai/v1/chat/completions \ --header "Authorization: Bearer {api_token}" \ --header "Content-Type: application/json" \ --data ' { "model": "@cf/meta/llama-3.1-8b-instruct", "messages": [ { "role": "user", "content": "how to build a wooden spoon in 3 short steps? give as short as answer as possible" } ] } ' ``` ### Fail fast when capacity is unavailable For synchronous Chat Completions, set `options.rejectIfBusy` in the top-level request body. This makes the request fail instead of waiting in a capacity queue. OpenAI clients that preserve custom fields can send this option. Clients that remove unknown fields do not apply it, so requests proceed normally. Refer to [Reject busy requests](https://developers.cloudflare.com/workers-ai/features/reject-if-busy/) for examples and error behavior. ### Responses API for GPT-OSS The Responses API is supported only by the [`@cf/openai/gpt-oss-120b`](https://developers.cloudflare.com/workers-ai/models/gpt-oss-120b/) and [`@cf/openai/gpt-oss-20b`](https://developers.cloudflare.com/workers-ai/models/gpt-oss-20b/) models. Responses requests must be non-streaming, only `stream: false` is supported. *GPT-OSS Responses API examplejs* ```js import OpenAI from "openai"; const openai = new OpenAI({ apiKey: env.CLOUDFLARE_API_KEY, baseURL: `https://api.cloudflare.com/client/v4/accounts/${env.CLOUDFLARE_ACCOUNT_ID}/ai/v1`, }); const response = await openai.responses.create({ model: "@cf/openai/gpt-oss-120b", input: "Talk to me about open source", }); ``` ### AI Gateway These endpoints are also compatible with [AI Gateway](https://developers.cloudflare.com/ai-gateway/usage/providers/workersai/#openai-compatible-endpoints). Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/configuration/open-ai-compatibility/#page","headline":"OpenAI compatible API endpoints","description":"Use the OpenAI SDK with Workers AI Chat Completions, embeddings, and supported GPT-OSS Responses requests.","url":"https://developers.cloudflare.com/workers-ai/configuration/open-ai-compatibility/","inLanguage":"en","image":"https://developers.cloudflare.com/workers-ai/configuration/open-ai-compatibility/og.png?v=43d7808c7c4959b5","dateModified":"2026-09-18","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Use Workers AI models with Pi AI through createAI from agents/models/pi-ai. title: Pi AI image: https://developers.cloudflare.com/workers-ai/configuration/pi-ai/og.png?v=927ed23e0b5e5a1a --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/workers-ai/llms.txt > Use this file to discover all available pages before exploring further. # Pi AI Last updated Nov 3, 2026|Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/configuration/pi-ai/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) To use Workers AI with [Pi AI ↗︎](https://github.com/earendil-works/pi/tree/main/packages/ai), use `createAI` from `agents/models/pi-ai`. It returns Pi AI `Model` objects, so Workers AI models work with Pi AI's `stream` and `complete`, with [Pi Durable ↗︎](https://github.com/earendil-works/pi/tree/main/packages/durable), and with the [Pi harness](https://developers.cloudflare.com/agents/harnesses/pi/). This page covers Workers AI models. For AI Gateway catalog models, fallback, gateway options, and the full reference, refer to [Pi AI](https://developers.cloudflare.com/agents/models/pi-ai/). ## Setup Install the Agents SDK and Pi AI. `createAI` needs `@earendil-works/pi-ai@~1.5.0`: npmyarnpnpmbun ``` npm i agents @earendil-works/pi-ai ``` ``` yarn add agents @earendil-works/pi-ai ``` ``` pnpm add agents @earendil-works/pi-ai ``` ``` bun add agents @earendil-works/pi-ai ``` Add the `AI` binding to your Wrangler configuration: ```jsonc { "ai": { "binding": "AI", }, } ``` ```toml [ai] binding = "AI" ``` ## Generate text Name a Workers AI model by its `@cf/` id. The provider carries Pi AI's entry points, so nothing is registered globally: ```js import { createAI } from "agents/models/pi-ai"; export default { async fetch(request, env) { const ai = createAI({ binding: env.AI }); const reply = await ai.complete(ai("@cf/zai-org/glm-4.7-flash"), { messages: [ { role: "user", content: "What is Cloudflare?", timestamp: Date.now() }, ], }); // `content` holds text, thinking, and tool call parts. const text = reply.content .filter((part) => part.type === "text") .map((part) => part.text) .join(""); return Response.json({ text }); }, }; ``` ```ts import { createAI } from "agents/models/pi-ai"; export default { async fetch(request: Request, env: Env) { const ai = createAI({ binding: env.AI }); const reply = await ai.complete(ai("@cf/zai-org/glm-4.7-flash"), { messages: [ { role: "user", content: "What is Cloudflare?", timestamp: Date.now() }, ], }); // `content` holds text, thinking, and tool call parts. const text = reply.content .filter((part) => part.type === "text") .map((part) => part.text) .join(""); return Response.json({ text }); }, }; ``` ## Stream text `ai.stream()` returns Pi AI's event stream. `ai.streamSimple()` and `ai.completeSimple()` take Pi AI's simple options, such as a `reasoning` level and `maxTokens`: ```ts const model = ai("@cf/zai-org/glm-4.7-flash"); const context = { messages: [{ role: "user", content: "Hi", timestamp: Date.now() }], }; for await (const event of ai.stream(model, context)) { // text_delta, thinking_delta, toolcall_*, done, error } ``` A compatibility layer turns each Workers AI model's response into the shape Pi AI's parsers read, so tool calls and reasoning work the same across model families. ## Set model details A Workers AI id that Pi AI does not list resolves with a 128,000 token context window, 8,192 output tokens, and zero cost. Pass the real values when you know them: ```ts const model = ai("@cf/zai-org/glm-4.7-flash", { contextWindow: 131_072, maxTokens: 16_384, }); ``` ## Generate images ```ts const images = ai.images("@cf/black-forest-labs/flux-1-schnell", { steps: 8 }); const result = await ai.generateImages(images, { input: [{ type: "text", text: "A red bicycle" }], }); ``` ## Use AI Gateway Every request goes through AI Gateway. Set the gateway with `id`, which defaults to `default`: ```ts const ai = createAI({ binding: env.AI, id: "my-gateway" }); ``` To call from outside a Worker, pass `accountId` and `apiToken` instead of `binding`. For gateway options, catalog models, and fallback, refer to [Pi AI](https://developers.cloudflare.com/agents/models/pi-ai/). ## Related resources - [Pi AI provider reference](https://developers.cloudflare.com/agents/models/pi-ai/) - [Pi harness](https://developers.cloudflare.com/agents/harnesses/pi/) - [Workers AI models](https://developers.cloudflare.com/workers-ai/models/) - [Vercel AI SDK](https://developers.cloudflare.com/workers-ai/configuration/ai-sdk/) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/configuration/pi-ai/#page","headline":"Pi AI","description":"Use Workers AI models with Pi AI through createAI from agents/models/pi-ai.","url":"https://developers.cloudflare.com/workers-ai/configuration/pi-ai/","inLanguage":"en","image":"https://developers.cloudflare.com/workers-ai/configuration/pi-ai/og.png?v=927ed23e0b5e5a1a","dateModified":"2026-11-03","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Queue large inference workloads for asynchronous processing with the Workers AI Batch API. title: Asynchronous Batch API image: https://developers.cloudflare.com/workers-ai/features/batch-api/og.png?v=77cfb4ef80ef74d9 --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/workers-ai/llms.txt > Use this file to discover all available pages before exploring further. # Asynchronous Batch API Last updated Apr 21, 2026|Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/features/batch-api/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Asynchronous batch processing lets you send a collection (batch) of inference requests in a single call. Instead of expecting immediate responses for every request, the system queues them for processing and returns the results later. Batch processing is useful for large workloads such as summarization or embeddings when there is no human interaction. Using the batch API will guarantee that your requests are fulfilled eventually, rather than erroring out if Cloudflare does not have enough capacity at a given time. When you send a batch request, the API immediately acknowledges receipt with a status like `queued` and provides a unique `request_id`. This ID is later used to poll for the final responses once the processing is complete. You can use the Batch API by either creating and deploying a Cloudflare Worker that leverages the [Batch API with the AI binding](https://developers.cloudflare.com/workers-ai/features/batch-api/workers-binding/), using the [REST API](https://developers.cloudflare.com/workers-ai/features/batch-api/rest-api/) directly or by starting from a [template ↗︎](https://github.com/craigsdennis/batch-please-workers-ai). Note Ensure that the total payload is under 10 MB. ## Demo application If you want to get started quickly, click the button below: [![Deploy to Workers](https://deploy.workers.cloudflare.com/button)](https://deploy.workers.cloudflare.com/?url=https://github.com/craigsdennis/batch-please-workers-ai) This will create a repository in your GitHub account and deploy a ready-to-use Worker that demonstrates how to use Cloudflare's Asynchronous Batch API. The template includes preconfigured AI bindings, and examples for sending and retrieving batch requests with and without external references. Once deployed, you can visit the live Worker and start experimenting with the Batch API immediately. ## Supported Models Refer to our [model catalog](https://developers.cloudflare.com/workers-ai/models/?capabilities=Batch) for supported models. Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/features/batch-api/#page","headline":"Asynchronous Batch API","description":"Queue large inference workloads for asynchronous processing with the Workers AI Batch API.","url":"https://developers.cloudflare.com/workers-ai/features/batch-api/","inLanguage":"en","image":"https://developers.cloudflare.com/workers-ai/features/batch-api/og.png?v=77cfb4ef80ef74d9","dateModified":"2026-04-21","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Send and retrieve batch inference requests using the Workers AI REST API. title: REST API image: https://developers.cloudflare.com/workers-ai/features/batch-api/rest-api/og.png?v=5b1fb19ca805e26f --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/workers-ai/llms.txt > Use this file to discover all available pages before exploring further. # REST API Last updated Apr 21, 2026|Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/features/batch-api/rest-api/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) If you prefer to work directly with the REST API instead of a [Cloudflare Worker](https://developers.cloudflare.com/workers-ai/features/batch-api/workers-binding/), below are the steps on how to do it: ## 1. Sending a Batch Request Make a POST request using the following pattern. You can pass `external_reference` as a unique ID per-request that will be returned in the response. *Sending a batch requestbash* ```bash curl "https://api.cloudflare.com/client/v4/accounts/$ACCOUNT_ID/ai/run/@cf/baai/bge-m3?queueRequest=true" \ --header "Authorization: Bearer $API_TOKEN" \ --header 'Content-Type: application/json' \ --json '{ "requests": [ { "query": "This is a story about Cloudflare", "contexts": [ { "text": "This is a story about an orange cloud" }, { "text": "This is a story about a llama" }, { "text": "This is a story about a hugging emoji" } ], "external_reference": "reference-1" } ] }' ``` ```json { "result": { "status": "queued", "request_id": "768f15b7-4fd6-4498-906e-ad94ffc7f8d2", "model": "@cf/baai/bge-m3" }, "success": true, "errors": [], "messages": [] } ``` ## 2. Retrieving the Batch Response After receiving a `request_id` from your initial POST, you can poll for or retrieve the results with another POST request: *Retrieving a responsebash* ```bash curl "https://api.cloudflare.com/client/v4/accounts/$ACCOUNT_ID/ai/run/@cf/baai/bge-m3?queueRequest=true" \ --header "Authorization: Bearer $API_TOKEN" \ --header 'Content-Type: application/json' \ --json '{ "request_id": "" }' ``` ```json { "result": { "responses": [ { "id": 0, "result": { "response": [ { "id": 0, "score": 0.73974609375 }, { "id": 1, "score": 0.642578125 }, { "id": 2, "score": 0.6220703125 } ] }, "success": true, "external_reference": "reference-1" } ], "usage": { "prompt_tokens": 12, "completion_tokens": 0, "total_tokens": 12 } }, "success": true, "errors": [], "messages": [] } ``` Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/features/batch-api/rest-api/#page","headline":"REST API","description":"Send and retrieve batch inference requests using the Workers AI REST API.","url":"https://developers.cloudflare.com/workers-ai/features/batch-api/rest-api/","inLanguage":"en","image":"https://developers.cloudflare.com/workers-ai/features/batch-api/rest-api/og.png?v=5b1fb19ca805e26f","dateModified":"2026-04-21","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Send and retrieve batch inference requests using a Workers AI binding. title: Workers Binding image: https://developers.cloudflare.com/workers-ai/features/batch-api/workers-binding/og.png?v=4c5daf24a38904a5 --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/workers-ai/llms.txt > Use this file to discover all available pages before exploring further. # Workers Binding Last updated Apr 21, 2026|Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/features/batch-api/workers-binding/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) You can use Workers Bindings to interact with the Batch API. ## Send a Batch request Send your initial batch inference request by composing a JSON payload containing an array of individual inference requests and the `queueRequest: true` property (which is what controls queueing behavior). Note Ensure that the total payload is under 10 MB. *src/index.tsts* ```ts export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const embeddings = await env.AI.run( "@cf/baai/bge-m3", { requests: [ { query: "This is a story about Cloudflare", contexts: [ { text: "This is a story about an orange cloud", }, { text: "This is a story about a llama", }, { text: "This is a story about a hugging emoji", }, ], }, ], }, { queueRequest: true }, ); return Response.json(embeddings); }, } satisfies ExportedHandler; ``` ```json { "status": "queued", "model": "@cf/baai/bge-m3", "request_id": "000-000-000" } ``` You will get a response with the following values: - **`status`**: Indicates that your request is queued. - **`request_id`**: A unique identifier for the batch request. - **`model`**: The model used for the batch inference. Of these, the `request_id` is important for when you need to [poll the batch status](#poll-batch-status). ### Poll batch status Once your batch request is queued, use the `request_id` to poll for its status. During processing, the API returns a status `queued` or `running` indicating that the request is still in the queue or being processed. ```typescript export interface Env { AI: Ai; } export default { async fetch(request, env): Promise { const status = await env.AI.run("@cf/baai/bge-m3", { request_id: "000-000-000", }); return Response.json(status); }, } satisfies ExportedHandler; ``` ```json { "responses": [ { "id": 0, "result": { "response": [ { "id": 0, "score": 0.73974609375 }, { "id": 1, "score": 0.642578125 }, { "id": 2, "score": 0.6220703125 } ] }, "success": true, "external_reference": "reference-1" } ], "usage": { "prompt_tokens": 12, "completion_tokens": 0, "total_tokens": 12 } } ``` When the inference is complete, the API returns a final HTTP status code of `200` along with an array of responses. Each response object corresponds to an individual input prompt, identified by an `id` that maps to the index of the prompt in your original request. Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/features/batch-api/workers-binding/#page","headline":"Workers Binding","description":"Send and retrieve batch inference requests using a Workers AI binding.","url":"https://developers.cloudflare.com/workers-ai/features/batch-api/workers-binding/","inLanguage":"en","image":"https://developers.cloudflare.com/workers-ai/features/batch-api/workers-binding/og.png?v=4c5daf24a38904a5","dateModified":"2026-04-21","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Run fine-tuned inference on Workers AI using LoRA adapters. title: Fine-tunes image: https://developers.cloudflare.com/workers-ai/features/fine-tunes/og.png?v=4894bcc654d8841f --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/workers-ai/llms.txt > Use this file to discover all available pages before exploring further. # Fine-tunes Last updated Apr 21, 2026|Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/features/fine-tunes/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Learn how to use Workers AI to get fine-tuned inference. [Fine-tuned inference with LoRAs](https://developers.cloudflare.com/workers-ai/features/fine-tunes/loras/) Upload a LoRA adapter and run fine-tuned inference with one of our base models. Run inference with LoRAs --- ## What is fine-tuning? Fine-tuning is a general term for modifying an AI model by continuing to train it with additional data. The goal of fine-tuning is to increase the probability that a generation is similar to your dataset. Training a model from scratch is not practical for many use cases given how expensive and time consuming they can be to train. By fine-tuning an existing pre-trained model, you benefit from its capabilities while also accomplishing your desired task. [Low-Rank Adaptation ↗︎](https://arxiv.org/abs/2106.09685) (LoRA) is a specific fine-tuning method that can be applied to various model architectures, not just LLMs. It is common that the pre-trained model weights are directly modified or fused with additional fine-tune weights in traditional fine-tuning methods. LoRA, on the other hand, allows for the fine-tune weights and pre-trained model to remain separate, and for the pre-trained model to remain unchanged. The end result is that you can train models to be more accurate at specific tasks, such as generating code, having a specific personality, or generating images in a specific style. Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"WebPage","@id":"https://developers.cloudflare.com/workers-ai/features/fine-tunes/#page","headline":"Fine-tunes","description":"Run fine-tuned inference on Workers AI using LoRA adapters.","url":"https://developers.cloudflare.com/workers-ai/features/fine-tunes/","inLanguage":"en","image":"https://developers.cloudflare.com/workers-ai/features/fine-tunes/og.png?v=4894bcc654d8841f","dateModified":"2026-04-21","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Upload and use LoRA adapters to get fine-tuned inference on Workers AI. title: Using LoRA adapters image: https://developers.cloudflare.com/workers-ai/features/fine-tunes/loras/og.png?v=5ab5aeb1f6d4d456 --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/workers-ai/llms.txt > Use this file to discover all available pages before exploring further. # Using LoRA adapters Last updated Apr 21, 2026|Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/features/fine-tunes/loras/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Workers AI supports fine-tuned inference with adapters trained with [Low-Rank Adaptation ↗︎](https://blog.cloudflare.com/fine-tuned-inference-with-loras). This feature is in open beta and free during this period. ## Limitations - We only support LoRAs for a [variety of models](https://developers.cloudflare.com/workers-ai/models/?capabilities=LoRA) (must not be quantized) - Adapter must be trained with rank `r <=8` as well as larger ranks if up to 32. You can check the rank of a pre-trained LoRA adapter through the adapter's `config.json` file - LoRA adapter file must be < 300MB - LoRA adapter files must be named `adapter_config.json` and `adapter_model.safetensors` exactly - You can test up to 100 LoRA adapters per account --- ## Choosing compatible LoRA adapters ### Finding open-source LoRA adapters We have started a [Hugging Face Collection ↗︎](https://huggingface.co/collections/Cloudflare/workers-ai-compatible-loras-6608dd9f8d305a46e355746e) that lists a few LoRA adapters that are compatible with Workers AI. Generally, any LoRA adapter that fits our limitations above should work. ### Training your own LoRA adapters To train your own LoRA adapter, follow the [tutorial](https://developers.cloudflare.com/workers-ai/guides/tutorials/fine-tune-models-with-autotrain/). --- ## Uploading LoRA adapters In order to run inference with LoRAs on Workers AI, you'll need to create a new fine tune on your account and upload your adapter files. You should have a `adapter_model.safetensors` file with model weights and `adapter_config.json` with your config information. *Note that we only accept adapter files in these types.* Right now, you can't edit a fine tune's asset files after you upload it. We will support this soon, but for now you will need to create a new fine tune and upload files again if you would like to use a new LoRA. Before you upload your LoRA adapter, you'll need to edit your `adapter_config.json` file to include `model_type` as one of `mistral`, `gemma` or `llama` like below. ```json { "alpha_pattern": {}, "auto_mapping": null, ... "target_modules": [ "q_proj", "v_proj" ], "task_type": "CAUSAL_LM", "model_type": "mistral", } ``` ### Wrangler You can create a finetune and upload your LoRA adapter via wrangler with the following commands: *wrangler CLIbash* ```bash npx wrangler ai finetune create #🌀 Creating new finetune "test-lora" for model "@cf/mistral/mistral-7b-instruct-v0.2-lora"... #🌀 Uploading file "/Users/abcd/Downloads/adapter_config.json" to "test-lora"... #🌀 Uploading file "/Users/abcd/Downloads/adapter_model.safetensors" to "test-lora"... #✅ Assets uploaded, finetune "test-lora" is ready to use. npx wrangler ai finetune list ┌──────────────────────────────────────┬─────────────────┬─────────────┐ │ finetune_id │ name │ description │ ├──────────────────────────────────────┼─────────────────┼─────────────┤ │ 00000000-0000-0000-0000-000000000000 │ test-lora │ │ └──────────────────────────────────────┴─────────────────┴─────────────┘ ``` ### REST API Alternatively, you can use our REST API to create a finetune and upload your adapter files. You will need a Cloudflare API Token with `Workers AI: Edit` permissions to make calls to our REST API, which you can generate via the Cloudflare Dashboard. #### Creating a fine-tune on your account
Required API token permissions At least one of the following token permissions is required: - Workers AI Write
*Create a new Finetunebash* ```bash curl "https://api.cloudflare.com/client/v4/accounts/$ACCOUNT_ID/ai/finetunes" \ --request POST \ --header "Authorization: Bearer $CLOUDFLARE_API_TOKEN" \ --json '{ "model": "SUPPORTED_MODEL_NAME", "name": "FINETUNE_NAME", "description": "OPTIONAL_DESCRIPTION" }' ``` #### Uploading your adapter weights and config You have to call the upload endpoint each time you want to upload a new file, so you usually run this once for `adapter_model.safetensors` and once for `adapter_config.json`. Make sure you include the `@` before your path to files. You can either use the finetune `name` or `id` that you used when you created the fine tune. *cURLbash* ```bash ## Input: finetune_id, adapter_model.safetensors, then adapter_config.json ## Output: success true/false curl -X POST https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai/finetunes/{FINETUNE_ID}/finetune-assets/ \ -H 'Authorization: Bearer {API_TOKEN}' \ -H 'Content-Type: multipart/form-data' \ -F 'file_name=adapter_model.safetensors' \ -F 'file=@{PATH/TO/adapter_model.safetensors}' ``` #### List fine-tunes in your account You can call this method to confirm what fine-tunes you have created in your account
Required API token permissions At least one of the following token permissions is required: - Workers AI Write - Workers AI Read
*List Finetunesbash* ```bash curl "https://api.cloudflare.com/client/v4/accounts/$ACCOUNT_ID/ai/finetunes" \ --request GET \ --header "Authorization: Bearer $CLOUDFLARE_API_TOKEN" ``` ```json { "success": true, "result": [ [ { "id": "00000000-0000-0000-0000-000000000", "model": "@cf/meta-llama/llama-2-7b-chat-hf-lora", "name": "llama2-finetune", "description": "test" }, { "id": "00000000-0000-0000-0000-000000000", "model": "@cf/mistralai/mistral-7b-instruct-v0.2-lora", "name": "mistral-finetune", "description": "test" } ] ] } ``` --- ## Running inference with LoRAs To make inference requests and apply the LoRA adapter, you will need your model and finetune `name` or `id`. You should use the chat template that your LoRA was trained on, but you can try running it with `raw: true` and the messages template like below. ```javascript const response = await env.AI.run( "@cf/mistralai/mistral-7b-instruct-v0.2-lora", //the model supporting LoRAs { messages: [{ role: "user", content: "Hello world" }], raw: true, //skip applying the default chat template lora: "00000000-0000-0000-0000-000000000", //the finetune id OR name }, ); ``` ```bash curl https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai/run/@cf/mistral/mistral-7b-instruct-v0.2-lora \ -H 'Authorization: Bearer {API_TOKEN}' \ -d '{ "messages": [{"role": "user", "content": "Hello world"}], "raw": "true", "lora": "00000000-0000-0000-0000-000000000" }' ``` Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/features/fine-tunes/loras/#page","headline":"Using LoRA adapters","description":"Upload and use LoRA adapters to get fine-tuned inference on Workers AI.","url":"https://developers.cloudflare.com/workers-ai/features/fine-tunes/loras/","inLanguage":"en","image":"https://developers.cloudflare.com/workers-ai/features/fine-tunes/loras/og.png?v=5ab5aeb1f6d4d456","dateModified":"2026-04-21","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Cloudflare offers a few public LoRA adapters that are immediately ready for use. title: Public LoRA adapters image: https://developers.cloudflare.com/workers-ai/features/fine-tunes/public-loras/og.png?v=814cbee783f5a721 --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/workers-ai/llms.txt > Use this file to discover all available pages before exploring further. # Public LoRA adapters Last updated Jun 5, 2026|Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/features/fine-tunes/public-loras/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Cloudflare offers a few public LoRA adapters that can immediately be used for fine-tuned inference. You can try them out immediately via our [playground ↗︎](https://playground.ai.cloudflare.com). Public LoRAs will have the name `cf-public-x`, and the prefix will be reserved for Cloudflare. Note Have more LoRAs you would like to see? Let us know on [Discord ↗︎](https://discord.cloudflare.com). | Name | Description | Compatible with | | --- | --- | --- | | [cf-public-magicoder ↗︎](https://huggingface.co/predibase/magicoder) | Coding tasks in multiple languages | `@cf/mistral/mistral-7b-instruct-v0.1`
`@hf/mistral/mistral-7b-instruct-v0.2` | | [cf-public-jigsaw-classification ↗︎](https://huggingface.co/predibase/jigsaw) | Toxic comment classification | `@cf/mistral/mistral-7b-instruct-v0.1`
`@hf/mistral/mistral-7b-instruct-v0.2` | | [cf-public-cnn-summarization ↗︎](https://huggingface.co/predibase/cnn) | Article summarization | `@cf/mistral/mistral-7b-instruct-v0.1`
`@hf/mistral/mistral-7b-instruct-v0.2` | You can also list these public LoRAs with an API call:
Required API token permissions At least one of the following token permissions is required: - Workers AI Write - Workers AI Read
*List Public Finetunesbash* ```bash curl "https://api.cloudflare.com/client/v4/accounts/$ACCOUNT_ID/ai/finetunes/public" \ --request GET \ --header "Authorization: Bearer $CLOUDFLARE_API_TOKEN" ``` ## Running inference with public LoRAs To run inference with public LoRAs, you just need to define the LoRA name in the request. We recommend that you use the prompt template that the LoRA was trained on. You can find this in the HuggingFace repos linked above for each adapter. ### cURL ```bash curl https://api.cloudflare.com/client/v4/accounts/{account_id}/ai/run/@cf/mistral/mistral-7b-instruct-v0.1 \ --header 'Authorization: Bearer {cf_token}' \ --data '{ "messages": [ { "role": "user", "content": "Write a python program to check if a number is even or odd." } ], "lora": "cf-public-magicoder" }' ``` ### JavaScript ```js const answer = await env.AI.run("@cf/mistral/mistral-7b-instruct-v0.1", { stream: true, raw: true, messages: [ { role: "user", content: "Summarize the following: Some newspapers, TV channels and well-known companies publish false news stories to fool people on 1 April. One of the earliest examples of this was in 1957 when a programme on the BBC, the UKs national TV channel, broadcast a report on how spaghetti grew on trees. The film showed a family in Switzerland collecting spaghetti from trees and many people were fooled into believing it, as in the 1950s British people didn't eat much pasta and many didn't know how it was made! Most British people wouldnt fall for the spaghetti trick today, but in 2008 the BBC managed to fool their audience again with their Miracles of Evolution trailer, which appeared to show some special penguins that had regained the ability to fly. Two major UK newspapers, The Daily Telegraph and the Daily Mirror, published the important story on their front pages.", }, ], lora: "cf-public-cnn-summarization", }); ``` Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/features/fine-tunes/public-loras/#page","headline":"Public LoRA adapters","description":"Cloudflare offers a few public LoRA adapters that are immediately ready for use.","url":"https://developers.cloudflare.com/workers-ai/features/fine-tunes/public-loras/","inLanguage":"en","image":"https://developers.cloudflare.com/workers-ai/features/fine-tunes/public-loras/og.png?v=814cbee783f5a721","dateModified":"2026-06-05","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Enable Workers AI models to execute functions and interact with external APIs. title: Function calling image: https://developers.cloudflare.com/workers-ai/features/function-calling/og.png?v=2747bbb068ab45ae --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/workers-ai/llms.txt > Use this file to discover all available pages before exploring further. # Function calling Last updated Apr 21, 2026|Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/features/function-calling/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Function calling enables people to take Large Language Models (LLMs) and use the model response to execute functions or interact with external APIs. The developer usually defines a set of functions and the required input schema for each function, which we call `tools`. The model then intelligently understands when it needs to do a tool call, and it returns a JSON output which the user needs to feed to another function or API. In essence, function calling allows you to perform actions with LLMs by executing code or making additional API calls. ## How can I use function calling? Workers AI has [embedded function calling](https://developers.cloudflare.com/workers-ai/features/function-calling/embedded/) which allows you to execute function code alongside your inference calls. We have a package called [`@cloudflare/ai-utils` ↗︎](https://www.npmjs.com/package/@cloudflare/ai-utils) to help facilitate this, which we have open-sourced on [Github ↗︎](https://github.com/cloudflare/ai-utils). For industry-standard function calling, take a look at the documentation on [Traditional Function Calling](https://developers.cloudflare.com/workers-ai/features/function-calling/traditional/). To show you the value of embedded function calling, take a look at the example below that compares traditional function calling with embedded function calling. Embedded function calling allowed us to cut down the lines of code from 77 to 31. ```sh # The ai-utils package enables embedded function calling npm i @cloudflare/ai-utils ``` *Embedded function calling examplejs* ```js import { createToolsFromOpenAPISpec, runWithTools, autoTrimTools, } from "@cloudflare/ai-utils"; export default { async fetch(request, env, ctx) { const response = await runWithTools( env.AI, "@hf/nousresearch/hermes-2-pro-mistral-7b", { messages: [{ role: "user", content: "Who is Cloudflare on github?" }], tools: [ // You can pass the OpenAPI spec link or contents directly ...(await createToolsFromOpenAPISpec( "https://gist.githubusercontent.com/mchenco/fd8f20c8f06d50af40b94b0671273dc1/raw/f9d4b5cd5944cc32d6b34cad0406d96fd3acaca6/partial_api.github.com.json", { overrides: [ { // for all requests on *.github.com, we'll need to add a User-Agent. matcher: ({ url, method }) => { return url.hostname === "api.github.com"; }, values: { headers: { "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/112.0.0.0 Safari/537.36", }, }, }, ], }, )), ], }, ).then((response) => { return response; }); return new Response(JSON.stringify(response)); }, }; ``` *Traditional function calling examplejs* ```js export default { async fetch(request, env, ctx) { const response = await env.AI.run( "@hf/nousresearch/hermes-2-pro-mistral-7b", { messages: [{ role: "user", content: "Who is Cloudflare on GitHub?" }], tools: [ { name: "getGithubUser", description: "Provides publicly available information about someone with a GitHub account.", parameters: { type: "object", properties: { username: { type: "string", description: "The handle for the GitHub user account.", }, }, required: ["username"], }, }, ], }, ); const selected_tool = response.tool_calls[0]; let res; if (selected_tool.name == "getGithubUser") { try { const username = selected_tool.arguments.username; const url = `https://api.github.com/users/${username}`; res = await fetch(url, { headers: { // Github API requires a User-Agent header "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/112.0.0.0 Safari/537.36", }, }).then((res) => res.json()); } catch (error) { return error; } } const finalResponse = await env.AI.run( "@hf/nousresearch/hermes-2-pro-mistral-7b", { messages: [ { role: "user", content: "Who is Cloudflare on GitHub?", }, { role: "assistant", content: JSON.stringify(selected_tool), }, { role: "tool", content: JSON.stringify(res), }, ], tools: [ { name: "getGithubUser", description: "Provides publicly available information about someone with a GitHub account.", parameters: { type: "object", properties: { username: { type: "string", description: "The handle for the GitHub user account.", }, }, required: ["username"], }, }, ], }, ); return new Response(JSON.stringify(finalResponse)); }, }; ``` ## What models support function calling? There are open-source models which have been fine-tuned to do function calling. When browsing our [model catalog](https://developers.cloudflare.com/workers-ai/models/), look for models with the function calling property beside it. For example, [@hf/nousresearch/hermes-2-pro-mistral-7b](https://developers.cloudflare.com/workers-ai/models/hermes-2-pro-mistral-7b/) is a fine-tuned variant of Mistral 7B that you can use for function calling. Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"WebPage","@id":"https://developers.cloudflare.com/workers-ai/features/function-calling/#page","headline":"Function calling","description":"Enable Workers AI models to execute functions and interact with external APIs.","url":"https://developers.cloudflare.com/workers-ai/features/function-calling/","inLanguage":"en","image":"https://developers.cloudflare.com/workers-ai/features/function-calling/og.png?v=2747bbb068ab45ae","dateModified":"2026-04-21","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"},"keywords":["LLM"]} ``` --- --- description: Execute function code alongside inference calls using Workers AI embedded function calling. title: Embedded image: https://developers.cloudflare.com/workers-ai/features/function-calling/embedded/og.png?v=8834403aa34ffd28 --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/workers-ai/llms.txt > Use this file to discover all available pages before exploring further. # Embedded Last updated Apr 21, 2026|Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/features/function-calling/embedded/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Cloudflare has a unique [embedded function calling ↗︎](https://blog.cloudflare.com/embedded-function-calling) feature that allows you to execute function code alongside your tool call inference. Our npm package [`@cloudflare/ai-utils` ↗︎](https://www.npmjs.com/package/@cloudflare/ai-utils) is the developer toolkit to get started. Embedded function calling can be used to easily make complex agents that interact with websites and APIs, like using natural language to create meetings on Google Calendar, saving data to Notion, automatically routing requests to other APIs, saving data to an R2 bucket - or all of this at the same time. All you need is a prompt and an OpenAPI spec to get started. REST API support Embedded function calling depends on features native to the Workers platform. This means that embedded function calling is only supported via [Cloudflare Workers](https://developers.cloudflare.com/workers-ai/get-started/workers-wrangler/), not via the [REST API](https://developers.cloudflare.com/workers-ai/get-started/rest-api/). ## Resources - [Get Started](https://developers.cloudflare.com/workers-ai/features/function-calling/embedded/get-started/) - [Examples](https://developers.cloudflare.com/workers-ai/features/function-calling/embedded/examples/) - [API Reference](https://developers.cloudflare.com/workers-ai/features/function-calling/embedded/api-reference/) - [Troubleshooting](https://developers.cloudflare.com/workers-ai/features/function-calling/embedded/troubleshooting/) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/features/function-calling/embedded/#page","headline":"Embedded","description":"Execute function code alongside inference calls using Workers AI embedded function calling.","url":"https://developers.cloudflare.com/workers-ai/features/function-calling/embedded/","inLanguage":"en","image":"https://developers.cloudflare.com/workers-ai/features/function-calling/embedded/og.png?v=8834403aa34ffd28","dateModified":"2026-04-21","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Reference for the runWithTools and autoTrimTools methods in embedded function calling. title: API Reference image: https://developers.cloudflare.com/workers-ai/features/function-calling/embedded/api-reference/og.png?v=b717d7f8d4ddf28e --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/workers-ai/llms.txt > Use this file to discover all available pages before exploring further. # API Reference Last updated Apr 21, 2026|Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/features/function-calling/embedded/api-reference/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Learn more about the API reference for [embedded function calling](https://developers.cloudflare.com/workers-ai/features/function-calling/embedded). ## runWithTools This wrapper method enables you to do embedded function calling. You pass it the AI binding, model, inputs (`messages` array and `tools` array), and optional configurations. - `AI Binding`Ai - The AI binding, such as `env.AI`. - `model`BaseAiTextGenerationModels - The ID of the model that supports function calling. For example, `@hf/nousresearch/hermes-2-pro-mistral-7b`. - `input`Object - `messages`RoleScopedChatInput\[] - `tools`AiTextGenerationToolInputWithFunction\[] - `config`Object - `streamFinalResponse`boolean optional - `maxRecursiveToolRuns`number optional - `strictValidation`boolean optional - `verbose`boolean optional - `trimFunction`boolean optional - For the `trimFunction`, you can pass it `autoTrimTools`, which is another helper method we've devised to automatically choose the correct tools (using an LLM) before sending it off for inference. This means that your final inference call will have fewer input tokens. ## createToolsFromOpenAPISpec This method lets you automatically create tool schemas based on OpenAPI specs, so you don't have to manually write or hardcode the tool schemas. You can pass the OpenAPI spec for any API in JSON or YAML format. `createToolsFromOpenAPISpec` has a config input that allows you to perform overrides if you need to provide headers like Authentication or User-Agent. - `spec`string - The OpenAPI specification in either JSON or YAML format, or a URL to a remote OpenAPI specification. - `config`Config optional - Configuration options for the createToolsFromOpenAPISpec function - `overrides`ConfigRule\[] optional - `matchPatterns`RegExp\[] optional - `options` Object optional { `verbose` boolean optional } Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/features/function-calling/embedded/api-reference/#page","headline":"API Reference","description":"Reference for the runWithTools and autoTrimTools methods in embedded function calling.","url":"https://developers.cloudflare.com/workers-ai/features/function-calling/embedded/api-reference/","inLanguage":"en","image":"https://developers.cloudflare.com/workers-ai/features/function-calling/embedded/api-reference/og.png?v=b717d7f8d4ddf28e","dateModified":"2026-04-21","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Learn how to use the fetch() handler in Cloudflare Workers AI to enable LLMs to perform API calls, like retrieving a 5-day weather forecast using function calling. title: Use fetch() handler image: https://developers.cloudflare.com/workers-ai/features/function-calling/embedded/examples/fetch/og.png?v=840b375b65ba0cf7 --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/workers-ai/llms.txt > Use this file to discover all available pages before exploring further. # Use fetch() handler Last updated Apr 21, 2026|Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/features/function-calling/embedded/examples/fetch/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) A very common use case is to provide the LLM with the ability to perform API calls via function calling. In this example the LLM will retrieve the weather forecast for the next 5 days. To do so a `getWeather` function is defined that is passed to the LLM as tool. The `getWeather`function extracts the user's location from the request and calls the external weather API via the Workers' [`Fetch API`](https://developers.cloudflare.com/workers/runtime-apis/fetch/) and returns the result. *Embedded function calling example with fetch()ts* ```ts import { runWithTools } from '@cloudflare/ai-utils'; type Env = { AI: Ai; }; export default { async fetch(request, env, ctx) { // Define function const getWeather = async (args: { numDays: number }) => { const { numDays } = args; // Location is extracted from request based on // https://developers.cloudflare.com/workers/runtime-apis/request/#incomingrequestcfproperties const lat = request.cf?.latitude const long = request.cf?.longitude // Interpolate values for external API call const response = await fetch( `https://api.open-meteo.com/v1/forecast?latitude=${lat}&longitude=${long}&daily=temperature_2m_max,precipitation_sum&timezone=GMT&forecast_days=${numDays}` ); return response.text(); }; // Run AI inference with function calling const response = await runWithTools( env.AI, // Model with function calling support '@hf/nousresearch/hermes-2-pro-mistral-7b', { // Messages messages: [ { role: 'user', content: 'What the weather like the next 5 days? Respond as text', }, ], // Definition of available tools the AI model can leverage tools: [ { name: 'getWeather', description: 'Get the weather for the next [numDays] days', parameters: { type: 'object', properties: { numDays: { type: 'numDays', description: 'number of days for the weather forecast' }, }, required: ['numDays'], }, // reference to previously defined function function: getWeather, }, ], } ); return new Response(JSON.stringify(response)); }, } satisfies ExportedHandler; ``` Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/features/function-calling/embedded/examples/fetch/#page","headline":"Use fetch() handler","description":"Learn how to use the fetch() handler in Cloudflare Workers AI to enable LLMs to perform API calls, like retrieving a 5-day weather forecast using function calling.","url":"https://developers.cloudflare.com/workers-ai/features/function-calling/embedded/examples/fetch/","inLanguage":"en","image":"https://developers.cloudflare.com/workers-ai/features/function-calling/embedded/examples/fetch/og.png?v=840b375b65ba0cf7","dateModified":"2026-04-21","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"},"keywords":["AI"]} ``` --- --- description: Learn how to use Cloudflare Workers AI to interact with KV storage, enabling persistent data handling with embedded function calling in a few lines of code. title: Use KV API image: https://developers.cloudflare.com/workers-ai/features/function-calling/embedded/examples/kv/og.png?v=39ed64a561ba3ea0 --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/workers-ai/llms.txt > Use this file to discover all available pages before exploring further. # Use KV API Last updated Oct 13, 2025|Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/features/function-calling/embedded/examples/kv/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Interact with persistent storage to retrieve or store information enables for powerful use cases. In this example we show how embedded function calling can interact with other resources on the Cloudflare Developer Platform with a few lines of code. ## Pre-Requisites For this example to work, you need to provision a [KV](https://developers.cloudflare.com/kv/) namespace first. To do so, follow the [KV - Get started](https://developers.cloudflare.com/kv/get-started/) guide. Importantly, your Wrangler file must be updated to include the `KV` binding definition to your respective namespace. ## Worker code *Embedded function calling example with KV APIts* ```ts import { runWithTools } from "@cloudflare/ai-utils"; type Env = { AI: Ai; KV: KVNamespace; }; export default { async fetch(request, env, ctx) { // Define function const updateKvValue = async ({ key, value, }: { key: string; value: string; }) => { const response = await env.KV.put(key, value); return `Successfully updated key-value pair in database: ${response}`; }; // Run AI inference with function calling const response = await runWithTools( env.AI, "@hf/nousresearch/hermes-2-pro-mistral-7b", { messages: [ { role: "system", content: "Put user given values in KV" }, { role: "user", content: "Set the value of banana to yellow." }, ], tools: [ { name: "KV update", description: "Update a key-value pair in the database", parameters: { type: "object", properties: { key: { type: "string", description: "The key to update", }, value: { type: "string", description: "The value to update", }, }, required: ["key", "value"], }, function: updateKvValue, }, ], }, ); return new Response(JSON.stringify(response)); }, } satisfies ExportedHandler; ``` ## Verify results To verify the results, run the following command ```sh npx wrangler kv key get banana --binding KV --local ``` Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"WebPage","@id":"https://developers.cloudflare.com/workers-ai/features/function-calling/embedded/examples/kv/#page","headline":"Use KV API","description":"Learn how to use Cloudflare Workers AI to interact with KV storage, enabling persistent data handling with embedded function calling in a few lines of code.","url":"https://developers.cloudflare.com/workers-ai/features/function-calling/embedded/examples/kv/","inLanguage":"en","image":"https://developers.cloudflare.com/workers-ai/features/function-calling/embedded/examples/kv/og.png?v=39ed64a561ba3ea0","dateModified":"2025-10-13","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"},"keywords":["AI"]} ``` --- --- description: Generate Workers AI function calling tools from an OpenAPI specification using the ai-utils package. title: Tools based on OpenAPI Spec image: https://developers.cloudflare.com/workers-ai/features/function-calling/embedded/examples/openapi/og.png?v=d57cb6344a7eb170 --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/workers-ai/llms.txt > Use this file to discover all available pages before exploring further. # Tools based on OpenAPI Spec Last updated Apr 21, 2026|Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/features/function-calling/embedded/examples/openapi/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) Oftentimes APIs are defined and documented via [OpenAPI specification ↗︎](https://swagger.io/specification/). The Cloudflare `ai-utils` package's `createToolsFromOpenAPISpec` function creates tools from the OpenAPI spec, which the LLM can then leverage to fulfill the prompt. In this example the LLM will describe the a Github user, based Github's API and its OpenAPI spec. *Embedded function calling example from OpenAPI Spects* ```ts import { createToolsFromOpenAPISpec, runWithTools } from '@cloudflare/ai-utils'; type Env = { AI: Ai; }; const APP_NAME = 'cf-fn-calling-example-app'; export default { async fetch(request, env, ctx) { const toolsFromOpenAPISpec = [ // You can pass the OpenAPI spec link or contents directly ...(await createToolsFromOpenAPISpec( 'https://gist.githubusercontent.com/mchenco/fd8f20c8f06d50af40b94b0671273dc1/raw/f9d4b5cd5944cc32d6b34cad0406d96fd3acaca6/partial_api.github.com.json', { overrides: [ { matcher: ({ url }) => { return url.hostname === 'api.github.com'; }, // for all requests on *.github.com, we'll need to add a User-Agent. values: { headers: { 'User-Agent': APP_NAME, }, }, }, ], } )), ]; const response = await runWithTools( env.AI, '@hf/nousresearch/hermes-2-pro-mistral-7b', { messages: [ { role: 'user', content: 'Who is cloudflare on Github and how many repos does the organization have?', }, ], tools: toolsFromOpenAPISpec, } ); return new Response(JSON.stringify(response)); }, } satisfies ExportedHandler; ``` Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"WebPage","@id":"https://developers.cloudflare.com/workers-ai/features/function-calling/embedded/examples/openapi/#page","headline":"Tools based on OpenAPI Spec","description":"Generate Workers AI function calling tools from an OpenAPI specification using the ai-utils package.","url":"https://developers.cloudflare.com/workers-ai/features/function-calling/embedded/examples/openapi/","inLanguage":"en","image":"https://developers.cloudflare.com/workers-ai/features/function-calling/embedded/examples/openapi/og.png?v=d57cb6344a7eb170","dateModified":"2026-04-21","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"},"keywords":["AI"]} ``` --- --- description: Set up and deploy your first Workers AI project with embedded function calling. title: Get Started image: https://developers.cloudflare.com/workers-ai/features/function-calling/embedded/get-started/og.png?v=8085b109897093fc --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/workers-ai/llms.txt > Use this file to discover all available pages before exploring further. # Get Started Last updated Apr 21, 2026|Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/features/function-calling/embedded/get-started/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) This guide will instruct you through setting up and deploying your first Workers AI project with embedded function calling. You will use Workers, a Workers AI binding, the [`ai-utils package` ↗︎](https://github.com/cloudflare/ai-utils), and a large language model (LLM) to deploy your first AI-powered application on the Cloudflare global network with embedded function calling. ## 1. Create a Worker project with Workers AI Follow the [Workers AI Get Started Guide](https://developers.cloudflare.com/workers-ai/get-started/workers-wrangler/) until step 2. ## 2. Install additional npm package Next, run the following command in your project repository to install the Worker AI utilities package. npmyarnpnpmbun ``` npm i @cloudflare/ai-utils ``` ``` yarn add @cloudflare/ai-utils ``` ``` pnpm add @cloudflare/ai-utils ``` ``` bun add @cloudflare/ai-utils ``` ## 3. Add Workers AI Embedded function calling Update the `index.ts` file in your application directory with the following code: *index.jsjs* ```js import { runWithTools } from "@cloudflare/ai-utils"; export default { async fetch(request, env, ctx) { // Define function const sum = (args) => { const { a, b } = args; return Promise.resolve((a + b).toString()); }; // Run AI inference with function calling const response = await runWithTools( env.AI, // Model with function calling support "@hf/nousresearch/hermes-2-pro-mistral-7b", { // Messages messages: [ { role: "user", content: "What the result of 123123123 + 10343030?", }, ], // Definition of available tools the AI model can leverage tools: [ { name: "sum", description: "Sum up two numbers and returns the result", parameters: { type: "object", properties: { a: { type: "number", description: "the first number" }, b: { type: "number", description: "the second number" }, }, required: ["a", "b"], }, // reference to previously defined function function: sum, }, ], }, ); return new Response(JSON.stringify(response)); }, }; ``` *index.tsts* ```ts import { runWithTools } from "@cloudflare/ai-utils"; type Env = { AI: Ai; }; export default { async fetch(request, env, ctx) { // Define function const sum = (args: { a: number; b: number }): Promise => { const { a, b } = args; return Promise.resolve((a + b).toString()); }; // Run AI inference with function calling const response = await runWithTools( env.AI, // Model with function calling support "@hf/nousresearch/hermes-2-pro-mistral-7b", { // Messages messages: [ { role: "user", content: "What the result of 123123123 + 10343030?", }, ], // Definition of available tools the AI model can leverage tools: [ { name: "sum", description: "Sum up two numbers and returns the result", parameters: { type: "object", properties: { a: { type: "number", description: "the first number" }, b: { type: "number", description: "the second number" }, }, required: ["a", "b"], }, // reference to previously defined function function: sum, }, ], }, ); return new Response(JSON.stringify(response)); }, } satisfies ExportedHandler; ``` This example imports the utils with `import { runWithTools} from "@cloudflare/ai-utils"` and follows the API reference below. Moreover, in this example we define and describe a list of tools that the LLM can leverage to respond to the user query. Here, the list contains of only one tool, the `sum` function. Abstracted by the `runWithTools` function, the following steps occur: ``` sequenceDiagram participant Worker as Worker participant WorkersAI as Workers AI Worker->>+WorkersAI: Send messages, function calling prompt, and available tools WorkersAI->>+Worker: Select tools and arguments for function calling Worker-->>-Worker: Execute function Worker-->>+WorkersAI: Send messages, function calling prompt and function result WorkersAI-->>-Worker: Send response incorporating function output ``` The `ai-utils package` is also open-sourced on [Github ↗︎](https://github.com/cloudflare/ai-utils). ## 4. Local development & deployment Follow steps 4 and 5 of the [Workers AI Get Started Guide](https://developers.cloudflare.com/workers-ai/get-started/workers-wrangler/) for local development and deployment. Workers AI Embedded Function Calling charges Embedded function calling runs Workers AI inference requests. Standard charges for inference (e.g. tokens) usage will be charged. Resources consumed (e.g. CPU time) during embedded functions' code execution will be charged just as any other Worker's code execution. ## API reference For more details, refer to [API reference](https://developers.cloudflare.com/workers-ai/features/function-calling/embedded/api-reference/). Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/features/function-calling/embedded/get-started/#page","headline":"Get Started","description":"Set up and deploy your first Workers AI project with embedded function calling.","url":"https://developers.cloudflare.com/workers-ai/features/function-calling/embedded/get-started/","inLanguage":"en","image":"https://developers.cloudflare.com/workers-ai/features/function-calling/embedded/get-started/og.png?v=8085b109897093fc","dateModified":"2026-04-21","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Debug and resolve common issues with Workers AI embedded function calling. title: Troubleshooting image: https://developers.cloudflare.com/workers-ai/features/function-calling/embedded/troubleshooting/og.png?v=e7f024c00e0086e0 --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/workers-ai/llms.txt > Use this file to discover all available pages before exploring further. # Troubleshooting Last updated Apr 21, 2026|Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/features/function-calling/embedded/troubleshooting/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) This section will describe tools for troubleshooting and address common errors. ## Logging General [logging](https://developers.cloudflare.com/workers/observability/logs/) capabilities for Workers also apply to embedded function calling. ### Function invocations The invocations of tools can be logged as in any Worker using `console.log()`: *Logging tool invocationsts* ```ts export default { async fetch(request, env, ctx) { const sum = (args: { a: number; b: number }): Promise => { const { a, b } = args; // Logging from within embedded function invocations console.log(`The sum function has been invoked with the arguments a: ${a} and b: ${b}`) return Promise.resolve((a + b).toString()); }; ... } } ``` ### Logging within `runWithTools` The `runWithTools` function has a `verbose` mode that emits helpful logs for debugging of function calls as well input and output statistics. *Enabled verbose modets* ```ts const response = await runWithTools( env.AI, '@hf/nousresearch/hermes-2-pro-mistral-7b', { messages: [ ... ], tools: [ ... ], }, // Enable verbose mode { verbose: true } ); ``` ## Performance To respond to a LLM prompt with embedded function, potentially multiple AI inference requests and function invocations are needed, which can have an impact on user experience. Consider the following to improve performance: - Shorten prompts (to reduce time for input processing) - Reduce number of tools provided - Stream the final response to the end user (to minimize the time to interaction). See example below: *Streamed response examplets* ```ts async fetch(request, env, ctx) { const response = (await runWithTools( env.AI, '@hf/nousresearch/hermes-2-pro-mistral-7b', { messages: [ ... ], tools: [ ... ], }, { // Enable response streaming streamFinalResponse: true, } )) as ReadableStream; // Set response headers for streaming return new Response(response, { headers: { 'content-type': 'text/event-stream', }, }); } ``` ## Common Errors If you are getting a `BadInput` error, your inputs may exceed our current context window for our models. Try reducing input tokens to resolve this error. Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/features/function-calling/embedded/troubleshooting/#page","headline":"Troubleshooting","description":"Debug and resolve common issues with Workers AI embedded function calling.","url":"https://developers.cloudflare.com/workers-ai/features/function-calling/embedded/troubleshooting/","inLanguage":"en","image":"https://developers.cloudflare.com/workers-ai/features/function-calling/embedded/troubleshooting/og.png?v=e7f024c00e0086e0","dateModified":"2026-04-21","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Define tools and schemas for industry-standard function calling with Workers AI models. title: Traditional image: https://developers.cloudflare.com/workers-ai/features/function-calling/traditional/og.png?v=a7f9d52f0e6e4365 --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/workers-ai/llms.txt > Use this file to discover all available pages before exploring further. # Traditional Last updated Apr 21, 2026|Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/features/function-calling/traditional/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) This page shows how you can do traditional function calling, as defined by industry standards. Workers AI also offers [embedded function calling](https://developers.cloudflare.com/workers-ai/features/function-calling/embedded/), which is drastically easier than traditional function calling. With traditional function calling, you define an array of tools with the name, description, and tool arguments. The example below shows how you would pass a tool called `getWeather` in an inference request to a model. *Traditional function calling examplejs* ```js const response = await env.AI.run("@hf/nousresearch/hermes-2-pro-mistral-7b", { messages: [ { role: "user", content: "what is the weather in london?", }, ], tools: [ { name: "getWeather", description: "Return the weather for a latitude and longitude", parameters: { type: "object", properties: { latitude: { type: "string", description: "The latitude for the given location", }, longitude: { type: "string", description: "The longitude for the given location", }, }, required: ["latitude", "longitude"], }, }, ], }); return new Response(JSON.stringify(response.tool_calls)); ``` The LLM will then return a JSON object with the required arguments and the name of the tool that was called. You can then pass this JSON object to make an API call. ```json [ { "arguments": { "latitude": "51.5074", "longitude": "-0.1278" }, "name": "getWeather" } ] ``` For a working example on how to do function calling, take a look at our [demo app ↗︎](https://github.com/craigsdennis/lightbulb-moment-tool-calling/blob/main/src/index.ts). Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/features/function-calling/traditional/#page","headline":"Traditional","description":"Define tools and schemas for industry-standard function calling with Workers AI models.","url":"https://developers.cloudflare.com/workers-ai/features/function-calling/traditional/","inLanguage":"en","image":"https://developers.cloudflare.com/workers-ai/features/function-calling/traditional/og.png?v=a7f9d52f0e6e4365","dateModified":"2026-04-21","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Force Workers AI text generation models to return valid JSON output using response_format or JSON schemas. title: JSON Mode image: https://developers.cloudflare.com/workers-ai/features/json-mode/og.png?v=7526f0fdc7cd915d --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/workers-ai/llms.txt > Use this file to discover all available pages before exploring further. # JSON Mode Last updated Sep 14, 2026|Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/features/json-mode/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) When we want text-generation AI models to interact with databases, services, and external systems programmatically, typically when using tool calling or building AI agents, we must have structured response formats rather than natural language. Workers AI supports JSON Mode, enabling applications to request a structured output response when interacting with AI models. ## Schema JSON Mode is compatible with OpenAI’s implementation; to enable add the `response_format` property to the request object using the following convention: ```json { response_format: { title: "JSON Mode", type: "object", properties: { type: { type: "string", enum: ["json_object", "json_schema"], }, json_schema: {}, } } } ``` Where `json_schema` must be a valid [JSON Schema ↗︎](https://json-schema.org/) declaration. ## JSON Mode example When using JSON Format, pass the schema as in the example below as part of the request you send to the LLM. ```json { "messages": [ { "role": "system", "content": "Extract data about a country." }, { "role": "user", "content": "Tell me about India." } ], "response_format": { "type": "json_schema", "json_schema": { "type": "object", "properties": { "name": { "type": "string" }, "capital": { "type": "string" }, "languages": { "type": "array", "items": { "type": "string" } } }, "required": [ "name", "capital", "languages" ] } } } ``` The LLM will follow the schema, and return a response such as below: ```json { "response": { "name": "India", "capital": "New Delhi", "languages": [ "Hindi", "English", "Bengali", "Telugu", "Marathi", "Tamil", "Gujarati", "Urdu", "Kannada", "Odia", "Malayalam", "Punjabi", "Sanskrit" ] } } ``` As you can see, the model is complying with the JSON schema definition in the request and responding with a validated JSON object. ## Supported Models This is the list of models that now support JSON Mode: - [@cf/meta/llama-3.3-70b-instruct-fp8-fast](https://developers.cloudflare.com/workers-ai/models/llama-3.3-70b-instruct-fp8-fast/) - [@cf/meta/llama-3-8b-instruct](https://developers.cloudflare.com/workers-ai/models/llama-3-8b-instruct/) - [@cf/meta/llama-3.1-8b-instruct](https://developers.cloudflare.com/workers-ai/models/llama-3.1-8b-instruct/) - [@hf/nousresearch/hermes-2-pro-mistral-7b](https://developers.cloudflare.com/workers-ai/models/hermes-2-pro-mistral-7b/) - [@hf/thebloke/deepseek-coder-6.7b-instruct-awq](https://developers.cloudflare.com/workers-ai/models/deepseek-coder-6.7b-instruct-awq/) - [@cf/deepseek-ai/deepseek-r1-distill-qwen-32b](https://developers.cloudflare.com/workers-ai/models/deepseek-r1-distill-qwen-32b/) We will continue extending this list to keep up with new, and requested models. Note that Workers AI can't guarantee that the model responds according to the requested JSON Schema. Depending on the complexity of the task and adequacy of the JSON Schema, the model may not be able to satisfy the request in extreme situations. If that's the case, then an error `JSON Mode couldn't be met` is returned and must be handled. JSON Mode currently doesn't support streaming. Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"WebPage","@id":"https://developers.cloudflare.com/workers-ai/features/json-mode/#page","headline":"JSON Mode","description":"Force Workers AI text generation models to return valid JSON output using response_format or JSON schemas.","url":"https://developers.cloudflare.com/workers-ai/features/json-mode/","inLanguage":"en","image":"https://developers.cloudflare.com/workers-ai/features/json-mode/og.png?v=7526f0fdc7cd915d","dateModified":"2026-09-14","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"},"keywords":["JSON"]} ``` --- --- description: Convert documents in multiple formats to Markdown using the Workers AI toMarkdown method. title: Markdown Conversion image: https://developers.cloudflare.com/workers-ai/features/markdown-conversion/og.png?v=c777cc4964f21363 --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/workers-ai/llms.txt > Use this file to discover all available pages before exploring further. # Markdown Conversion Last updated Apr 21, 2026|Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/features/markdown-conversion/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) [Markdown ↗︎](https://en.wikipedia.org/wiki/Markdown) is essential for text generation and large language models (LLMs) in training and inference because it can provide structured, semantic, human, and machine-readable input. Likewise, Markdown facilitates chunking and structuring input data for better retrieval and synthesis in the context of RAGs, and its simplicity and ease of parsing and rendering make it ideal for AI Agents. For these reasons, document conversion plays an important role when designing and developing AI applications. Workers AI provides the `toMarkdown` utility method that developers can use from the [`env.AI`](https://developers.cloudflare.com/workers-ai/features/markdown-conversion/usage/binding/) binding or the [REST APIs](https://developers.cloudflare.com/workers-ai/features/markdown-conversion/usage/rest-api/) for quick, easy, and convenient conversion and summary of documents in multiple formats to Markdown language. ## Pricing `toMarkdown` is free for most format conversions. In some cases, like image conversion, it can use Workers AI models for object detection and summarization, which may incur additional costs if it exceeds the Workers AI free allocation limits. Refer to [what models we use](https://developers.cloudflare.com/workers-ai/features/markdown-conversion/how-it-works/) and the [Workers AI pricing page](https://developers.cloudflare.com/workers-ai/platform/pricing/) for more details. ## Other Markdown conversion features - The Browser Run [/markdown](https://developers.cloudflare.com/browser-run/quick-actions/markdown-endpoint/) endpoint supports markdown conversion if you need to render a dynamic page or application in a real browser before converting it. - [Markdown for Agents](https://developers.cloudflare.com/fundamentals/reference/markdown-for-agents/) allows real-time document conversion for Cloudflare zones using content negotiation headers. Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"WebPage","@id":"https://developers.cloudflare.com/workers-ai/features/markdown-conversion/#page","headline":"Markdown Conversion","description":"Convert documents in multiple formats to Markdown using the Workers AI toMarkdown method.","url":"https://developers.cloudflare.com/workers-ai/features/markdown-conversion/","inLanguage":"en","image":"https://developers.cloudflare.com/workers-ai/features/markdown-conversion/og.png?v=c777cc4964f21363","dateModified":"2026-04-21","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Configure per-format options for Workers AI Markdown Conversion, including HTML and image settings. title: Conversion Options image: https://developers.cloudflare.com/workers-ai/features/markdown-conversion/conversion-options/og.png?v=3a136bcaa6901b43 --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/workers-ai/llms.txt > Use this file to discover all available pages before exploring further. # Conversion Options Last updated Oct 15, 2026|Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/features/markdown-conversion/conversion-options/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) By default, the `toMarkdown` service extracts text content from your files. To further extend the capabilities of the conversion process, you can pass options to the service to control how specific file types are converted. Options are organized by file type and are all optional. ## Available options ### Output ```typescript { output?: { format?: 'markdown' | 'text'; } } ``` - `format`: controls the format of the converted content. Defaults to `markdown`. Set to `text` to receive plain text with Markdown syntax removed. When `format` is `text`, the `format` field of the [`ConversionResult`](https://developers.cloudflare.com/workers-ai/features/markdown-conversion/usage/binding/#conversionresult-definition) is also set to `text`. ### Images ```typescript { image?: { descriptionLanguage?: 'en' | 'it' | 'de' | 'es' | 'fr' | 'pt'; } } ``` - `descriptionLanguage`: controls the language of the AI-generated image descriptions. Caution This option works on a *best-effort* basis: it is not guaranteed that the resulting text will be in the desired language. ### HTML ```typescript { html?: { hostname?: string; cssSelector?: string; } } ``` - `hostname`: absolute base URL used to resolve relative links in the HTML. Include the scheme and, when applicable, the page path — for example, `https://example.com/docs/page`. A bare hostname such as `example.com` is not sufficient. - `cssSelector`: string containing a CSS selector pattern to pick specific elements from your HTML. Refer to [how HTML is processed](https://developers.cloudflare.com/workers-ai/features/markdown-conversion/how-it-works/#html) for more details. ### PDF ```typescript { pdf?: { metadata?: boolean; } } ``` - `metadata`: Previously, all converted PDF files always included metadata information when converted. This option allows you to opt-out of this behavior. ## Examples ### Binding To configure custom options, pass a `conversionOptions` object inside the second argument of the binding call, like this: ```typescript await env.AI.toMarkdown(..., { conversionOptions: { html: { ... }, pdf: { ... }, ... } }) ``` ### REST API Since the REST API uses file uploads, the request's `Content-Type` will be `multipart/form-data`. As such, include a new form field with your stringified object as a value: ```bash curl https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai/tomarkdown \ -X POST \ -H 'Authorization: Bearer {API_TOKEN}' \ ... -F 'conversionOptions={ "html": { ... }, ... }' ``` Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/features/markdown-conversion/conversion-options/#page","headline":"Conversion Options","description":"Configure per-format options for Workers AI Markdown Conversion, including HTML and image settings.","url":"https://developers.cloudflare.com/workers-ai/features/markdown-conversion/conversion-options/","inLanguage":"en","image":"https://developers.cloudflare.com/workers-ai/features/markdown-conversion/conversion-options/og.png?v=3a136bcaa6901b43","dateModified":"2026-10-15","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Learn how Workers AI pre-processes and converts HTML, images, and other files to Markdown. title: How it works image: https://developers.cloudflare.com/workers-ai/features/markdown-conversion/how-it-works/og.png?v=54dd4472fb6acd18 --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/workers-ai/llms.txt > Use this file to discover all available pages before exploring further. # How it works Last updated Oct 15, 2026|Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/workers-ai/features/markdown-conversion/how-it-works/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/) ## Pre-processing When parsing files before converting them to Markdown, there are some cleanup tasks we do depending on the type of file you are trying to convert. ### HTML When we detect an HTML file, a series of things happen to the HTML content before it is converted: - Some elements are ignored, including `script` and `style` tags. - Meta tags are extracted. These include `title`, `description`, `og:title`, `og:description` and `og:image`. - [JSON-LD ↗︎](https://json-ld.org/) content is extracted, if it exists. This will be appended at the end of the converted markdown. - The base URL to use for resolving relative links is extracted from the `` element1, if it exists, according to the spec (that is, only the first instance of the base URL is counted). - If the `cssSelector` option is: - present, then only those elements that match the selector are kept for further processing; - missing, then elements such as `
`, `