Model provider API keys

There are two approaches to adding a model provider API key.

• In one scenario, you get one provider connection per organization, and this provides access to a set of whitelisted models. Examples include:

  • OpenAI
  • Vertex AI
  • Gemini

• In the second scenario, you add a separate API key per model. Examples include:

  • Azure OpenAI
  • Custom

Once a model is added via the API key, anyone in the organization who has access to the Prompts feature can select the associated models when executing their prompt.

You can see what API keys you have and add new ones by clicking API Keys in the top right of the Prompts page to open the Model Provider API Keys window:

OpenAI API key

You can only have one OpenAI key per organization. This grants you access to set of whitelisted models. For a list of these models, see Supported base models.

If you don’t already have one, you can create an OpenAI account here.

You can find your OpenAI API key on the API key page.

Once added, all supported models will appear in the base model drop-down when you draft your prompt.

Gemini API key

You can only have one Gemini key per organization. This grants you access to set of whitelisted models. For a list of these models, see Supported base models.

For information on getting a Gemini API key, see Get a Gemini API key.

Once added, all supported models will appear in the base model drop-down when you draft your prompt.

Vertex AI JSON credentials

You can only have one Vertex AI key per organization. This grants you access to set of whitelisted models. For a list of these models, see Supported base models.

Follow the instructions here to generate a credentials file in JSON format: Authenticate to Vertex AI Agent Builder - Client libraries or third-party tools

The JSON credentials are required. You can also optionally provide the project ID and location associated with your Google Cloud Platform environment.

Once added, all supported models will appear in the base model drop-down when you draft your prompt.

Azure OpenAI key

Each Azure OpenAI key is tied to a specific deployment, and each deployment comprises a single OpenAI model. So if you want to use multiple models through Azure, you will need to create a deployment for each model and then add each key to Label Studio.

For a list of the Azure OpenAI models we support, see Supported base models.

To use Azure OpenAI, you must first create the Azure OpenAI resource and then a model deployment:

1. From the Azure portal, create an Azure OpenAI resource.

2. From Azure OpenAI Studio, create a deployment. This is a base model endpoint.

When adding the key to Label Studio, you are asked for the following information:

Field	Description
Deployment	The is the name of the deployment. By default, this is the same as the model name, but you can customize it when you create the deployment. If they are different, you must use the deployment name and not the underlying model name.
Endpoint	This is the target URI provided by Azure.
API key	This is the key provided by Azure.

You can find all this information in the Details section of the deployment in Azure OpenAI Studio.

Custom LLM

You can use your own self-hosted and fine-tuned model as long as it meets the following criteria:

• Your server must provide JSON mode for the LLM.
• The server API must follow OpenAI format.

Examples of compatible LLMs include Ollama and sglang.

To add a custom model, enter the following:

• A name for the model.
• The endpoint URL for the model. For example, https://my.openai.endpoint.com/v1
• An API key to access the model. An API key is tied to a specific account, but the access is shared within the org if added. (Optional)
• An auth token to access the model API. An auth token provides API access at the server level. (Optional)