Skip to content
Draft
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/plenty-bulldogs-complain.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@firebase/ai': minor
---

Added ability to specify thinking levels in `thinkingConfig`.
12 changes: 12 additions & 0 deletions common/api-review/ai.api.md
Original file line number Diff line number Diff line change
Expand Up @@ -1371,8 +1371,20 @@ export interface TextPart {
export interface ThinkingConfig {
includeThoughts?: boolean;
thinkingBudget?: number;
thinkingLevel?: ThinkingLevel;
}

// @public
export const ThinkingLevel: {
MINIMAL: string;
LOW: string;
MEDIUM: string;
HIGH: string;
};

// @public
export type ThinkingLevel = (typeof ThinkingLevel)[keyof typeof ThinkingLevel];

// Warning: (ae-incompatible-release-tags) The symbol "Tool" is marked as @public, but its signature references "CodeExecutionTool" which is marked as @beta
// Warning: (ae-incompatible-release-tags) The symbol "Tool" is marked as @public, but its signature references "URLContextTool" which is marked as @beta
//
Expand Down
27 changes: 27 additions & 0 deletions docs-devsite/ai.md
Original file line number Diff line number Diff line change
Expand Up @@ -176,6 +176,7 @@ The Firebase AI Web SDK.
| [POSSIBLE\_ROLES](./ai.md#possible_roles) | Possible roles. |
| [ResponseModality](./ai.md#responsemodality) | <b><i>(Public Preview)</i></b> Generation modalities to be returned in generation responses. |
| [SchemaType](./ai.md#schematype) | Contains the list of OpenAPI data types as defined by the [OpenAPI specification](https://swagger.io/docs/specification/data-models/data-types/) |
| [ThinkingLevel](./ai.md#thinkinglevel) | A preset that controls the model's "thinking" process. Use <code>ThinkingLevel.LOW</code> for faster responses on less complex tasks, and <code>ThinkingLevel.HIGH</code> for better reasoning on more complex tasks. |
| [URLRetrievalStatus](./ai.md#urlretrievalstatus) | <b><i>(Public Preview)</i></b> The status of a URL retrieval. |

## Type Aliases
Expand Down Expand Up @@ -208,6 +209,7 @@ The Firebase AI Web SDK.
| [ResponseModality](./ai.md#responsemodality) | <b><i>(Public Preview)</i></b> Generation modalities to be returned in generation responses. |
| [Role](./ai.md#role) | Role is the producer of the content. |
| [SchemaType](./ai.md#schematype) | Contains the list of OpenAPI data types as defined by the [OpenAPI specification](https://swagger.io/docs/specification/data-models/data-types/) |
| [ThinkingLevel](./ai.md#thinkinglevel) | A preset that controls the model's "thinking" process. Use <code>ThinkingLevel.LOW</code> for faster responses on less complex tasks, and <code>ThinkingLevel.HIGH</code> for better reasoning on more complex tasks. |
| [Tool](./ai.md#tool) | Defines a tool that model can call to access external knowledge. |
| [TypedSchema](./ai.md#typedschema) | A type that includes all specific Schema types. |
| [URLRetrievalStatus](./ai.md#urlretrievalstatus) | <b><i>(Public Preview)</i></b> The status of a URL retrieval. |
Expand Down Expand Up @@ -827,6 +829,21 @@ SchemaType: {
}
```

## ThinkingLevel

A preset that controls the model's "thinking" process. Use `ThinkingLevel.LOW` for faster responses on less complex tasks, and `ThinkingLevel.HIGH` for better reasoning on more complex tasks.

<b>Signature:</b>

```typescript
ThinkingLevel: {
MINIMAL: string;
LOW: string;
MEDIUM: string;
HIGH: string;
}
```

## URLRetrievalStatus

> This API is provided as a preview for developers and may change based on feedback that we receive. Do not use this API in a production environment.
Expand Down Expand Up @@ -1142,6 +1159,16 @@ Contains the list of OpenAPI data types as defined by the [OpenAPI specification
export type SchemaType = (typeof SchemaType)[keyof typeof SchemaType];
```

## ThinkingLevel

A preset that controls the model's "thinking" process. Use `ThinkingLevel.LOW` for faster responses on less complex tasks, and `ThinkingLevel.HIGH` for better reasoning on more complex tasks.

<b>Signature:</b>

```typescript
export type ThinkingLevel = (typeof ThinkingLevel)[keyof typeof ThinkingLevel];
```

## Tool

Defines a tool that model can call to access external knowledge.
Expand Down
27 changes: 25 additions & 2 deletions docs-devsite/ai.thinkingconfig.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,8 @@ export interface ThinkingConfig
| Property | Type | Description |
| --- | --- | --- |
| [includeThoughts](./ai.thinkingconfig.md#thinkingconfigincludethoughts) | boolean | Whether to include "thought summaries" in the model's response. |
| [thinkingBudget](./ai.thinkingconfig.md#thinkingconfigthinkingbudget) | number | The thinking budget, in tokens.<!-- -->This parameter sets an upper limit on the number of tokens the model can use for its internal "thinking" process. A higher budget may result in higher quality responses for complex tasks but can also increase latency and cost.<!-- -->If you don't specify a budget, the model will determine the appropriate amount of thinking based on the complexity of the prompt.<!-- -->An error will be thrown if you set a thinking budget for a model that does not support this feature or if the specified budget is not within the model's supported range. |
| [thinkingBudget](./ai.thinkingconfig.md#thinkingconfigthinkingbudget) | number | The thinking budget, in tokens. |
| [thinkingLevel](./ai.thinkingconfig.md#thinkingconfigthinkinglevel) | [ThinkingLevel](./ai.md#thinkinglevel) | If not specified, Gemini will use the model's default dynamic thinking level. |

## ThinkingConfig.includeThoughts

Expand All @@ -45,12 +46,34 @@ The thinking budget, in tokens.

This parameter sets an upper limit on the number of tokens the model can use for its internal "thinking" process. A higher budget may result in higher quality responses for complex tasks but can also increase latency and cost.

If you don't specify a budget, the model will determine the appropriate amount of thinking based on the complexity of the prompt.
The range of supported thinking budget values depends on the model.

<ul> <li>To use the default thinking budget or thinking level for a model, leave this value undefined.</li>

<li>To disable thinking, when supported by the model, set this value to `0`<!-- -->.</li>

<li>To use dynamic thinking, allowing the model to decide on the thinking budget based on the task, set this value to `-1`<!-- -->.</li> </ul>

An error will be thrown if you set a thinking budget for a model that does not support this feature or if the specified budget is not within the model's supported range.

The model will also error if `thinkingLevel` and `thinkingBudget` are both set.

<b>Signature:</b>

```typescript
thinkingBudget?: number;
```

## ThinkingConfig.thinkingLevel

If not specified, Gemini will use the model's default dynamic thinking level.

Note: The model will error if `thinkingLevel` and `thinkingBudget` are both set.

Important: Gemini 2.5 series models do not support thinking levels; use `thinkingBudget` to set a thinking budget instead.

<b>Signature:</b>

```typescript
thinkingLevel?: ThinkingLevel;
```
17 changes: 16 additions & 1 deletion packages/ai/src/models/generative-model.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,8 @@ import {
AI,
InferenceMode,
AIErrorCode,
ChromeAdapter
ChromeAdapter,
ThinkingLevel
} from '../public-types';
import * as request from '../requests/request';
import { SinonStub, match, restore, stub } from 'sinon';
Expand Down Expand Up @@ -53,6 +54,20 @@ const fakeAI: AI = {
};

describe('GenerativeModel', () => {
it('throws if generationConfig is invalid', () => {
expect(
() =>
new GenerativeModel(fakeAI, {
model: 'my-model',
generationConfig: {
thinkingConfig: {
thinkingBudget: 1000,
thinkingLevel: ThinkingLevel.LOW
}
}
})
).to.throw(AIErrorCode.UNSUPPORTED);
});
it('passes params through to generateContent', async () => {
const genModel = new GenerativeModel(
fakeAI,
Expand Down
21 changes: 20 additions & 1 deletion packages/ai/src/models/generative-model.ts
Original file line number Diff line number Diff line change
Expand Up @@ -41,9 +41,10 @@ import {
formatGenerateContentInput,
formatSystemInstruction
} from '../requests/request-helpers';
import { AI } from '../public-types';
import { AI, AIErrorCode } from '../public-types';
import { AIModel } from './ai-model';
import { ChromeAdapter } from '../types/chrome-adapter';
import { AIError } from '../errors';

/**
* Class for generative model APIs.
Expand All @@ -65,6 +66,7 @@ export class GenerativeModel extends AIModel {
) {
super(ai, modelParams.model);
this.generationConfig = modelParams.generationConfig || {};
validateGenerationConfig(this.generationConfig);
this.safetySettings = modelParams.safetySettings || [];
this.tools = modelParams.tools;
this.toolConfig = modelParams.toolConfig;
Expand Down Expand Up @@ -165,3 +167,20 @@ export class GenerativeModel extends AIModel {
);
}
}

/**
* Client-side validation of some common `GenerationConfig` pitfalls, in order
* to save the developer a wasted request.
*/
function validateGenerationConfig(generationConfig: GenerationConfig): void {
if (
// != allows for null and undefined. 0 is considered "set" by the model
generationConfig.thinkingConfig?.thinkingBudget != null &&
generationConfig.thinkingConfig.thinkingLevel
) {
throw new AIError(
AIErrorCode.UNSUPPORTED,
`Cannot set both thinkingBudget and thinkingLevel in a config.`
);
}
}
Comment on lines 175 to 186
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

While the current logic is correct due to && short-circuiting, it's a bit subtle. Accessing generationConfig.thinkingConfig.thinkingLevel could be misinterpreted as unsafe if one overlooks the short-circuiting behavior. Refactoring to first check for thinkingConfig's existence improves readability and maintainability.

Suggested change
function validateGenerationConfig(generationConfig: GenerationConfig): void {
if (
// != allows for null and undefined. 0 is considered "set" by the model
generationConfig.thinkingConfig?.thinkingBudget != null &&
generationConfig.thinkingConfig.thinkingLevel
) {
throw new AIError(
AIErrorCode.UNSUPPORTED,
`Cannot set both thinkingBudget and thinkingLevel in a config.`
);
}
}
function validateGenerationConfig(generationConfig: GenerationConfig): void {
const { thinkingConfig } = generationConfig;
if (
thinkingConfig &&
// != allows for null and undefined. 0 is considered "set" by the model
thinkingConfig.thinkingBudget != null &&
thinkingConfig.thinkingLevel
) {
throw new AIError(
AIErrorCode.UNSUPPORTED,
`Cannot set both thinkingBudget and thinkingLevel in a config.`
);
}
}

23 changes: 23 additions & 0 deletions packages/ai/src/types/enums.ts
Original file line number Diff line number Diff line change
Expand Up @@ -432,3 +432,26 @@ export const Language = {
* @beta
*/
export type Language = (typeof Language)[keyof typeof Language];

/**
* A preset that controls the model's "thinking" process. Use
* `ThinkingLevel.LOW` for faster responses on less complex tasks, and
* `ThinkingLevel.HIGH` for better reasoning on more complex tasks.
*
* @public
*/
export const ThinkingLevel = {
MINIMAL: 'MINIMAL',
LOW: 'LOW',
MEDIUM: 'MEDIUM',
HIGH: 'HIGH'
};

/**
* A preset that controls the model's "thinking" process. Use
* `ThinkingLevel.LOW` for faster responses on less complex tasks, and
* `ThinkingLevel.HIGH` for better reasoning on more complex tasks.
*
* @public
*/
export type ThinkingLevel = (typeof ThinkingLevel)[keyof typeof ThinkingLevel];
33 changes: 30 additions & 3 deletions packages/ai/src/types/requests.ts
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,8 @@ import {
HarmBlockThreshold,
HarmCategory,
InferenceMode,
ResponseModality
ResponseModality,
ThinkingLevel
} from './enums';
import { ObjectSchemaRequest, SchemaRequest } from './schema';

Expand Down Expand Up @@ -436,18 +437,44 @@ export interface ThinkingConfig {
/**
* The thinking budget, in tokens.
*
* @remarks
* This parameter sets an upper limit on the number of tokens the model can use for its internal
* "thinking" process. A higher budget may result in higher quality responses for complex tasks
* but can also increase latency and cost.
*
* If you don't specify a budget, the model will determine the appropriate amount
* of thinking based on the complexity of the prompt.
* The range of supported thinking budget values depends on the model.
*
* <ul>
* <li>To use the default thinking budget or thinking level for a model, leave
* this value undefined.</li>
*
* <li>To disable thinking, when supported by the model, set this value
* to `0`.</li>
*
* <li>To use dynamic thinking, allowing the model to decide on the thinking
* budget based on the task, set this value to `-1`.</li>
* </ul>
*
* An error will be thrown if you set a thinking budget for a model that does not support this
* feature or if the specified budget is not within the model's supported range.
*
* The model will also error if `thinkingLevel` and `thinkingBudget` are
* both set.
*/
thinkingBudget?: number;

/**
* If not specified, Gemini will use the model's default dynamic thinking level.
*
* @remarks
* Note: The model will error if `thinkingLevel` and `thinkingBudget` are
* both set.
*
* Important: Gemini 2.5 series models do not support thinking levels; use
* `thinkingBudget` to set a thinking budget instead.
*/
thinkingLevel?: ThinkingLevel;

/**
* Whether to include "thought summaries" in the model's response.
*
Expand Down
Loading