Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .changeset/plenty-bulldogs-complain.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
'@firebase/ai': minor
'firebase': minor
---

Added ability to specify thinking levels in `thinkingConfig`.
12 changes: 12 additions & 0 deletions common/api-review/ai.api.md
Original file line number Diff line number Diff line change
Expand Up @@ -1371,8 +1371,20 @@ export interface TextPart {
export interface ThinkingConfig {
includeThoughts?: boolean;
thinkingBudget?: number;
thinkingLevel?: ThinkingLevel;
}

// @public
export const ThinkingLevel: {
MINIMAL: string;
LOW: string;
MEDIUM: string;
HIGH: string;
};

// @public
export type ThinkingLevel = (typeof ThinkingLevel)[keyof typeof ThinkingLevel];

// Warning: (ae-incompatible-release-tags) The symbol "Tool" is marked as @public, but its signature references "CodeExecutionTool" which is marked as @beta
// Warning: (ae-incompatible-release-tags) The symbol "Tool" is marked as @public, but its signature references "URLContextTool" which is marked as @beta
//
Expand Down
27 changes: 27 additions & 0 deletions docs-devsite/ai.md
Original file line number Diff line number Diff line change
Expand Up @@ -176,6 +176,7 @@ The Firebase AI Web SDK.
| [POSSIBLE\_ROLES](./ai.md#possible_roles) | Possible roles. |
| [ResponseModality](./ai.md#responsemodality) | <b><i>(Public Preview)</i></b> Generation modalities to be returned in generation responses. |
| [SchemaType](./ai.md#schematype) | Contains the list of OpenAPI data types as defined by the [OpenAPI specification](https://swagger.io/docs/specification/data-models/data-types/) |
| [ThinkingLevel](./ai.md#thinkinglevel) | A preset that controls the model's "thinking" process. Use <code>ThinkingLevel.LOW</code> for faster responses on less complex tasks, and <code>ThinkingLevel.HIGH</code> for better reasoning on more complex tasks. |
| [URLRetrievalStatus](./ai.md#urlretrievalstatus) | <b><i>(Public Preview)</i></b> The status of a URL retrieval. |

## Type Aliases
Expand Down Expand Up @@ -208,6 +209,7 @@ The Firebase AI Web SDK.
| [ResponseModality](./ai.md#responsemodality) | <b><i>(Public Preview)</i></b> Generation modalities to be returned in generation responses. |
| [Role](./ai.md#role) | Role is the producer of the content. |
| [SchemaType](./ai.md#schematype) | Contains the list of OpenAPI data types as defined by the [OpenAPI specification](https://swagger.io/docs/specification/data-models/data-types/) |
| [ThinkingLevel](./ai.md#thinkinglevel) | A preset that controls the model's "thinking" process. Use <code>ThinkingLevel.LOW</code> for faster responses on less complex tasks, and <code>ThinkingLevel.HIGH</code> for better reasoning on more complex tasks. |
| [Tool](./ai.md#tool) | Defines a tool that model can call to access external knowledge. |
| [TypedSchema](./ai.md#typedschema) | A type that includes all specific Schema types. |
| [URLRetrievalStatus](./ai.md#urlretrievalstatus) | <b><i>(Public Preview)</i></b> The status of a URL retrieval. |
Expand Down Expand Up @@ -827,6 +829,21 @@ SchemaType: {
}
```

## ThinkingLevel

A preset that controls the model's "thinking" process. Use `ThinkingLevel.LOW` for faster responses on less complex tasks, and `ThinkingLevel.HIGH` for better reasoning on more complex tasks.

<b>Signature:</b>

```typescript
ThinkingLevel: {
MINIMAL: string;
LOW: string;
MEDIUM: string;
HIGH: string;
}
```

## URLRetrievalStatus

> This API is provided as a preview for developers and may change based on feedback that we receive. Do not use this API in a production environment.
Expand Down Expand Up @@ -1142,6 +1159,16 @@ Contains the list of OpenAPI data types as defined by the [OpenAPI specification
export type SchemaType = (typeof SchemaType)[keyof typeof SchemaType];
```

## ThinkingLevel

A preset that controls the model's "thinking" process. Use `ThinkingLevel.LOW` for faster responses on less complex tasks, and `ThinkingLevel.HIGH` for better reasoning on more complex tasks.

<b>Signature:</b>

```typescript
export type ThinkingLevel = (typeof ThinkingLevel)[keyof typeof ThinkingLevel];
```

## Tool

Defines a tool that model can call to access external knowledge.
Expand Down
27 changes: 25 additions & 2 deletions docs-devsite/ai.thinkingconfig.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,8 @@ export interface ThinkingConfig
| Property | Type | Description |
| --- | --- | --- |
| [includeThoughts](./ai.thinkingconfig.md#thinkingconfigincludethoughts) | boolean | Whether to include "thought summaries" in the model's response. |
| [thinkingBudget](./ai.thinkingconfig.md#thinkingconfigthinkingbudget) | number | The thinking budget, in tokens.<!-- -->This parameter sets an upper limit on the number of tokens the model can use for its internal "thinking" process. A higher budget may result in higher quality responses for complex tasks but can also increase latency and cost.<!-- -->If you don't specify a budget, the model will determine the appropriate amount of thinking based on the complexity of the prompt.<!-- -->An error will be thrown if you set a thinking budget for a model that does not support this feature or if the specified budget is not within the model's supported range. |
| [thinkingBudget](./ai.thinkingconfig.md#thinkingconfigthinkingbudget) | number | The thinking budget, in tokens. |
| [thinkingLevel](./ai.thinkingconfig.md#thinkingconfigthinkinglevel) | [ThinkingLevel](./ai.md#thinkinglevel) | If not specified, Gemini will use the model's default dynamic thinking level. |

## ThinkingConfig.includeThoughts

Expand All @@ -45,12 +46,34 @@ The thinking budget, in tokens.

This parameter sets an upper limit on the number of tokens the model can use for its internal "thinking" process. A higher budget may result in higher quality responses for complex tasks but can also increase latency and cost.

If you don't specify a budget, the model will determine the appropriate amount of thinking based on the complexity of the prompt.
The range of supported thinking budget values depends on the model.

<ul> <li>To use the default thinking budget or thinking level for a model, leave this value undefined.</li>

<li>To disable thinking, when supported by the model, set this value to `0`<!-- -->.</li>

<li>To use dynamic thinking, allowing the model to decide on the thinking budget based on the task, set this value to `-1`<!-- -->.</li> </ul>

An error will be thrown if you set a thinking budget for a model that does not support this feature or if the specified budget is not within the model's supported range.

The model will also error if `thinkingLevel` and `thinkingBudget` are both set.

<b>Signature:</b>

```typescript
thinkingBudget?: number;
```

## ThinkingConfig.thinkingLevel

If not specified, Gemini will use the model's default dynamic thinking level.

Note: The model will error if `thinkingLevel` and `thinkingBudget` are both set.

Important: Gemini 2.5 series models do not support thinking levels; use `thinkingBudget` to set a thinking budget instead.

<b>Signature:</b>

```typescript
thinkingLevel?: ThinkingLevel;
```
17 changes: 16 additions & 1 deletion packages/ai/src/models/generative-model.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,8 @@ import {
AI,
InferenceMode,
AIErrorCode,
ChromeAdapter
ChromeAdapter,
ThinkingLevel
} from '../public-types';
import * as request from '../requests/request';
import { SinonStub, match, restore, stub } from 'sinon';
Expand Down Expand Up @@ -53,6 +54,20 @@ const fakeAI: AI = {
};

describe('GenerativeModel', () => {
it('throws if generationConfig is invalid', () => {
expect(
() =>
new GenerativeModel(fakeAI, {
model: 'my-model',
generationConfig: {
thinkingConfig: {
thinkingBudget: 1000,
thinkingLevel: ThinkingLevel.LOW
}
}
})
).to.throw(AIErrorCode.UNSUPPORTED);
});
it('passes params through to generateContent', async () => {
const genModel = new GenerativeModel(
fakeAI,
Expand Down
21 changes: 20 additions & 1 deletion packages/ai/src/models/generative-model.ts
Original file line number Diff line number Diff line change
Expand Up @@ -41,9 +41,10 @@ import {
formatGenerateContentInput,
formatSystemInstruction
} from '../requests/request-helpers';
import { AI } from '../public-types';
import { AI, AIErrorCode } from '../public-types';
import { AIModel } from './ai-model';
import { ChromeAdapter } from '../types/chrome-adapter';
import { AIError } from '../errors';

/**
* Class for generative model APIs.
Expand All @@ -65,6 +66,7 @@ export class GenerativeModel extends AIModel {
) {
super(ai, modelParams.model);
this.generationConfig = modelParams.generationConfig || {};
validateGenerationConfig(this.generationConfig);
this.safetySettings = modelParams.safetySettings || [];
this.tools = modelParams.tools;
this.toolConfig = modelParams.toolConfig;
Expand Down Expand Up @@ -165,3 +167,20 @@ export class GenerativeModel extends AIModel {
);
}
}

/**
* Client-side validation of some common `GenerationConfig` pitfalls, in order
* to save the developer a wasted request.
*/
function validateGenerationConfig(generationConfig: GenerationConfig): void {
if (
// != allows for null and undefined. 0 is considered "set" by the model
generationConfig.thinkingConfig?.thinkingBudget != null &&
generationConfig.thinkingConfig?.thinkingLevel
) {
throw new AIError(
AIErrorCode.UNSUPPORTED,
`Cannot set both thinkingBudget and thinkingLevel in a config.`
);
}
}
23 changes: 23 additions & 0 deletions packages/ai/src/types/enums.ts
Original file line number Diff line number Diff line change
Expand Up @@ -432,3 +432,26 @@ export const Language = {
* @beta
*/
export type Language = (typeof Language)[keyof typeof Language];

/**
* A preset that controls the model's "thinking" process. Use
* `ThinkingLevel.LOW` for faster responses on less complex tasks, and
* `ThinkingLevel.HIGH` for better reasoning on more complex tasks.
*
* @public
*/
export const ThinkingLevel = {
MINIMAL: 'MINIMAL',
LOW: 'LOW',
MEDIUM: 'MEDIUM',
HIGH: 'HIGH'
};

/**
* A preset that controls the model's "thinking" process. Use
* `ThinkingLevel.LOW` for faster responses on less complex tasks, and
* `ThinkingLevel.HIGH` for better reasoning on more complex tasks.
*
* @public
*/
export type ThinkingLevel = (typeof ThinkingLevel)[keyof typeof ThinkingLevel];
33 changes: 30 additions & 3 deletions packages/ai/src/types/requests.ts
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,8 @@ import {
HarmBlockThreshold,
HarmCategory,
InferenceMode,
ResponseModality
ResponseModality,
ThinkingLevel
} from './enums';
import { ObjectSchemaRequest, SchemaRequest } from './schema';

Expand Down Expand Up @@ -436,18 +437,44 @@ export interface ThinkingConfig {
/**
* The thinking budget, in tokens.
*
* @remarks
* This parameter sets an upper limit on the number of tokens the model can use for its internal
* "thinking" process. A higher budget may result in higher quality responses for complex tasks
* but can also increase latency and cost.
*
* If you don't specify a budget, the model will determine the appropriate amount
* of thinking based on the complexity of the prompt.
* The range of supported thinking budget values depends on the model.
*
* <ul>
* <li>To use the default thinking budget or thinking level for a model, leave
* this value undefined.</li>
*
* <li>To disable thinking, when supported by the model, set this value
* to `0`.</li>
*
* <li>To use dynamic thinking, allowing the model to decide on the thinking
* budget based on the task, set this value to `-1`.</li>
* </ul>
*
* An error will be thrown if you set a thinking budget for a model that does not support this
* feature or if the specified budget is not within the model's supported range.
*
* The model will also error if `thinkingLevel` and `thinkingBudget` are
* both set.
*/
thinkingBudget?: number;

/**
* If not specified, Gemini will use the model's default dynamic thinking level.
*
* @remarks
* Note: The model will error if `thinkingLevel` and `thinkingBudget` are
* both set.
*
* Important: Gemini 2.5 series models do not support thinking levels; use
* `thinkingBudget` to set a thinking budget instead.
*/
thinkingLevel?: ThinkingLevel;

/**
* Whether to include "thought summaries" in the model's response.
*
Expand Down
Loading