Skip to content

Commit 5223b6a

Browse files
pathfinder-pfpathfinder-fp
andauthored
fix return logprobs doc (#508)
Co-authored-by: pathfinder-fp <[email protected]>
1 parent 1e3e697 commit 5223b6a

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

docs/features/global_jit_compile.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@
1111

1212
The wrapped forward function is `jitted_run_model` and `jitted_sampler`, which are used in prefill and decode. The input parameter `forward_batch`, `logits_metadata`, `logits_output` and `sampling_metadata` have to be registered as PyTrees. At the same time, the subclasses in it are required to register too. Besides, we use `nnx.split` and `nnx.merge` on model to keep satisfy the `jit` requirements.
1313

14-
Note: `return_logprob` is not supported in `jitted_run_model` and `jitted_sampler`, this feature with jit may be supported in the future.
14+
Note: `return_logprob` is supported in `jitted_run_model` and `jitted_sampler` with performance degradation, It will be optimized later.
1515

1616
### Precompile and padding
1717

0 commit comments

Comments
 (0)