[Feature Request] Support RoPE Scaling (YaRN) for Long Context Extension Training

[dawson-chen]

We are using the Qwen3-32B model for RL training. This model natively supports 32K context length and can extend the context window to 64K or even 128K using the YaRN method.

During the previous SFT stage, we enabled YaRN for long context training.

We hope ROLL can support rope_scaling configuration, specifically including:

Training Stage (Trainer): Support passing rope_scaling configuration when loading the model
Inference Stage (Rollout/vLLM/SGLang): Support corresponding RoPE scaling configuration during generation

[PanAndy]

inference strategy has already support RoPE config, vllm like

    strategy_name: vllm
    strategy_config:
      max_model_len: 65536
      gpu_memory_utilization: 0.8
      block_size: 16
      # load_format: auto
      tensor_parallel_size: 8
      rope_scaling:
        rope_type: yarn
        factor: 2.0
        original_max_position_embeddings: 32768

#274

[dawson-chen]

Okey, thank you for relay.

Please also support rope scaling during trainning, including both megatron and deepspeed.