Skip to content

fix: Litellm preserve streamed reasoning deltas in LiteLLM adapter#4952

Open
lorenzbaraldi wants to merge 3 commits intogoogle:mainfrom
lorenzbaraldi:fix/reasoning-accumulation
Open

fix: Litellm preserve streamed reasoning deltas in LiteLLM adapter#4952
lorenzbaraldi wants to merge 3 commits intogoogle:mainfrom
lorenzbaraldi:fix/reasoning-accumulation

Conversation

@lorenzbaraldi
Copy link

Please ensure you have read the contribution guide before creating a pull request.

Link to Issue or Description of Change

1. Link to an existing issue (if applicable):

2. Or, if no issue exists, describe the change:

No existing issue linked for this fix.

Problem:
In LiteLlm message conversion, reasoning parts were combined with newline
injection:

reasoning_content = _NEW_LINE.join(text for text in reasoning_texts if text)

For providers that stream reasoning in delta fragments (for example, vLLM-style
reasoning chunks), this mutates the original stream by inserting extra
separators. The reconstructed reasoning can differ compared to provider
output.

Solution:
Preserve reasoning text exactly as streamed by concatenating fragments without
adding separators:

reasoning_content = "".join(text for text in reasoning_texts if text)

This avoids corruption of chunked reasoning while still preserving explicit
newlines already present in fragments.

Also added targeted regression tests to lock behavior:

  • test_content_to_message_param_preserves_chunked_reasoning_deltas
  • test_content_to_message_param_preserves_reasoning_newlines

Testing Plan

Unit Tests:

  • I have added or updated unit tests for my change.
  • All unit tests pass locally.

Summary of local pytest runs:

  1. python -m pytest tests/unittests/models/test_litellm.py -k "content_to_message_param_assistant_thought_and_content_message or preserves_chunked_reasoning_deltas or preserves_reasoning_newlines"
  • Result: 3 passed, 244 deselected
  1. python -m pytest tests/unittests/models/test_litellm.py -k "preserves_chunked_reasoning_deltas or preserves_reasoning_newlines"
  • Result: 2 passed, 245 deselected

Manual End-to-End (E2E) Tests:

Checklist

  • I have read the CONTRIBUTING.md document.
  • I have performed a self-review of my own code.
  • I have commented my code, particularly in hard-to-understand areas.
  • I have added tests that prove my fix is effective or that my feature works.
  • New and existing unit tests pass locally with my changes.
  • I have manually tested my changes end-to-end.
  • Any dependent changes have been merged and published in downstream modules.

Additional context

Scope is intentionally minimal and low risk:

  • 1-line behavior change in reasoning-content reconstruction.
  • 2 regression tests added.
  • Anthropic thinking_blocks path is unchanged.

@adk-bot adk-bot added the models [Component] Issues related to model support label Mar 22, 2026
@rohityan rohityan self-assigned this Mar 23, 2026
@rohityan rohityan added the request clarification [Status] The maintainer need clarification or more information from the author label Mar 23, 2026
@rohityan
Copy link
Collaborator

Hi @lorenzbaraldi, Thank you for your contribution! We appreciate you taking the time to submit this pull request. Can you please fix the formatting errors. You can use autoformat.sh

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

models [Component] Issues related to model support request clarification [Status] The maintainer need clarification or more information from the author

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants