Repository navigation
fix(anthropic): preserve streaming input and cache usage - #1498
Open
pcbeingused333 wants to merge 1 commit into
Open
pcbeingused333 wants to merge 1 commit into
pcbeingused333 wants to merge 1 commit into
Conversation
Contributor
Qodo reviews are paused for this user.Troubleshooting steps vary by plan Learn more → On a Teams plan? Using GitHub Enterprise Server, GitLab Self-Managed, or Bitbucket Data Center? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Anthropic streaming replaces the saved usage with each event's usage. The initial message contains input and cache counts, while later message deltas contain cumulative output counts. Replacing the object loses the input and cache counts, and because the final
message_stopevent carries no usage, streamed Anthropic completions currently report 0 for every token count passed toAfterGenerated. This is a sibling of #1486; there is no linked issue.Preserve the initial usage object and update its output count when later usage arrives. Events without usage retain the accumulated statistics. Output counts are assigned rather than summed because Anthropic reports cumulative output usage.
AnthropicStreamedUsageTestscovers initial input/cache counts, text events without usage, successive output counts in both supported usage locations, the final message-stop event, an output-only stream and a stream with no usage. Run withdotnet test tests/BotSharp.Core.UnitTests/BotSharp.Core.UnitTests.csproj --filter FullyQualifiedName~AnthropicStreamedUsageTests.This change was prepared with the assistance of an AI coding tool; the author is responsible for the submission.