Skip to content

feat(runner): implement chunked prefill to eliminate decode blocking during continuous batching - #2473

Open
presentyourlove wants to merge 1 commit into
exo-explore:mainfrom
presentyourlove:feature/chunked-prefill-continuous-batching
Open

feat(runner): implement chunked prefill to eliminate decode blocking during continuous batching#2473
presentyourlove wants to merge 1 commit into
exo-explore:mainfrom
presentyourlove:feature/chunked-prefill-continuous-batching

feat(runner): implement chunked prefill continuous batching (TODO #7)

ea0b789
Select commit
Loading
Failed to load commit list.

Workflow runs completed with no jobs