Directories
¶
Path | Synopsis |
---|---|
cmd
|
|
epp
command
Package main contains the "Endpoint Picker (EPP)" program for scheduling inference requests.
|
Package main contains the "Endpoint Picker (EPP)" program for scheduling inference requests. |
pkg
|
|
plugins
Package plugins provides plugins for the scheduler.
|
Package plugins provides plugins for the scheduler. |
plugins/filter
Package filter provides filter plugins for the epp.
|
Package filter provides filter plugins for the epp. |
plugins/pre-request
Package prerequest provides pre-request plugins for GIE.
|
Package prerequest provides pre-request plugins for GIE. |
plugins/profile
Package profile provides profile handler plugin for the epp.
|
Package profile provides profile handler plugin for the epp. |
plugins/scorer
Package scorer provides scorer plugins for the scheduler.
|
Package scorer provides scorer plugins for the scheduler. |
scheduling/pd
Package pd implements disaggregated Prefill/Decode scheduling
|
Package pd implements disaggregated Prefill/Decode scheduling |
Click to show internal directories.
Click to hide internal directories.