Class ModelManager

constructor

• new ModelManager(params?: ModelManagerParams): ModelManager

• Parameters

  • params: ModelManagerParams = {}

  Returns ModelManager

cacheManager

logger

params

clear

• clear(): Promise<void>

• Remove all models from the cache

  Returns Promise<void>

downloadModel

• downloadModel(url: string, options?: DownloadOptions): Promise<Model>

• Download a model from the given URL.

  The URL must end with .gguf

  Parameters

  • url: string
  • options: DownloadOptions = {}

  Returns Promise<Model>

getModelOrDownload

• getModelOrDownload(url: string, options?: DownloadOptions): Promise<Model>

• Get a model from the cache or download it if it's not available.

  Parameters

  • url: string
  • options: DownloadOptions = {}

  Returns Promise<Model>

getModels

• getModels(opts?: { includeInvalid?: boolean }): Promise<Model[]>

• Get all models in the cache

  Parameters

  • opts: { includeInvalid?: boolean } = {}

  Returns Promise<Model[]>

StaticparseModelUrl

• parseModelUrl(modelUrl: string | string[]): string[]

• Parses a model URL and returns an array of URLs based on the following patterns:

  • If the input URL is an array, it returns the array itself.
  • If the input URL is a string in the gguf-split format, it returns an array containing the URL of each shard in ascending order.
  • Otherwise, it returns an array containing the input URL as a single element array.

  Parameters

  • modelUrl: string | string[]

    URL or list of URLs

  Returns string[]