a safetensors directory; or
If you are creating a model from a safetensors directory or from a GGUF file, you must create a blob for each of the files and then use the file name and SHA256 digest associated with each blob in the files
field.Parameters#
model
: name of the model to create
from
: (optional) name of an existing model to create the new model from
files
: (optional) a dictionary of file names to SHA256 digests of blobs to create the model from
adapters
: (optional) a dictionary of file names to SHA256 digests of blobs for LORA adapters
template
: (optional) the prompt template for the model
license
: (optional) a string or list of strings containing the license or licenses for the model
system
: (optional) a string containing the system prompt for the model
parameters
: (optional) a dictionary of parameters for the model (see Modelfile for a list of parameters)messages
: (optional) a list of message objects used to create a conversation
stream
: (optional) if false
the response will be returned as a single response object, rather than a stream of objects
quantize
(optional): quantize a non-quantized (e.g. float16) model
Quantization types#
Type | Recommended |
---|
q2_K | |
q3_K_L | |
q3_K_M | |
q3_K_S | |
q4_0 | |
q4_1 | |
q4_K_M | * |
q4_K_S | |
q5_0 | |
q5_1 | |
q5_K_M | |
q5_K_S | |
q6_K | |
q8_0 | * |
Modified at 2025-03-14 08:17:25