Export a trained model

Publish an already-trained model to Hugging Face, GGUF, or Ollama without re-running training

Publish a model you already trained to Hugging Face, GGUF, or Ollama — no re-training, one command.

Quick Start

You already have a trained model in lora_model/. Pick a target and publish.

1. Publish to Hugging Face

   Merge and push FP16 weights to a Hub repo.

   praisonai-train export hf --model-dir lora_model --hf me/my-model

2. Publish a GGUF build

   Write a local .gguf — add --hf to also push it to the Hub.

   # Local .gguf only
   praisonai-train export gguf --model-dir lora_model --quant q4_k_m
   
   # Local .gguf + push to the Hub
   praisonai-train export gguf --model-dir lora_model --hf me/my-model --quant q4_k_m

3. Publish to Ollama

   Create and push a quantized model to ollama.com, or keep it on your machine with --no-push.

   # Push to ollama.com (default)
   praisonai-train export ollama --model-dir lora_model --ollama me/my-model --quant q4_k_m
   
   # Create locally only — do not upload to ollama.com
   praisonai-train export ollama --model-dir lora_model --ollama my-model --quant q4_k_m --no-push

When to use this

You already trained a model (e.g. lora_model/ exists) and don’t want to spend an hour re-running training just to publish it to a different target, fix a chat_template, or try a different quantization. export loads the trained weights once and publishes them.

How It Works

Export skips the training-only preflight, so it runs on CPU — there’s nothing to fine-tune. The base model name drives chat-template selection so the exported model isn’t garbled.

CLI flags

How the base model is inferred

The base model name selects the chat template, so the exported model formats prompts correctly. Resolution order:

1. --base-model if passed.
2. else model_name in --config.
3. else _name_or_path from <model-dir>/config.json.
4. else the model-dir name itself.

Prerequisites

export HF_TOKEN=hf_...
# or
huggingface-cli login

Valid --quant values

Any of these 35 quantization methods (from VALID_QUANTIZATION_METHODS — 24 standard plus 11 imatrix).

Standard (24)

bf16  f16  f32  fast_quantized  not_quantized  quantized
q2_k  q2_k_l  q3_k_l  q3_k_m  q3_k_s  q3_k_xs
q4_0  q4_1  q4_k  q4_k_m  q4_k_s
q5_0  q5_1  q5_k  q5_k_m  q5_k_s
q6_k  q8_0

IMatrix — large-model / laptop-fit (11)

Importance-matrix quants let a 30B model fit on a laptop. Unsloth resolves the matrix itself, downloading it from unsloth/<base>-GGUF when none is supplied.

iq1_m  iq1_s  iq2_m  iq2_s  iq2_xs  iq2_xxs
iq3_m  iq3_s  iq3_xxs  iq4_nl  iq4_xs

See Unsloth Surface for the “which quant should I pick” decision guide.

Failure modes → clean errors

Every known failure prints a single actionable message instead of a raw traceback.

Keep an Ollama export local

Trained on private data? Pass --no-push (or set ollama_push: false in config.yaml). The command still runs ollama create and prints the local tag, so ollama run <tag> works immediately — nothing is uploaded to ollama.com.

praisonai-train export ollama --model-dir lora_model --ollama my-model --no-push
# Created local Ollama model `my-model:latest` (not pushed). Run it with: ollama run my-model:latest

Precedence: --push / --no-push on the CLI > ollama_push in config.yaml > default (push).

Python API

Script exports with the TrainModel.for_export() classmethod — it builds an export-only trainer (no dataset required) and validates the quantization up front.

from praisonai_train.train.llm.trainer import TrainModel

trainer = TrainModel.for_export({
    "final_model_dir": "lora_model",
    "model_name": "unsloth/gemma-2-2b-it-bnb-4bit",
    "hf_model_name": "me/my-model",
    "quantization_method": "q4_k_m",
})
model, tokenizer = trainer.load_model()
trainer.model, trainer.hf_tokenizer = model, tokenizer

trainer.save_model_merged()      # Hugging Face merged 16-bit
# trainer.save_model_gguf()      # local GGUF
# trainer.push_model_gguf()      # push GGUF to the Hub
# trainer.create_and_push_ollama_model()  # Ollama (add "ollama_push": False to keep it local)

Hub push options

These keys flow through config.yaml (not CLI flags today) into every push_to_hub_merged / push_to_hub_gguf call. Pass them in the --config file or the for_export({...}) dict.

Best Practices

Related