Skip to main content
praisonai-train fine-tunes vision-language models on image-caption datasets, auto-selecting the vision trainer for any model whose name contains vision, -vl-, or visionmodel.

​
Quick Start

1

Write a vision config

Point model_name at a vision model and list an image-caption dataset. A plain config trains locally and publishes nothing.
2

Train

​
How It Works

The vision trainer converts each image-caption row into a chat conversation, attaches vision-specific LoRA adapters via Unsloth’s FastVisionModel, and trains with TRL’s SFTTrainer.
Config now honoured (PraisonAI #4879). Before #4879 the vision trainer ignored every lora_* / random_state / use_rslora key and always trained at the hardcoded literals r=16, lora_alpha=16, lora_dropout=0, random_state=3407, use_rslora=False. It now reads them from config with those literals as defaults, so an unchanged config produces an identical run.

​
Configuration Options

These keys are read by the vision trainer. Defaults match the source so an omitted key trains as before.

​
LoRA & adapter targeting

​
Run control & publishing

Publishing is opt-in (PraisonAI #4879). A vision config that omits huggingface_save used to push to the Hub — or crash with KeyError: 'hf_model_name' after training completed. Publishing now runs only when the flag and its target are both set, matching the LLM trainer. See Hub Privacy for repo visibility.

​
Common Patterns

Train locally, publish nothing — omit every publish key:
Train and publish a private LoRA — set the flag, its target, and keep the default private visibility:
Override the LoRA rank — now honoured on the vision path:

​
Best Practices

Omit huggingface_save / huggingface_save_gguf / ollama_save entirely for local experiments — no target, no push, no crash. Set the flag and its target only when publishing.
Drop the lora_* keys to keep the r=16, lora_alpha=16, lora_dropout=0 defaults, or set them deliberately — as of #4879 they take effect on the vision path instead of being ignored.
vision_instruction becomes the user turn for every sample. Set it to describe the task your captions answer, so the fine-tune learns the right response style.
Both train: true and train: "true" work in vision configs from #4879 — the old code crashed calling .lower() on a real boolean.

Training Reference

Full config reference for the LLM and vision trainers.

Hub Privacy

Private-by-default Hub pushes and visibility options.