Praison AI

vLLM Embeddings

Generate embeddings using self-hosted vLLM server

Overview

vLLM provides high-throughput embedding inference for self-hosted deployments.

Quick Start

python
from praisonaiagents import embedding

result = embedding(
    input="Hello world",
    model="hosted_vllm/intfloat/e5-mistral-7b-instruct",
    api_base="http://localhost:8000"
)
print(f"Dimensions: {len(result.embeddings[0])}")

CLI Usage

bash
praisonai embed "Hello world" --model hosted_vllm/intfloat/e5-mistral-7b-instruct

Setup

  1. Start vLLM server with embedding model:
bash
python -m vllm.entrypoints.openai.api_server \
    --model intfloat/e5-mistral-7b-instruct \
    --task embed
  1. Set environment variable:
bash
export HOSTED_VLLM_API_BASE="http://localhost:8000"

Related