Skip to main content
Summarise clips, detect scenes, and understand video content with a single Agent backed by a vision model. 
from praisonaiagents import Agent, Task, AgentTeam

agent = Agent(
    name="VideoAnalyst",
    instructions="Describe video content in detail.",
    llm="gpt-4o-mini",
)

task = Task(
    description="Analyze this video and summarize key events",
    expected_output="Video analysis",
    agent=agent,
    images=["video.mp4"],
)

AgentTeam(agents=[agent], tasks=[task]).start()
Video analysis agent using vision models for content understanding.

​
Quick Start

1

Simple Usage

Pass a video file to a task for analysis.
from praisonaiagents import Agent, Task, AgentTeam

agent = Agent(
    name="VideoAnalyst",
    instructions="Describe video content in detail.",
    llm="gpt-4o-mini",
)

task = Task(
    description="Analyze this video and summarize key events",
    expected_output="Video analysis",
    agent=agent,
    images=["video.mp4"],
)

AgentTeam(agents=[agent], tasks=[task]).start()
2

With Configuration

Return structured fields with a Pydantic schema.
from praisonaiagents import Agent, Task, AgentTeam
from pydantic import BaseModel

class VideoAnalysis(BaseModel):
    scenes: list[str]
    key_events: list[str]
    summary: str

agent = Agent(
    name="VideoAnalyst",
    instructions="Analyze videos and return structured results.",
    llm="gpt-4o-mini",
)

task = Task(
    description="Analyze this video in detail",
    expected_output="Structured video analysis",
    agent=agent,
    images=["video.mp4"],
    output_pydantic=VideoAnalysis,
)

AgentTeam(agents=[agent], tasks=[task]).start()

​
How It Works

​
Simple

Agents: 1 — Single agent with vision capabilities analyzes video content.

​
Workflow

  1. Receive video file
  2. Process frames with vision model
  3. Generate comprehensive analysis

​
Setup

pip install praisonaiagents praisonai
export OPENAI_API_KEY="your-key"

​
Run — Python

from praisonaiagents import Agent, Task, AgentTeam

agent = Agent(
    name="VideoAnalyst",
    instructions="Describe video content in detail.",
    llm="gpt-4o-mini"
)

task = Task(
    description="Analyze this video and summarize key events",
    expected_output="Video analysis",
    agent=agent,
    images=["video.mp4"]
)

agents = AgentTeam(agents=[agent], tasks=[task])
result = agents.start()
print(result)

​
Run — CLI

praisonai "Summarize this video" --image video.mp4

​
Run — agents.yaml

framework: praisonai
topic: Video Analysis
roles:
  video_analyst:
    role: Video Analysis Specialist
    goal: Analyze videos and describe content
    backstory: You are an expert in video analysis
    llm: gpt-4o-mini
    tasks:
      analyze:
        description: Analyze this video and summarize key events
        expected_output: Video analysis
        images:
          - video.mp4

praisonai agents.yaml

​
Serve API

from praisonaiagents import Agent

agent = Agent(
    name="VideoAnalyst",
    instructions="You are a video analysis expert.",
    llm="gpt-4o-mini"
)

agent.launch(port=8080)

curl -X POST http://localhost:8080/chat \
  -H "Content-Type: application/json" \
  -d '{"message": "Analyze this video: https://example.com/video.mp4"}'

​
Advanced Workflow (All Features)

Agents: 1 — Single agent with memory, persistence, structured output, and session resumability.

​
Workflow

  1. Initialize session for video tracking
  2. Configure SQLite persistence for analysis history
  3. Analyze video with structured output
  4. Store results in memory for comparison
  5. Resume session for follow-up analysis

​
Setup

pip install praisonaiagents praisonai pydantic
export OPENAI_API_KEY="your-key"

​
Run — Python

from praisonaiagents import Agent, Task, AgentTeam, Session
from pydantic import BaseModel

class VideoAnalysis(BaseModel):
    duration: str
    scenes: list[str]
    key_events: list[str]
    summary: str

session = Session(session_id="video-001", user_id="user-1")

agent = Agent(
    name="VideoAnalyst",
    instructions="Analyze videos and return structured results.",
    llm="gpt-4o-mini",
    memory=True
)

task = Task(
    description="Analyze this video in detail",
    expected_output="Structured video analysis",
    agent=agent,
    images=["video.mp4"],
    output_pydantic=VideoAnalysis
)

agents = AgentTeam(
    agents=[agent],
    tasks=[task],
    memory=True
)

result = agents.start()
print(result)

​
Run — CLI

praisonai "Analyze this video" --image video.mp4 --memory --verbose

​
Run — agents.yaml

framework: praisonai
topic: Video Analysis
memory: true
memory_config:
  provider: sqlite
  db_path: videos.db
roles:
  video_analyst:
    role: Video Analysis Specialist
    goal: Analyze videos with structured output
    backstory: You are an expert in video analysis
    llm: gpt-4o-mini
    memory: true
    tasks:
      analyze:
        description: Analyze this video in detail
        expected_output: Structured video analysis
        images:
          - video.mp4
        output_json:
          duration: string
          scenes: array
          key_events: array
          summary: string

praisonai agents.yaml --verbose

​
Serve API

from praisonaiagents import Agent

agent = Agent(
    name="VideoAnalyst",
    instructions="Analyze videos and return structured results.",
    llm="gpt-4o-mini",
    memory=True
)

agent.launch(port=8080)

curl -X POST http://localhost:8080/chat \
  -H "Content-Type: application/json" \
  -d '{"message": "Analyze video", "session_id": "video-001"}'

​
Monitor / Verify

praisonai "test video" --image test.mp4 --verbose

​
Cleanup

rm -f videos.db

​
Features Demonstrated

FeatureImplementation
WorkflowVision-based video analysis
DB PersistenceSQLite via memory_config
Observability--verbose flag
ResumabilitySession with session_id
Structured OutputPydantic VideoAnalysis model

​
Best Practices

Vision models read media from the images=[...] field on a Task. A file path in the prompt text is invisible to the model.
Set llm="gpt-4o-mini" or another multimodal model. Text-only models cannot process video frames.
Long videos dilute detail. Analyse in segments and combine results when you need frame-level accuracy across a full recording.
Add output_pydantic with scenes and key_events so a search index or timeline UI can consume the output directly.

Analyze still images frame by frame.

Extract on-screen text with OCR.