OCR

Extract text from images

Agents can read text from images – receipts, documents, signs, and screenshots.

Quick Start

1. Simple Usage

   import { Agent } from 'praisonai';
   
   const agent = new Agent({
     instructions: 'Extract all text from images',
     llm: 'gpt-4o'  // Vision-capable model
   });
   
   await agent.chat([
     { role: 'user', content: [
       { type: 'text', text: 'What text is on this receipt?' },
       { type: 'image', url: 'https://example.com/receipt.jpg' }
     ]}
   ]);
   // "Store: Coffee Shop, Total: $4.50..."

2. With Configuration

   const agent = new Agent({
     instructions: 'Extract data as JSON',
     outputFormat: 'json'
   });
   
   await agent.chat([
     { role: 'user', content: [
       { type: 'text', text: 'Extract items and prices from this receipt' },
       { type: 'image', path: './receipt.jpg' }
     ]}
   ]);
   // { items: [...], total: 4.50, date: '...' }

User Interaction Flow

Configuration Levels

// Level 1: Bool - Enable with vision model
const agent = new Agent({
  llm: 'gpt-4o',
  vision: true
});

// Level 2: String - High detail for small text
const agent = new Agent({
  llm: 'gpt-4o',
  vision: 'high'
});

// Level 3: Dict - Full options
const agent = new Agent({
  vision: {
    detail: 'high',
    ocr: true,
    language: 'en'
  }
});

Common Uses

Dedicated OCRAgent class

OCRAgent calls a real OCR backend through a required extractor function.

import { OCRAgent, type OCRExtractor } from 'praisonai';

const mistralExtractor: OCRExtractor = async ({ source, document, model, options, apiKey, baseUrl }) => {
  // Call your OCR backend (Mistral OCR, Tesseract, Cloud Vision, ...)
  // Return { text, pages: [{ index, markdown }], images?, metadata? }
  return { text: '...', pages: [{ index: 0, markdown: '...' }] };
};

const agent = new OCRAgent({
  name: 'Reader',
  extractor: mistralExtractor,
  model: 'mistral/mistral-ocr-latest',
});

const result = await agent.extract('receipt.jpg');
const textOnly = await agent.read('receipt.jpg');

OCRAgentConfig

OCRConfig

OCRExtractRequest

The extractor receives this request:

| Resolved OCR options | | baseUrl | string | Custom API endpoint URL, if configured | | apiKey | string` | API key, if configured |

OCRResult

The extractor returns this shape:

API Reference

Best Practices

Related