Praison AI

Tokenize Dataset • AI Agents Framework

tokenize_dataset: API reference for TrainModel.tokenize_dataset

tokenize_dataset

Method

</div>

This is a method of the TrainModel class in the train module.

Signature

python
def tokenize_dataset(dataset: Any) -> Any

Parameters

ParamField Any

No description available.

Returns

ResponseField Any

The result of the operation.

Uses

  • tokenize_function
  • tokenized_dataset.remove_columns

Used By

Source

View on GitHub

praisonai/train.py at line 195