pycognaize.common.langchain_loader.LangchainLoader

class LangchainLoader(**kwargs)[source]

Bases: object

Convert Pycognaize Document Object to Langchain Document Object

Methods

count_tokens

Tokenize the text and count the number of tokens

get_document_src

Get the SHA of the document

load_and_split

Attributes

INPUT_FIELDS

LIMIT

OVERLAP

count_tokens(text)[source]

Tokenize the text and count the number of tokens

Parameters:

text (str)

Return type:

int

get_document_src()[source]

Get the SHA of the document

Return type:

str