Contextual embedding models encode whole documents before chunk pooling
Original titleRetrieval systems often split long documents into chunks, but this strips away the surrounding context.
AISummary
Retrieval systems that split long documents into chunks lose surrounding context. Contextual embedding models address this by encoding the entire document once and pooling chunk vectors afterward. They are usually trained using one gold chunk per query.
Source: Perplexity · x.comPublished · added here