Long-Context Retrieval Models with Monarch Mixer
We could have retrieved embeddings for long documents – instead, we’ve retrieved the vectors only for each single word.