NLP

We propose NextLevelBERT, a Masked Language Model that operates on text-embedding representations of whole chunks instead of tokens, and show it handles long-document tasks effectively while outperforming much larger embedding models.

We introduce SCAN, a novel analogy dataset spanning dissimilar domains, and show that state-of-the-art pretrained language models struggle with complex analogical reasoning.