Why tiny late interaction models win

A 150M parameter late interaction model from LightOn outperforms models up to 8B parameters on the BrowseComp benchmark, which tests complex research-style queries. Late interaction models win by learning query and document representations separately, then computing token-level interactions only at the final step (MaxSim). This Show more