Kimi Delta Attention's recency bias shapes how models retain earlier information
AIModels using Kimi Delta Attention show a recency bias, as the mechanism controlling how much earlier information is kept shapes memory. As new information mixes into that memory, nearby words share more information, which gives the model clues about how far apart words are.










