Source: MeanCache, user-centric semantic caching (Gill et al., 2024)
A client-side semantic cache that stores conversation context with each query to avoid false hits on follow-ups, and trains its similarity model with federated learning so raw queries never leave the user; IPDPS 2025.
Contents
- What: research paper, arXiv 2403.02694 (v4, 2025-03-07), IEEE IPDPS 2025
- Authors: Waris Gill, Mohamed Elidrisi, Pallavi Kalapatapu, Ammar Ahmed, Ali Anwar, Muhammad Ali Gulzar (Virginia Tech, Cisco, University of Minnesota)
- Date: 2024-03-05
- Original: arXiv:2403.02694
Summary
Puts the semantic cache on the user’s device rather than the server, so cached queries stay private, and trains the query-similarity model across users with federated learning. Each cached query is stored with its context chain so a follow-up like “and in Python?” is not matched to an unrelated earlier question with the same words.
Key claims
- About 31% of queries in their study were repeats, motivating caching.
- Existing caches mishandle contextual (follow-up) queries, giving unacceptable false-hit and false-miss rates.
- Against a state-of-the-art semantic cache baseline: about 17% higher F-score and 20% higher precision on hit/miss decisions, more on contextual queries; 83% less storage; 11% faster decisions.