Source: GPTCache usage documentation
GPTCache's library docs: Config similarity_threshold, temperature-driven cache skipping, cache_skip, sessions, and pluggable scalar/vector stores with LRU or FIFO eviction.
Contents
- What: official library documentation (“Usage”)
- Publisher: Zilliz (GPTCache project)
- Date: accessed 2026-11-14 (project largely unchanged since 2023-2024)
- Original: gptcache.readthedocs.io usage
Summary
How to configure the GPTCache library that the GPTCache paper describes: it wraps an OpenAI-style client and inserts cache lookup and write-back around the call.
Key claims
Config(similarity_threshold=...)sets how close embeddings must be for a hit; the page gives no default or range (its example uses 1).temperature(0-2, default 0) doubles as a cache knob: the higher it is, the more likely the cache lookup is skipped; at 2 it is always skipped. The page says the default post-processor istemperature_softmax, but theCache.initsignature it shows listspost_process_messages_func=first, so check the installed version.cache_skip=Truebypasses lookup but still writes the fresh answer to the cache.Sessionobjects let you scope and apply rules to hits per conversation.- Data manager = scalar store (SQLite, MySQL, PostgreSQL) + vector store (FAISS, Milvus) with
max_sizeandeviction='LRU'|'FIFO'by entry count. - The docs are inconsistent on the evaluator argument name (
similarity_evaluationvsevaluation_func), so check the installed version.