Semantic Caching for LLM Agents with Redis LangCache

Where this fits

You're building an agent that repeatedly asks an LLM similar questions—maybe a support bot, a research assistant, or a code reviewer. Each call costs money and adds latency, even when the answer barely changes. The redis-semantic-cache skill shows you how to use Redis Cloud's LangCache to store embeddings of prompts and return cached responses for semantically similar queries, cutting both cost and response time without sacrificing quality.

Why agents benefit

• Slash API spend: Cache responses for repeated or slightly rephrased prompts, reducing the number of calls to OpenAI, Anthropic, or other providers.
• Cut latency: Serve cached answers in milliseconds instead of waiting on model inference, making your agent feel snappier.
• Tune precision vs. hit rate: Adjust the similarity threshold to control how aggressively you cache—higher thresholds give more exact matches, lower ones catch more variations but risk stale answers.
• Isolate workloads: Separate caches per task type (e.g., summarization vs. classification) to avoid cross-contamination and keep each cache tuned appropriately.
• Filter with custom attributes: Use metadata like user ID, version, or content type to further refine what gets cached and retrieved, so you can serve personalized or context-specific results.

Practical scenarios

RAG answer caching

Your agent retrieves documents and synthesizes an answer. Users often ask the same question in different ways. With semantic caching, you store the final answer keyed by the embedding of the full prompt (including retrieved context). A similar follow-up question returns the cached answer instantly, skipping both retrieval and generation.

Classification and extraction

For deterministic tasks like sentiment analysis or entity extraction, the output is usually stable. Cache the classification result for a given input text. When a user rephrases the same text, the semantic cache returns the same label without calling the model again.

Multi-tenant agent apps

If you run one agent for many clients, each with distinct data or preferences, you can create separate LangCache caches per tenant. The skill explains how to set up and filter caches using custom attributes, so you never mix responses across contexts.

Add it to your agent workflow

Install the skill via npx:

npx skills add redis/agent-skills --skill redis-semantic-cache

Then follow the README to configure your Redis Cloud instance and wrap your LLM calls. A minimal example using the SDK:

from redis import Redis
from redis.commands.search.query import Query

# Connect to your Redis Cloud with LangCache enabled
r = Redis.from_url("redis://...")

# Cache check before calling the model
cached = r.ft("langcache").search(
    Query("@embedding:[...]").dialect(2)
).docs
if cached:
    return cached[0].response
else:
    response = call_llm(prompt)
    r.ft("langcache").add(prompt_embedding, response)
    return response

Compare before adopting

Before you commit, compare semantic caching against simpler alternatives:

• Exact-match caching: Cheaper and simpler, but misses rephrased queries.
• Embedding-based custom solutions: You can build your own with Redis Stack, but LangCache handles indexing and search for you.
• No caching: If your workload is highly diverse, the overhead of embeddings might not pay off.

Also consider freshness: cached responses can become stale if your underlying data or model changes. The skill's guidance on tuning thresholds and separating caches helps mitigate that.

Why it is worth tracking

The skill comes from Redis, the company behind the ecosystem, and has 140 stars on GitHub—a healthy signal. LangCache is in preview, so expect API changes, but the core approach is sound. If you're building agents that make repeated LLM calls, this skill is worth evaluating now to understand the cost-saving potential. Check the OpenAgentSkill page for updates and community feedback.