Module 4: LangCache

---

What you're building

Users ask the same questions over and over. LangCache caches responses by meaning, so similar questions return instantly without calling the LLM. This can save up to 90% on token costs.

"What are NVIDIA's margins?" and "Show me NVIDIA's gross profit trends" hit the same cache entry. LangCache matches by meaning, not exact text.

---

Cloud setup

1. Create a LangCache instance

1. In the Redis Cloud console, find LangCache in the left sidebar.

2. Click Quick Create.

3. Name your cache (e.g. iris-workshop) and click Create.

2. Save your service key

Copy the service key immediately. You won't see it again.

Save it in your .env file:

LANGCACHE_API_KEY=<your-service-key>

3. Save your Host and Cache ID

Find the remaining credentials on the instance details page:

Save them in your .env file:

LANGCACHE_HOST=<URL List>
LANGCACHE_CACHE_ID=<cache-id>

4. Seed the cache

macOS / LinuxWindows (PowerShell)

make seed-langcache

.\workshop.ps1 seed-langcache

This loads one cached response so you can test a cache HIT right away.

---

Exercise

1. Open exercises/finance/langcache.py in your IDE.

2. Tell your agent how to search the cache. Replace return None with:

   return {
       "prompt": prompt,
       "similarityThreshold": ___,       # pick a number between 0 and 1
       "searchStrategies": ["semantic"],
   }
   

   similarityThreshold ranges from 0 to 1:

   • 0 loosely related questions match
   • 0.5 similar questions match
   • 1 only exact matches count

3. Pick your similarity threshold. Replace ___ with a value between 0 and 1. You can always change it later to see the difference.

Show solution

return {
    "prompt": prompt,
    "similarityThreshold": 0.82,
    "searchStrategies": ["semantic"],
}

---

Verify

Stop the running server (Ctrl+C) and restart:

macOS / LinuxWindows (PowerShell)

make dev

.\workshop.ps1 dev

Open localhost:3040.

• Ask: "How do NVIDIA, AMD, and Broadcom gross margins compare recently?"
• Click Langcache in the activity panel. You should see HIT.
• Ask: "Walk me through Oracle's latest quarter" You should see MISS.

---

Before moving on

Set your similarityThreshold to 0.82 before starting Module 5. The remaining modules are tuned to this value.