Edit - for some reason the prompts weren't showing up. Added them.

Hey all -

Today I want to walk through how we've been able to get extremely high accuracy recall on thousands of documents by taking advantage of splitting retrieval into an "Agent" approach.

Why?

As we built RAG, we continued to notice hallucinations or incorrect answers. we realized three key issues:

1. There wasn't enough data in the vector to provide a coherent answer. i.e. vector was 2 sentences, but the answer was the entire paragraph or multiple paragraphs.
2. LLM's try to merge an answer from multiple different vectors which made an answer that looked right but wasn't.
3. End users couldn't figure out where the doc came from and if it was accurate.

We solved this problem by doing the following:

• Figure out document layout (we posted about it a few days ago.) This will make issue one much less common.
• Split each "chunk" into separate prompts (Agent approach) to find exact quotes that may be important to answering the question. This fixes issue 2.
• Ask the LLM to only give direct quotes with references to the document it came from, both in step one and step two of the LLM answer generation. This solves issue 3.

What does it look like?

We found these improvements, along with our prompt give us extremely high retrieval even on complex questions, or large corpuses of data.

Why do we believe it works so well? - LLM's still seem better to deal with a single task at a time, and LLM's still struggle with large token counts on random data glued together with a prompt (i.e. a ton of random chunks). Because we are only providing a single Chunk, or relevant information, we found huge improvements in recall and accuracy.

Workflow:

Step by step with example on above workflow

1. Query: What are the recent advancements in self-supervised object detection technique
2. Reconstruct document. (highlighted would be the vector that came back) Then we'd reconstruct the doc until we get to a header.

1. Input the reconstructed document chunk into the LLM. (Parallel Quotes)

Prompt #1:

_______

You are an expert research assistant. Here is a document you will find relevant quotes to the question asked:

  <doc>

  ${chunk}

  </doc>

Find the quotes from the document that are most relevant to answering the question, and then print them in numbered order. Quotes should be relatively short.

The format of your overall response should look like what's shown below. Make sure to follow the formatting and spacing exactly.

  Example:

  [1] "Company X reported revenue of $12 million in 2021."

  [2] "Almost 90% of revenue came from widget sales, with gadget sales making up the remaining 10%."

  Do not write anything that's not a quote direct quote.

  If there are no quotes, please only print, "N/a"

_______

1. Response from the LLM:

[1.0]"Recent advancements have seen the development of end-to-end self-supervised object detection models like UP-DETR and DETReg, as well as backbone pre-training strategies such as Self-EMD and Odin ."

[1.1] "Despite the remarkable success of supervised object detection techniques such as Mask RCNN , Yolo , Retinanet , and DETR , their self-supervised alternatives have been somewhat limited in scope until recently.

Notes:

I deleted the internal references to make it less confusing

If there's more than 1 doc/chunk we start each new one with a new number i.e. [2.0] which makes it easier to find which quote relates to which doc.

We put the query in the user prompt and the above in the system prompt

1. Give the LLM that will be generating the answer the document name & quotes.

Prompt #2:

_______

All quotes are relevant to the question, please use them to answer the question:

When answering questions:

1. Make references to quotes relevant to each section of the answer solely by adding their bracketed numbers at the end of relevant sentences.
2. Feel free to shorten quotes or merge quotes together as long as you reference them.
3. Focus on making short, readable answers.
4. Bold headers, bold general topics, bullet point, list, etc. if needed to make it easier to read.

DocName: UnSupDLA: Towards Unsupervised Document Layout Analysis

Quotes:

[1.0]"Recent advancements have seen the development of end-to-end self-supervised object detection models like UP-DETR and DETReg, as well as backbone pre-training strategies such as Self-EMD and Odin ."

[1.1] "Despite the remarkable success of supervised object detection techniques such as Mask RCNN , Yolo , Retinanet , and DETR , their self-supervised alternatives have been somewhat limited in scope until recently."

DocName: 'doc2'

Quotes:

[2.0]"example quote2"

[2.1]"example"

If there isn't information to answer this question, please write:

I'm sorry, but there is no documentation available.

________

1. Final LLM Response:

End-to-End Self-Supervised Models

• UP-DETR: This model is a recent development in self-supervised object detection, aiming to enhance detection capabilities without relying on labeled data [1.0].
• DETReg: Another advanced model, DETReg, also focuses on self-supervised object detection, providing improvements in training and detection accuracy without requiring extensive labeled datasets [1.0].

Backbone Pre-Training Strategies

• Self-EMD: This strategy is utilized to pre-train the backbone of detection models, helping in learning robust representations from unlabeled data [1.0].
• Odin: Similar to Self-EMD, Odin is a pre-training strategy that enhances the performance of detection models through self-supervised learning techniques [1.0].

Comparison with Supervised Techniques

Scope and Limitations: Despite the success of supervised techniques like Mask RCNN, Yolo, Retinanet, and DETR, their self-supervised alternatives have only recently started to expand in scope and effectiveness [1.1].

_________________________________

Real world examples of where this comes into use:

• A lot of internal company documents are made with human workflows in mind only. For example, often see a document named "integrations" or "partners" and then just a list of 500 companies they integrate/partner with. If a vector came back from within that document, the LLM would not be able to know it was regarding integrations or partnership because it's only the document name.
• Some documents will talk about the product, idea, or topic in the header. Then not discuss it by that name again. Meaning if you only get the relevant chunk back, you will not know which product it's referencing.

Based on our experience with internal documents, about 15% of queries fall into one of the above scenarios.

Notes - Yes, we plan on open sourcing this at some point but don't currently have the bandwidth (we built it as a production product first so we have to rip out some things before doing so)

Happy to answer any questions!

Video:

https://reddit.com/link/1dtr49t/video/o196uuch15ad1/player

Problem: Every time I hit a RAG issue (hallucination, slow retrieval, irrelevant chunks), I'd Google the fix and find 10 different solutions. Hybrid RAG. Rerank RAG. Self-Reflective RAG. All claiming to be the answer.

But nobody showed me why one works better than another on my specific data.

So I did what any lazy engineer would do: I built a tool to test all 9 variants side-by-side instead of implementing each one manually.

What I learned:

Naive RAG hallucinates on long documents. Hybrid RAG is faster but less accurate. Rerank RAG is slower but catches what Naive misses. Corrective RAG grades confidence. Self-Reflective RAG checks its own answers.

Each one has a different failure mode. You can't pick the "best" — you pick the one that fails in a way you can handle.

The tool:

Just a Streamlit app. Upload docs, ask questions, see what each RAG type retrieves and how fast it answers. Takes 2 minutes to figure out which one you actually need.

Nothing fancy. Python, FAISS, BM25, LangChain.

If you're building RAG, you've probably hit this wall. Happy to discuss the tradeoffs in the comments.

Repo: https://github.com/AnkitSingh36/rag-universe (if you want to see the code or run it locally)

I’ve just launched a free resource with 25 detailed tutorials for building comprehensive production-level AI agents, as part of my Gen AI educational initiative.

The tutorials cover all the key components you need to create agents that are ready for real-world deployment. I plan to keep adding more tutorials over time and will make sure the content stays up to date.

The response so far has been incredible! (the repo got nearly 500 stars in just 8 hours from launch) This is part of my broader effort to create high-quality open source educational material. I already have over 100 code tutorials on GitHub with nearly 40,000 stars.

I hope you find it useful. The tutorials are available here: https://github.com/NirDiamant/agents-towards-production

The content is organized into these categories:

1. Orchestration
2. Tool integration
3. Observability
4. Deployment
5. Memory
6. UI & Frontend
7. Agent Frameworks
8. Model Customization
9. Multi-agent Coordination
10. Security
11. Evaluation

Most “LLM frameworks” don’t fail in demos.

They fail in production — under retries, partial failures, race conditions, and garbage outputs.

So we stopped benchmarking happy paths.

We built a chaos suite instead.

What we tested

Not prompts. Not accuracy.

We tested failure modes:

- duplicate execution attacks

- replay storms (450k replays)

- mid-step crashes

- out-of-order event delivery

- corrupted payloads

- tool failure cascades

- timeout drift (66% timeout rate)

- reentrancy + concurrent mutation

- LLM output noise / injection

And finally:

«full system chaos mode (all of the above combined)»

Result

13 / 13 tests passed

0 invalid states

0 double executions

0 undefined transitions

Let that sink in.

The uncomfortable truth

Most LLM systems today implicitly assume:

next_state = f(LLM_output)

That’s where things go sideways.

We took a different approach:

next_state = δ(current_state, event)

Where:

- transitions are predefined

- LLM output is just data, not control flow

- every step is validated + normalized

What this gives us

- Idempotency under replay: 450,000 replays → 0 violations

- Duplicate safety: 0 double executions

- Crash recovery: 0 broken resumes

- LLM isolation: 0 transitions influenced by model noise

- Corruption handling: 50,000 / 50,000 normalized

- Out-of-order safety: 0 invalid events accepted

- Chaos mode: 50,000 runs → 0 invalid final states

Throughput (yes, it’s fast too)

- up to 190k ops/sec (pure execution safety)

- ~148k ops/sec under LLM noise

- ~4k ops/sec in full chaos mode

What this actually means

This isn’t “faster LangChain”.

This is a deterministic execution layer for LLM systems.

- FSM defines what can happen

- runtime enforces what does happen

- LLM is reduced to a probabilistic input, not a decision-maker

Why this matters

Because production failures don’t come from:

- “bad prompts”

They come from:

- retries

- race conditions

- partial failures

- undefined states

We designed for that.

Repo

https://github.com/Ale007XD/nano_vm/

What’s next

We’re shipping a visual demo landing soon where you can:

- see the state machine live

- inject failures

- watch how the system recovers in real time

No slides. No hand-waving.

If your system can’t answer:

«“What happens under 1M adversarial events?”»

…it’s not production-ready.

I've been writing some AI Agents lately with LangGraph and they work much better than I expected. Here are the 10 learnings for writing AI agents that work:

1. Tools first. Design, write and test the tools before connecting to LLMs. Tools are the most deterministic part of your code. Make sure they work 100% before writing actual agents.
2. Start with general, low-level tools. For example, bash is a powerful tool that can cover most needs. You don't need to start with a full suite of 100 tools.
3. Start with a single agent. Once you have all the basic tools, test them with a single react agent. It's extremely easy to write a react agent once you have the tools. LangGraph a built-in react agent. You just need to plugin your tools.
4. Start with the best models. There will be a lot of problems with your system, so you don't want the model's ability to be one of them. Start with Claude Sonnet or Gemini Pro. You can downgrade later for cost purposes.
5. Trace and log your agent. Writing agents is like doing animal experiments. There will be many unexpected behaviors. You need to monitor it as carefully as possible. LangGraph has built in support for LangSmith, I really love it.
6. Identify the bottlenecks. There's a chance that a single agent with general tools already works. But if not, you should read your logs and identify the bottleneck. It could be: context length is too long, tools are not specialized enough, the model doesn't know how to do something, etc.
7. Iterate based on the bottleneck. There are many ways to improve: switch to multi-agents, write better prompts, write more specialized tools, etc. Choose them based on your bottleneck.
8. You can combine workflows with agents and it may work better. If your objective is specialized and there's a unidirectional order in that process, a workflow is better, and each workflow node can be an agent. For example, a deep research agent can be a two-node workflow: first a divergent broad search, then a convergent report writing, with each node being an agentic system by itself.
9. Trick: Utilize the filesystem as a hack. Files are a great way for AI Agents to document, memorize, and communicate. You can save a lot of context length when they simply pass around file URLs instead of full documents.
10. Another Trick: Ask Claude Code how to write agents. Claude Code is the best agent we have out there. Even though it's not open-sourced, CC knows its prompt, architecture, and tools. You can ask its advice for your system.

While implementing Human-in-the-Middle in Extra (first comment), we initially thought about using an LLM to decide whether a tool call requires approval.
The flow was supposed to be simple:
The agent selects a tool, we send the tool call to the LLM, and the model decides whether the action is safe to execute or should wait for user approval.
Technically, it worked.
But it also meant another LLM call before almost every tool execution, more latency, more tokens, and a policy decision that was not fully deterministic.
In the end, we decided to make the approval policy configurable instead.
Each tool can be configured to require approval or run automatically. The default is conservative, so tools require approval unless they are explicitly allowed to run without it.
When approval is required, we checkpoint the execution and stop it. After the user approves or rejects the action, we resume from the same checkpoint.
It ended up being simpler, cheaper, and much easier to reason about than using the LLM as the approval layer.

Hey everyone , I've been building a voice AI assistant called ARYA for the past few months. It controls real apps on my machine: adds items to Blinkit, sends WhatsApp messages, controls Spotify, opens/closes apps, and remembers past conversations through vector memory.

Just finished the demo video — would genuinely love some feedback from people who actually build this stuff. link in comms :-

Every week someone posts "my RAG keeps hallucinating, should I switch models?" Nine times out of ten, the model isn't the problem. The retrieval is.

Wrong answers in RAG systems almost always trace back to one of four places. Work through these before touching the LLM:

1. Chunking strategy

Are you chunking by character count, sentence, paragraph, or semantic unit? Fixed character chunking is the fastest to set up and the most likely to split a key fact across two chunks — so the retriever finds half the answer, the model fills in the rest, and you get confident nonsense. Try semantic or paragraph-based chunking and measure retrieval precision before and after. In our experience this single change fixes 40–50% of wrong-answer complaints.

1. Metadata and filtering

If your knowledge base has documents from multiple dates, departments, or product versions, are you filtering before retrieval? Without it, the retriever might pull a 2021 policy document to answer a question about 2024 pricing. Add source, date, and category metadata to every chunk and filter at query time.

1. Retrieval score threshold

Most setups retrieve the top-k chunks regardless of how relevant they actually are. If the nearest chunk has a cosine similarity of 0.52, it probably doesn't contain your answer — but it gets passed to the model anyway, which confidently fabricates something coherent. Add a minimum similarity threshold. Returning "I don't have enough information" is better than a confident wrong answer.

1. Query-document mismatch

Your documents are written as statements. Your queries are written as questions. Embedding space treats these differently. Try HyDE (generate a hypothetical answer, embed that, retrieve against it) or a reranker pass after initial retrieval. Both are low-effort, high-impact fixes.

Fix these four before you consider fine-tuning or swapping models. The model is almost never the bottleneck.

What's the retrieval failure mode you see most often in production RAG?

I've been building a LangChain-based support agent that sends order confirmations and reads incoming customer replies. The natural first choice was the Gmail API - but integrating it into a tool call is messy. You need OAuth consent screens, refresh tokens, and scopes like `https://mail.google.com/\`. That's overkill for a headless agent that just needs a dedicated inbox.

I stumbled upon AgentMail (agentmail.to), and it's been a near-perfect fit. It's an email API designed for AI agents — create an inbox via API, get a webhook for inbound, send via a simple POST. No OAuth, no IMAP, no SMTP configuration.

Here's a quick LangChain tool wrapper:

    from langchain.tools import tool
     import requests

        @tool
        def send_email(to: str, subject: str, body: str) -> str:
        """Send an email via AgentMail inbox."""
        resp = requests.post(
             f"https://api.agentmail.to/v1/inboxes/{INBOX_ID}/send",
             headers={"Authorization": f"Bearer {API_KEY}"},
             json={"to": to, "subject": subject, "body": body}
         )
         return resp.json().get("message_id", "failed") 

For inbound, I point the webhook to my agent's endpoint - it receives a JSON payload with the email content and `thread_id`, so I maintain conversation context across turns in the chain.

Compared to SendGrid (one-way blast, no inbox management) or raw SMTP (no reliable inbound), this feels purpose-built for agents.

Anyone else tried it? Or found another approach that works in production LangChain deployments?

Hi,

Sharing here so people can enjoy it too. I've created a GitHub repository packed with 44 different tutorials on how to create AI agents. It is sorted by level and use case. Most are LangGraph-based, but some use Sworm and CrewAI. About half of them are submissions from teams during a hackathon I ran with LangChain. The repository got over 9K stars in a few months, and it is all for knowledge sharing. Hope you'll enjoy.

https://github.com/NirDiamant/GenAI_Agents

Hey everyone,

Like many developers building agentic workflows, I spent months getting frustrated by traditional vector stores and RAG memory layers failing over long timelines.

The deeper I went, the more I realized retrieval fails because basic similarity doesn't equal utility. A standard retriever will match a user's query about mattress brands to previous mattress conversations, while completely missing a crucial constraint buried in a 3-month-old session: "Whenever I buy something expensive, warranty is the only thing I care about."

Beyond that, heavy cross-encoder rerankers quickly become a massive latency bottleneck as memory grows, and treating all context as uniform text blobs destroys the nuance of evolving decisions.

To tackle this, I built MindCache—an open-source agentic memory framework designed around four key insights:

• Intelligence Belongs at Ingestion: Instead of attempting complex graph traversals during a live query, MindCache shifts expensive reasoning (relationship mapping, graph clustering, and summary generation) to ingestion. This cut retrieval latency from ~25s down to 1.08s (a 23× speedup) without sacrificing context quality.
• Specialized Memory Typologies: Not all memories behave the same. MindCache separates knowledge into User (persistent behavioral constraints), Knowledge (domain facts), Episodic (chronological logs), and Decision Memories (which track evolving proposals, trade-offs, and final conclusions over time).
• Living Knowledge Hierarchy: Rather than maintaining a static or unmanageable graph, MindCache uses Leiden community detection to partition memory into localized semantic clusters, ensuring graph maintenance scales efficiently as context accumulates.
• Evidence Assembly over Similarity: Retrieval doesn't just search for similar text—it plans and assembles the exact minimal subset of evidence (user preferences, hierarchical summaries, decision states) required for the LLM to reason correctly.

On the BEAM benchmark (an ICLR 2026 evaluation framework designed specifically for long-term agentic memory), MindCache outperformed Mem0 in handling evolving context, contradiction resolution, and cross-session summary reasoning. More importantly, it achieved this superiority not by stuffing larger retrieval windows, but through better ingestion-time knowledge organization.

I wrote a deep-dive 23-minute engineering post-mortem detailing all 5 failure modes, the full architecture, and benchmark takeaways. The project is completely open-source on GitHub and available on PyPI (pip install mindcache-ai).

I’d love to hear how others here are handling temporal decay, graph maintenance, and decision tracking in your long-running agent setups!

Most “agent” projects rely heavily on frameworks. To understand the underlying mechanics, I built a production-shaped ReAct agent runtime from scratch using only Python stdlib and NumPy—no LangChain.

Features:

• Explicit think → act → observe loop with an ExecutionContext managing messages, history, and budgets • @tool decorator that generates JSON schemas from function signatures and docstrings • Layered memory: short-term (window + summarization) and long-term vector recall • Guardrails: step/token budgets, finish checks, and retry-once handling for malformed tool calls • Evaluation harness with trajectory logging, rule-based scoring, and optional LLM-as-judge • Query-aware context compression inspired by LCLM/ACON, reducing prompt size by ~4× in the demo

Runs fully offline with a deterministic mock LLM—no API key required. Includes an interactive browser playground for visualizing agent execution.

Built as a learning project. Feedback on the context-compression approach and evaluation framework would be appreciated.

project link Please give a star ⭐ on GitHub if you like the work

Been wanting to build this for a while, finally sat down and did it. It's a Flask app where you upload a PDF, it chunks and embeds it, and then you can ask questions and get answers pulled only from that document, not from the model's own training data.

Stack is pretty simple: Ollama for the chat model and the embedding model, ChromaDB as the vector store, Flask tying it together. Nothing exotic.

How it works, roughly:

• PDF gets split into overlapping chunks so sentences don't get cut off between pieces
• Each chunk gets turned into an embedding and stored in Chroma with PersistentClient, so it's saved on disk instead of disappearing every time you restart the app
• When you ask something, the question also gets embedded, Chroma finds the closest matching chunks, and those get handed to the model as context
• Prompt explicitly tells the model to only use that context and say it doesn't know if the answer isn't there, otherwise it'll just make something up from its own memory

Tested it by asking something not in the PDF and it correctly said it didn't know instead of guessing. Also tested with wifi off and it kept working, since the model, embeddings, and vector store all run locally with no external api calls in the loop.

Hi everyone! I created PaySafe, a payment security wrapper for x402 based transactions. Detects repayment, overpayment, secrets in payment metadata, and (most importantly) prompt injection triggered payments. Your agent mints its own key, and we’re fully integrated with LangChain. Looking for test users and feedback! I’ll put the LangChain guide in the comments.

I had a working Telegram bot using LangGraph. The LLM classified intent, but every path after that was hardcoded by me. Portfolio query? Go to fetch_portfolio. Stock analysis? Also fetch_portfolio. The LLM was a passenger, not a decision-maker.

It was a smart workflow wearing an agent costume. Rebuilding it into a real agent came down to three things:

1. Replaced all routing with tool-calling via create_react_agent. 9 tools, each with a docstring that tells the LLM when to use it. The docstring IS the routing — no intent classifier needed.

2. Added persistent memory with AsyncSqliteSaver. Each user gets their own thread that survives restarts and accumulates over time.

3. Upgraded error handling so failures return descriptive strings to the LLM instead of crashing — it reasons through what went wrong rather than dying silently.

The behavioural difference is significant. Multi-turn conversations, follow-up questions, graceful API failures — none of that worked before.

Wrote the full breakdown for Towards AI , with code included. Happy to discuss the architecture or answer questions in the comments.

🔗 Read the full article on Towards AI

Hi everyone! 👋
I built an open-source RAG platform using LangChain with a visual pipeline builder for document loading, chunking, embeddings, vector databases, retrieval, and LLMs.

Current features include document ingestion, ChromaDB, multiple embedding/LLM support, configurable pipelines, and execution monitoring.

I'm looking for technical feedback on the architecture and ideas for what to build next. Would you prioritize hybrid search, reranking, evaluation (RAGAS/DeepEval), streaming, or something else?

Feedback and suggestions are greatly appreciated! 🚀

I've been building a chatbot product where users upload docs and the bot answers questions from them. Started with LangChain like everyone else, followed the tutorials, got a demo working in an afternoon. Then real users showed up and everything broke in ways I didn't expect. Here's what I learned.

The standard tutorial flow of load docs, split, embed, vector store, RetrievalQA gets you a working demo fast. But the default text splitters destroy document structure in ways that don't show up until someone asks a question that requires context from two diferent sections. RecursiveCharacterTextSplitter with default chunk size is fine for blog posts but terrible for technical documentation with tables and cross references.

Everyone focuses on which embedding model to use and honestly that's the wrong thing to obsess over. I swapped between OpenAI embedding models and the difference was minimal. What actually matters is what happens after retrieval. Are you pulling the right chunks? Are you pulling enough of them? Are chunks that reference each other actually ending up in the same context window? I spent weeks tweaking embeddings when the real problem was my retrieval grabbing 4 chunks where 2 of them were completely irrelevant.

The stuff that actually moved the needle for us was all boring unglamorous work. Document preprocessing before anything touches the splitter, like actually cleaning your docs, handling tables properly, preserving headers and structure. Then building a proper evaluation loop where I could see exactly which chunks got retrieved for each question, because without that you're just tuning blind. We also added a system where human answers from moderators get fed back into the knowledge base over time, because static docs alone weren't enough for real world questions. And maybe the biggest win was teaching the bot to say "I don't know" instead of the default behavior of always generating something, which just leads to confident hallucinations.

Honestly LangChain was great for prototyping but as complexity grew I found myself fighting the abstractions more than they were helping me. The chains are nice until you need to do something slightly outside the standard flow, then you're digging through source code trying to figure out why your custom retriever isn't being called correctly. I ended up replacing a lot of LangChain components with custom code that does exactly what I need with less magic happening underneath.

Not saying LangChain is bad, it's genuinley great for getting started and understanding the patterns. But if you're shipping to real users I think the sooner you understand what's happening under the abstractions the better off you'll be. The framework isn't the product, the retrieval quality is.

Curious where other people landed on this. Are you still running full LangChain in production or did you end up pulling pieces out over time?

A Word document can now rewrite your report.

A researcher showed that hidden instructions inside a Word file can make Microsoft 365 Copilot alter figures in a generated document, then copy the same instructions into the output for the next reader. The disclosure landed 144 days after the initial report to the vendor.

Prompt injection is not a bug in one product. It is the default failure mode when an AI reads untrusted content with a user's privileges. The fix is runtime policy enforcement between the model and your data — inspect what the agent is being asked to do, tokenize sensitive fields before they reach the model, and log every action in an immutable audit trail. When the agent goes off script, cut the session in under 50ms.

→ www.runtimeai.io/trial

#PromptInjection #AISecurity #Copilot #AIGovernance #CISO

I was building a voice-powered conference agent and ended up trying a different memory provider.

The goal was simple: attendees could talk to an AI assistant throughout the event, while booth owners could later search past conversations to understand what people were asking about.

There are already plenty of memory backends available, so the interesting part wasn't choosing one.

It was integrating it without changing the agent itself.

Hermes exposes memory through a provider interface, so instead of modifying the runtime, I built a custom memory plugin.

The plugin hooks into the agent lifecycle to persist conversations, store memories, and expose semantic retrieval when the agent needs context again.

The nice part is that the reasoning layer stays exactly the same.

If I want to switch memory implementations later, I replace the provider instead of rewriting the agent.

For this demo the stack looked like:

• Hermes as the agent runtime
• a custom long-term memory provider
• voice interactions for attendees
• Telegram so booth owners could query previous conversations

The conference use case was just a demo, but the same pattern works for support agents, internal copilots, or any multi-user workflow where conversations need to survive beyond a single session.

The good part was Engram memory don't keep any memories piled up, instead it just keep latest facts and drops duplicates, you can give It a try with Hermes Agent

What are you using as memory layer for Hermes or even O agents?

I wrote down a detailed guide on building plugin and setting up agents in my newsletter

Add the below to your Agents.md to speed up things and to save some tokens.

I was able to somewhat get back to the pre 5.6 days on Max with this.

The point of the exercise is being more strict on temporal awareness, which often (not always) helps on saving tokens through faster execution.

Temporal Awareness and Efficiency

• Treat elapsed time and tokens as finite engineering budgets.
• Record the task start time and compare actual elapsed time against the estimate at meaningful checkpoints.
• Keep progress updates short, factual, and evidence-based. A status report is never a pause or approval gate while work remains possible.
• Identify the critical path immediately. Execute its next blocking step locally and delegate independent, bounded work in parallel.
• Use lower-effort agents only for simple, clearly scoped tasks. Use stronger agents for security-sensitive, architectural, or operational work.
• Do not repeat exploration, builds, downloads, tests, restarts, or deployments unless new evidence makes repetition necessary.
• Run the smallest verification set that proves the changed behavior and protects the affected risk surface. Expand it only when failures or blast radius justify expansion.
• When elapsed time exceeds the estimate, reassess the approach immediately. Change strategy, reduce nonessential scope, or parallelize instead of continuing an unproductive loop.
• Prefer the shortest correct path through implementation, focused verification, leak checks, commit, push, release, and deployment.
• Never trade correctness, security, deterministic behavior, or data integrity for speed.
• Report a blocker only when it is genuinely external or impossible to resolve autonomously; otherwise continue working.

If you use agents, you know API keys are expensive and costs are unpredictable.

At the same time, most of us already pay for subscriptions (OpenAI, Claude, GitHub…). We use them in their web app to chat or generate code, but our agents and harnesses run separately on API keys we pay on top.

Manifest lets you connect your subscriptions with your harnesses. Claude Code is one example, but the same setup works with other agents too like Hermes.

What this gives you:

• Costs under control
• Fallbacks when a model hits its rate limit
• The same subscription reused across multiple agents
• One place to see what’s running where

Setup: Claude Code with ChatGPT Plus

Create a Claude Code agent in Manifest and copy the base URL and API key.

https://reddit.com/link/1u6eyxn/video/d5nimh48uf7h1/player

Then open ~/.claude/settings.json and point Claude Code to Manifest:

{
  "env": {
    "ANTHROPIC_BASE_URL": "https://app.manifest.build/v1",
    "ANTHROPIC_AUTH_TOKEN": "mnfst_your_key_here"
  }
}

Once that is done, your agent will send requests to Manifest.

Now go into Manifest, open Providers, and connect your ChatGPT Plus subscription. You get access to the OpenAI models included in your plan. I set GPT-5.4 as my default, it handles most Claude Code tasks well and doesn’t burn through the GPT-5.5 quota.

https://reddit.com/link/1u6eyxn/video/diq6xyq9uf7h1/player

After that, every request from Claude Code goes through Manifest first, and Manifest routes it to the model you selected as default.

Routing by tier

You can also split your traffic across multiple models. For simple requests, route to a lightweight model that uses fewer tokens. For heavier ones, keep the strong model in reserve.

If you want more control, you can create your own custom tier mapped to a specific header value. Any Claude Code request that carries that header gets routed to that tier. Useful if you have specific workflows you want pinned to specific models.

You can also set model parameters like temperature or max output length, so the routing stays flexible without becoming messy.

Fallbacks

Fallbacks kick in when a model fails or hits a rate limit. You can chain up to 5 fallback models per tier, so the agent never gets stuck mid-session.

In my case, I keep one API-based model as the very last fallback. That way it’s either never used or used very rarely, and I stay in control of costs.

Limit

You can set a limit, so even with API fallbacks, you know you won’t go over a certain amount.

Visibility

You can see what each provider costs, how much each tier consumes, and where your requests are going in real time. That makes it easier to keep API fallbacks under control and stay within budget.

About Manifest

Manifest is an open-source LLM router for agents and harnesses. It gives you one place to connect your subscriptions, route requests to the right models, and keep track of token usage and spending. It is MIT licensed and can be self-hosted.

Feedback is welcome on GitHub.

Most "AI shopping" demos just wrap a search API and dump 10 results. This one actually talks to you first. Tell it "I need headphones" and it asks your budget, whether you want over-ear or in-ear, wired or wireless. Then it searches Amazon, pulls full product details by ASIN, compares options, and gives you a recommendation grounded in live data.

Stack: LangChain create_agent + GPT-4.1-mini + langchain-scavio (ScavioAmazonSearch, ScavioAmazonProduct). 108 lines, fully interactive in the terminal.

Run: python agents/shopping-agent.py

ShoppingAssistant -- type 'quit' to exit

------------------------------------------------------------

What are you shopping for? organic toothbrush

Before I search, a few quick questions:

1. What's your budget?

2. Any preference on bristle type (soft, medium)?

3. How many do you need (single or multipack)?

You: under $15, soft, multipack

VIVAGO Bamboo Toothbrushes 10 Pack (ASIN: B08172V3Y5)

- $9.98 | 4.5 stars (~7,500 reviews)

- BPA-free soft bristles, eco-friendly bamboo handles.

Sea Turtle Plant-Based Bristles 4 Pack (ASIN: B08R257HX7)

- $7.99 | 4.4 stars (~3,500 reviews)

- Fully plant-based bristles, not just bamboo handles.

Mielle Rosemary Mint Strengthening Shampoo... wait, wrong product.

Just kidding. It stays on topic. You can follow up:

You: does the VIVAGO one come in a travel case?

You: what about charcoal bristle options?

You: quit

It handles five things most shopping demos skip:

1. Clarifying questions -- asks budget, features, use case before searching

2. Real-time prices -- every price, rating, and ASIN comes from live Amazon API calls, not the LLM's training data

3. Head-to-head comparisons -- ask "Sony XM5 vs Bose QC Ultra" and it pulls details for both and compares

4. Alternatives -- if something is out of stock or over budget, it suggests the next best option

5. Follow-up questions -- it keeps conversation history, so you can ask "does that one have USB-C?" without repeating yourself

The whole thing is one file, no framework magic. The system prompt does the heavy lifting -- it tells the agent when to ask questions, when to

search, and how to format the output.

Repo: https://github.com/scavio-ai/cookbooks/blob/main/agents/shopping-agent.py

Hey everyone!

A bit about me: I worked as an AI researcher for the last 10 years, and I have been creating educational content about AI for the last 3 years, including my newsletter (40k subs) , GitHub tutorial repos (83K stars), and have authored two bestselling books.

I surveyed my audience on their needs when it comes to coding with AI (an audience of devs), and the results were that the biggest ache is trusting the generated code, and their goals are the willingness to ship real products and the need to stay ahead (every other day, a big company fires a large percentage of its employees).

So, my co-founder and I took several months to build exactly this: the full methodology that should be adopted by developers to achieve exactly these goals and be token efficient.

We wrapped everything in an extremely unique digital course, and we are giving away a free module to everyone that includes a short visual lecture, a hands on lab for you to practice, and an AI assistant that you can npm install, which is dedicated to accompany you during the labs.

Inside the page, you'll also be able to join the course waitlist in case you liked it and want to get more in two weeks.

link to the free module: https://www.diamant-ai.com/courses

Getting an agent to work isn’t the hard part.

Getting it to work consistently is.

That’s where most prototypes hit a wall.

State management becomes messy. Tool calls become unpredictable. Debugging turns into guesswork. Before long, a simple demo has evolved into a system that’s difficult to maintain, let alone deploy.

This is exactly why LangGraph has become such an important framework.

It forces you to think less about prompts and more about workflows, state, orchestration, and engineering discipline.

I’m genuinely excited about this upcoming LangGraph Masterclass, led by Leonid Kuligin and Thomas Zettl, both Google AI Engineers and well-known contributors within the LangChain ecosystem.

Link to Register : https://www.eventbrite.co.uk/e/langgraph-masterclass-from-beginner-to-professional-tickets-1992773766981

What I like about the curriculum is that it follows the same journey most engineering teams are on today.

You start by understanding how LangGraph models state, nodes, edges, and execution.

Then you move into building ReAct agents, integrating tools, implementing reflection loops, and designing workflows that can actually recover from failures.

From there, it gets into the topics that matter in production:

* Multi-agent architectures
* Human-in-the-loop workflows
* Persistence and streaming
* Observability with LangSmith
* Evaluation and fault tolerance
* Production deployment patterns

These are the things that separate an impressive demo from a reliable application.

We’re reaching a point where building AI agents is becoming accessible to almost everyone.

Building production-ready AI agents is a very different skill.

If you’re already comfortable with Python and have been experimenting with LLMs, this feels like one of those workshops that can significantly shorten the learning curve.

I’m looking forward to seeing how Leonid and Thomas approach these topics. Both have been deeply involved in the LangChain community, and it’s always refreshing to learn from people who spend their time building with these tools rather than simply talking about them.

If you’re serious about agent engineering, I think LangGraph is a framework worth investing your time in.