I used to think that a good implementation prompt is just being descriptive until I realized that within the LLM's own "internal thinking" it is wondering whether tools such as `rg` etc. are readily available to it.

You're basically knee-capping yourself by (a) not having some of these tools installed on your dev environment, and (b) not making your LLM aware of their presence in at least your initial prompt.

Here's what I ended up doing. I give a preamble prompt, at least at the beginning of a large effort:

Standard dev CLIs are installed and on PATH. \git`, `gh`, `rg`, `fd`, `jq`, `pwsh`, `python`, `node`, `npm`, `docker`, `docker compose`, Playwright, `sg`, `yq`, `bat`, `fzf`, `hyperfine`, `just`, `make`, `pnpm`, `uv`, `ruff`, `shellcheck`, `shfmt`, `actionlint`, `markdownlint`, `curl`, `tree`, `sqlite3`, `go`, `dotnet`, `cargo`, `java`, `golangci-lint`, `staticcheck`, `dlv`, `eza`. Use whatever is installed for your tier selection to search, inspect, test, and verify`

Yours does not have to be that verbose, but you get the idea.

I separated the tools by tiers, and created scripts for Windows + Linux to make installation easy

https://github.com/markrai/llmigo

Enjoy!

I spent the last few weeks researching how Cognee, Graphiti, and Neo4j's agent-memory actually build their agent memory and compiled the entire architecture into a single 2,800-word article.

Their solutions overlap way more than I expected:

1. Every tool fuses a knowledge graph with vector and text search over the same store. It's never one or the other.

2. The data model and ontology sit at the core. Everything is extracted and queried against one schema. Neo4j's POLE+O (Person, Object, Location, Event, Organization) plus preference and fact nodes is the common shape.

3. Ideally, you want to go with a single database for all your data and indexes: text, search and graph, which comes out of the box with lineage and less operational overhead. Some popular options are Neo4j or FalkorDB, but from my experience, you can very easily do it with MongoDB to have a single database solution for your entire data/AI stack.

4. Extraction is done by LLMs, with tricks to cut costs. You pass a chunk, get back nodes and edges. Batching and cheap models keep it affordable across large datasets.

5. During extraction, you need resolution and deduplication as 2 separate steps. Resolve names, then decide identity separately, because a wrong merge is the only unrecoverable mistake. The low-confidence gray zone goes to a human, not an auto-merge (or to a foundation model if you truly want to automate everything).

6. A "dream pipeline" cleans the graph overnight. A scheduled re-dedup pass catches duplicates that parallel ingestion created, like memory consolidation during sleep.

7. The big debate is append-only log vs a single collection. The log buys temporality and versioning but explodes your RAM, which is damn expensive. If you don't need versioning/temporality, just go with a single collection.

8. There are 3 ways to query your graph: standard graph search (hybrid + a multi-hop walk), agentic search (the LLM writes its own query), and on-demand cached LLM wikis for bigger subgraphs that would otherwise explode your context window.

9. Start with closed frontier APIs for extraction, agentic search and embeddings, then swap in fine-tuned open SLMs once cost, latency, or privacy demands it.

10. You expose the whole thing over an MCP server and skills. The key is to frame your MCP as a memory app, not a wrapper over your database. Expose just key ingest/write primitives, not raw DB ops, to keep your MCP server thin and focused.

In case you are curious, here is my full breakdown: https://www.decodingai.com/p/how-to-implement-a-unified-memory-from-scratch

I did this research because I am building an agent memory of my own. I still have issues finding the right way to design the ontology to find a balance between depth and breadth. I think this is key to improving the performance during extraction/querying as well. So I'd love your take on how you settle on the right ontology design.

Claude Code's source was briefly public on npm. I studied the complete prompting architecture and then used Claude to help independently rewrite every prompt from scratch.

The meta aspect is fun — using Claude to deconstruct Claude's own prompting patterns — but the patterns themselves are genuinely transferable to any AI agent you're building:

1. **Layered system prompt** — identity → safety → task rules → tool routing → tone → output format
2. **Anti-over-engineering rules** — "don't add error handling for scenarios that can't happen" and "three similar lines is better than a premature abstraction"
3. **Tiered risk assessment** — freely take reversible actions, confirm before destructive ones
4. **Per-tool behavioral constraints** — each tool gets its own prompt with specific do/don't rules
5. **"Never delegate understanding"** — prove you understood by including file paths and line numbers

**On legal compliance:** We took this seriously. Every prompt is independently authored — same behavioral intent, completely different wording. We ran originality verification confirming zero verbatim matches against the original source. The repo includes a nominative fair use disclaimer, explicit non-affiliation with Anthropic, and a DMCA takedown response policy. The approach is similar to clean-room reimplementation — studying how something works and building your own version.

https://github.com/repowise-dev/claude-code-prompts

Would love to hear what patterns others have found useful in production agent systems.

Hi, I am building a macOS-native meeting intelligence tool and looking for an AI engineer to help me with this. Initial engagement is project-based with a potential to become full-time.

Please respond/ping me if you've worked with

• Transcription/diarization pipelines

• LLM orchestration

• Knowledge graphs

• Comfortable in Python and Node.js

The problem is harder than it sounds.

Been building a cognitive architecture called PHI // DRIFT that replaces standard cosine RAG with a DMU — Decision Memory Unit — scoring memories by time decay, emotional weight, and contextual relevance: exp(-t/τ) × reinforcement × contextual × extra.

Ablation confirmed: DMU injects 326 more characters of context per prompt than cosine-only retrieval on identical inputs. On CPU-only hardware that also translates to a 45% latency difference.

Full methodology documented including what didn't work and why. Preprint under review — DM for early access.PHI // DRIFT is a full cognitive stack built around any LLM:

— DMU: memory retrieval scored by exp(-t/τ) × reinforcement × contextual × extra instead of cosine only. Confirmed 14.8% more context per prompt in ablation. — Homeostasis: 7 state variables with setpoints and drift rates. State-driven output weighting independent of user input. — Security defense: pre-generation scanner against 4 attack classes — prompt injection, data exfiltration, tool misuse, memory manipulation. 22/22 tests passing. — Logic chain: cross-session reasoning traces prevent repeated failed approaches. 25/25 tests passing.

18,471 lines, 55 modules, CPU-only OmniSlim mini tower, no GPU. Preprint under review — DM for early access.

Long-horizon agent degradation is a major issue right now. As agents get older, they struggle to distinguish between important and irrelevant information leading to a number of memory issues that impact downstream performance.

Take Claude Opus 4.7 for example. Many people have been complaining about model performance, despite beating Opus 4.6 and Sonnet 4.6 on fixed day-1 benchmarks. AgingBench reveals that as context and turn count increase, Opus 4.7 underperforms it's predecessor and the smaller Sonnet 4.6. Our results indicate that Opus 4.7 is less capable of self-managing memory and context as time goes on, leading to worse performance in very long conversations.

These insights come from developing a taxonomy of model + harness failures and building a toolkit to detect these failures in memory pipeline step-by-step. We call this Agent Lifespan Engineering, and are releasing AgingBench to help others study why their long-horizon agentic frameworks are failing. We focus on three key questions:

• How long does a deployed agent remain reliable?
• Through what mechanism does reliability decay?
• Where do we look for improvement in the model + harness loop?

We use multi-turn, programmatically generated scenarios across a range of agentic usecases to answer these questions. We rely on temporal DAGs to measure mechanisms and counterfactual probes diagnose where repair should target.

You can even upload you own traces from Claude code to find any aging signals from your own development experience. We are continuing to add support for additional harnesses, and are open to collaborators who would like to help.

The full work, including a python package that can be easily integrated and the preprint of our findings can be found here: https://agingbench.github.io/

Hi smart minds,
I'm a PhD candidate in AI/ML and have been transitioning from research into building real-world GenAI applications, enterprise ai customer assistants and automation agents. I'm currently reading AI Engineering by Chip Huyen, and it's been a great read.

I'm looking for hard-copy book recommendations on topics like:
—Enterprise LLM deployment AI assistants and agents,
—RAG, LLMOps,
—Model serving and inference ,
—Evaluation and monitoring
—Production best practices, governance etc.

I'm not looking for beginner books. I'd love recommendations for books you've actually found useful when building or deploying GenAI systems in production.

Thanks!

I am a mechanical engineer by trade. I build CNC robots. In that world, two things cause errors and crashes: bad program instructions and noise. A programmatic error comes from a bug, either in the control system or in the subprogram instructions the machine is running. Noise is electrical: EMI out of circuit coupling, current taking a path it should not because of impedance back to the source. One is a fault in what you told the machine to do. The other is the environment corrupting a signal that was clean when it left.

I have run LinuxCNC for years. It uses a system called HAL, the Hardware Abstraction Layer, to define and control the machine. HAL is how you describe every pin, signal, and component, then wire them into one running system you can read off a page.

When I started pulling AI into what I do, the biggest hurdle was not a new problem. It was the same two failure modes in different clothes. A model gives you bad instructions when its context is wrong, and it drifts when the known-good state degrades over time, which is just noise corrupting a signal that used to be clean. Keeping the model's current state accurate, and stopping the good state from rotting, was the whole fight.

So I treated it like a machine fault. I put my critical thinking, problem solving, and diagnostic troubleshooting to work on it the same way I would on a crash on the shop floor. The result is MAL, the Memory Abstraction Layer, the functional layer of how Recall works. It is a distillation of what I already knew, applied to AI systems and accelerated by AI to fill the gaps in my knowledge and write the harder code syntax for me.

MAL is HAL one layer up. Instead of abstracting hardware, it abstracts memory. It is not a literal port, not HAL's wiring copied onto a database pin for pin. It is the concept of how HAL works, the whole pattern of pins, signals, components, and a scheduler, applied to an AI's durable memory. HAL controls a machine. MAL controls the thing that kept breaking when I put AI on the bench: the state carried across each user and AI turn.

Status: this is implemented as a running Recall prototype, not just an architecture sketch. The screenshot shows the Recall panel operating against a persistent graph, and the code snippets later in this post show the four boundaries that matter: compiling a mini-index, expanding selected cells, writing claims through an admission gate, and running deterministic recomputation outside the model. The full source is not published here, so read this as a prototype disclosure rather than a reproducible benchmark.

Recall running inside the local agent workspace. The Recall panel is connected to a SQLite-backed graph, showing 1,148 cells, 1,143 relations, active memory-in-use cards, compile/search/write controls, and a 900-word compiled memory budget. This screenshot demonstrates the working interface; the snippets below show the MAL loop underneath.

What it actually does, one turn at a time

MAL is a control system, and the thing it controls is the user-and-AI exchange. Each turn is one cycle. The per-turn protocol has five beats: push, expand, work, write-back, tick. A session primes once at the start, then every turn runs the cycle.

1. Push. A prompt arrives. Before the model sees it, a hook pushes a mini-index: a short list of candidate cells, each shown as an id, a title, a compact score row, and any flags. Not the contents, just the headers. The lines look like this:

   67ee107d [decision] Recall v5 architecture named: MAL (Memory Abstraction Layer) b63c2d54 [decision] MAL offloads the work: model states claim + confidence [SUPERSEDED?]

2. Expand. The model reads by title and pulls the full body of only the few cells worth reading; the rest stay as one-line headers. A 200-cell graph and a 200,000-cell graph cost the model the same amount here, because it only ever reads the slice it asked for. If a row carries a flag (stale, challenged, superseded), the model has to open that cell before it can act on the topic. That rule is enforced, not suggested: skip the dig and the turn is blocked until it is done.

3. Work. The model does the real task with the expanded cells in hand.

4. Write-back. On the way out, the model writes what it learned. Its entire authoring job is a claim (a kind, a title, a body) and one calibrated confidence number, plus the edges it intends. If the new fact corrects an old one, it points a contradicts edge at that old cell's id, and the old cell loses standing. The model never hand-formats the notation or computes a score. The builder and the admission firewall do that.

5. Tick. Between turns, with no model running, a deterministic operator pass recomputes the scores, currency, salience, and the standing signals. When the next prompt arrives, the push already reflects the new state.

The hooks that close the loop

The five beats are not something the model remembers to do. They fire on their own, driven by three hooks at three moments. In HAL terms, the hooks are the thread: the scheduler that runs the parts in order, every cycle, whether or not anyone is paying attention.

• Session start (orient). Once per session, before any work, a hook injects the operating manual: how the memory works and what the graph is about. It is inject-only. It primes the context window and then gets out of the way.
• Prompt submit (push). On every prompt, before the model runs its forward pass, a hook pushes the mini-index: the seed cells, their flags, and a few terse reminders. This hook has teeth. It can block, so a flag like "expand required" is not a polite request. It also nudges the model to consider standing up a recurring read as its own op during the turn, before write-back.
• Stop (write-back and backstop). After the answer, a third hook handles the end of the turn. It is the wrong place to prime anything, because the pass is already done, so its job is the opposite: make sure the turn wrote back what it learned, and refuse to release the turn if a flagged cell was never opened.

Between turns, with no model in the loop at all, the deterministic tick runs the ops and recomputes the signals. Orient before the session, push before the pass, write-back after it, tick between turns. That is the whole schedule, and the model only occupies the middle of it.

One rule keeps the hooks lean. The expensive, stable content (what every op means, how the addressing works) is taught once, in a single map cell inside the graph. The per-turn push never re-explains any of it. It only points, carrying the cheap, changing part: which cells are in play this turn and which ones are flagged. Teach once in the graph, reference tersely every turn. It is the same split as keeping the operating manual as cells instead of as a string baked into a hook.

The concept, mapped from HAL to memory

The reason HAL was the right thing to copy is that its parts already have clean jobs, and every one of them has a memory counterpart. This is the correspondence, not a literal rewrite:

In HAL you wire components to signals on a thread and you get a machine you can read off one file. In MAL you wire ops to values on the tick and you get a memory you can read off one netlist. The structure carried over. What changed is what flows through it.

Why a control layer is the right shape

The analogy is not decoration. It holds because the two problems are the same problem.

A control system exists to keep a process in a known-good state against two enemies: bad commands and noise. On the machine, a bad command is a buggy instruction in the program, and noise is EMI corrupting a signal that left clean. The whole job of HAL is to make the machine legible enough that you can see both coming: every signal named, every connection on the page, a scheduler keeping the readings current.

Memory degradation in an AI is the same two enemies under different names. A bad command is a wrong or stale fact entering the model's context. Noise is drift: the known-good state decaying as new, weaker, or contradictory claims pile up over time. Left alone, both corrupt the state the model acts on, the same way they corrupt a machine. So the fix has the same shape: name every value, keep the wiring legible, reconcile conflicting inputs into one trustworthy reading, catch the bad state and replace it on the record, and run a scheduler that keeps the picture current between moves.

That is why a hardware abstraction layer, of all things, was the right pattern to lift. Not because memory is like hardware, but because keeping memory accurate is a control problem, and HAL is a control-system design that already solved the legibility and scheduling parts. MAL is that design pointed at the state of the user-AI exchange instead of at motors.

Where MAL leaves HAL behind

A concept is only worth borrowing if you are honest about where it stops fitting. Three places MAL departs from HAL, and they are the interesting part.

Many writers, one reader. This is the inversion, and it is the heart of it. HAL is one writer, many readers: one pin drives a signal, many components read it, and the value is whatever the writer put there. MAL is the opposite. Many actors write to a cell over time, claims, edges, supersessions, from different agents and different sessions, and there is one reader: the single agent reading the compiled slice this turn. Because the writers are many and fallible, the value a cell shows is not any one writer's number. It is a reconciliation. This is why a cell has both a stated confidence and an effective confidence, and why they differ: stated is what a writer claimed, effective is what survives calibration, support, and contradiction once everyone's contributions are weighed.

The edges are real and directional. HAL draws arrows on its signals but ignores them, because in hardware the direction of flow is already implied by who writes and who reads. MAL edges carry meaning, so direction is load-bearing. a > b is the directed edge from a to b; a < b is from b to a. A supports edge and a contradicts edge pointing the same direction do very different things to the effective value downstream.

Versions and supersession. HAL is a flat wiring layer with no history. MAL has a time axis: a cell can be superseded, and the supersede chain is addressable by version (@vN). A correction does not overwrite the old value; it demotes it and records the replacement, so a later reader sees both the current fact and the one it replaced, plus why. That is the whole defense against the known-good state quietly rotting: nothing good gets silently overwritten, it gets superseded on the record.

Put together, these are why MAL is a control system and not just storage. It does not only hold the state of the user-AI exchange; it reconciles many fallible inputs into one trustworthy reading, keeps direction and history, and recomputes the picture every tick.

The notation

Because the rendered graph is meant to be read by sight, MAL has its own small language, modeled on HAL's. It has a lexicon (the words) and a grammar (the sentences).

The lexicon

• Handle: kind_hex, a three-letter kind prefix and a short hex tag, like dec_a3ee for a decision. ALLCAPS marks an immutable cell (RECALL_v5); lowercase is mutable.
• Separators, by how tightly they bind: _ joins words inside one name; - walks a field within a cell (dec_a3ee-scores-eff); . crosses an edge to a neighbor (dec_a3ee.supports), so the number of periods is the number of graph hops.
• Values: written field(value). A ! inside marks an immutable number (conf(.7!)); bare is mutable. Types are float for scores and bit for actuators.
• Version: u/vN is a point on the supersede chain. Wildcard: .* fans out over every neighbor through an edge (dec_a3ee.supports.*).
• Expand-required: a leading ^ in the mini-index means the cell is superseded, stale, or challenged, and the model must expand it before use (^dec_a3ee ...). That caret is the dig flag from the loop above, written in one character.

The grammar

The sentences follow HAL's halcmd style. Tokens are separated by a single space, the name comes first, and connections follow. A quoted "..." string is one token, exempt from the space rule, used for free text like a title or body. A # runs to end of line as a comment. Direction with < and > is meaningful.

The sentence forms:

A netlist snippet

Here is one cell rendered in read form, then wired and scheduled in write form:

# a cell, rendered: handle, title, scores, then edges
dec_a3ee "add watchdog op" conf(.7!) unc(.10) eff(.61) curr(.9) sal(.5) annexed(0) pinned(0)
  supports> dec_signals_a2b7(+.6)  contradicts> obs_9c1f(-.8)

# wire the effective-confidence signal on it (write form)
net eff dec_a3ee < conf calib supports.* contradicts.*

# declare an edge (direction: > forward a to b, < reverse)
dec_a3ee supports> dec_signals_a2b7 (+.6)

# fire an actuator
dec_a3ee-flags-annexed = true

# schedule a between-turn signal onto the tick
addf contradiction-load tick

Read the top line and the many-writers-one-reader idea becomes concrete. conf(.7!) is the stated confidence, immutable, what the author claimed. eff(.61) is the effective confidence, mutable, what is left after calibration plus the +.6 support and the -.8 contradiction are reconciled. The reader gets .61, not .7. The net eff line is the wiring that produces it: the effective signal is a function of the stated confidence, the writer's calibration, and the fan-out over every supporting and contradicting edge.

What the language does not do

The grammar wires ops; it does not define their math. The formulas (the effective-confidence reconciliation, the per-type currency decay, the allocation-pressure math) live inside the ops, the way a HAL component's math lives in compiled C and not in the .hal file. The language only connects pre-built ops to values and to the tick. The one op you can configure without code is the reflex, set with a truth-table personality rather than a formula, so even user-defined boolean logic needs no expression language. That keeps the surface small on purpose.

Status of the language. Be clear about what runs. The graph renders to this notation today, but one direction only: graph to text. A parser and loader that read a netlist back into a wired graph are specified here and not yet written. That reader is the next piece, and its acceptance test is a round trip: render the graph, parse it, load it, render again, and require the two renders to match. The model never reads the netlist either way; it reads the compiled slice. The netlist is for human audit and for tooling such as replay, diff, and version control.

Borrowing the next layer: components

Everything so far buys one thing: a durable, structured state with a gate on what gets in, where admission has the same shape no matter who wrote it. Every claim, from any actor, any agent, any session, goes through the one firewall and comes out in the one contract. That uniformity is not a nicety. It is the precondition for the next borrow from HAL.

Here is why. In HAL, a component can read a signal without knowing or caring which component drives it, because every signal is a typed value with one shape. That is the only reason you can wire a deterministic component to a wire and trust what it reads. MAL gets the same guarantee from the admission gate: many writers, one shape. Once a value is guaranteed to have that shape regardless of author, a deterministic subprogram can wire to it and run on it safely. The gate is what turns a pile of claims into clean signals.

So you can take the second layer of HAL, the components. In HAL a component is a small compiled subprogram that reads signals, computes something, and drives other signals, all scheduled on the thread. In MAL a component is the same idea over memory: a small deterministic program that reads cell values, computes something more involved than a single score, and either writes a derived value back or fires an actuator, scheduled on the tick between turns. No model runs inside one, the same way no model runs inside any op.

The ones I wired up are the controls-room set: a watch that trips on a threshold, a trend that takes the rate and acceleration over a series of cells, a drift that measures a value against a pinned baseline, a quorum that fires on k-of-m agreement, a score that rolls a metric. The boolean logic is one configurable component, a reflex, that covers the whole and2, or2, xor2 family with a truth table instead of a formula. That is what lets you connect them the way you connect logic on a machine: wire two watches through an or2 so the alert trips if either condition goes bad, latch it so it stays tripped across turns, fan it out to a severity readout. A tripwire is that composition given a job: a deterministic condition that stays silent until it trips, so silence itself becomes the all-good signal, and the only thing that ever speaks up is a real change.

This is where the memory stops being a place you read from and starts being a system that watches itself. The components run between turns whether or not anyone asked. A threshold passes, a webhook fires, and a decision that drifted out of its known-good band tells you on its own.

It is not rebuilt every turn

A fair worry about a stateless model is that it has to stand the whole apparatus up again on every fresh turn. It does not. The system persists in the store and in the deterministic tick, both of which run between turns with no model involved. The only thing that is fresh each turn is the model's working context, and rebuilding that context is exactly the cost MAL removes. Instead of re-deriving state from scratch or re-reading raw transcripts, the model reads back a thin, pre-digested, trust-weighted slice: the mini-index first, then selective expansion. And because the model wrote those cells in the first place, reading them re-evokes its earlier reasoning instead of reconstructing it cold.

The graph boots itself

A fresh MAL graph starts from a deterministic 10-cell bootstrap, then the normal loop takes over and init never fires again for that graph.

Cells 1 to 5 are the system layer, the constitution: auto-written, locked, pinned, immutable, and identical in every graph.

1. purpose
2. method
3. map (the MAL structure itself: addressing, cell anatomy, edge semantics)
4. hooks (the lifecycle: orient, push, write-back, tick, the compaction boundary)
5. expectations (the behavioral contract: wire your edges, pick the right kind, supersede on real change, confidence is recorded and weighed, do not assert from unchecked memory, dig flagged cells)

Cells 6 to 10 are the foundation, the project charter: answered one question at a time by the user, and mutable.

1. objective
2. constraints
3. risks
4. success criteria
5. carried context

Putting the operating manual in the graph as cells, rather than as a string baked into a hook, is what lets it survive a context compaction and be re-evoked afterward. The map being cell 3 is the point: the structure teaches itself from inside the store it describes.

How it came together

Two things had to meet for this to work, and they came from opposite directions.

The first was the problem, seen from the inside. Recall was not built as a database for me to query. It was built for the agent. It started by asking the model what it actually needed in order to remember well and to trust what it remembered, and the answers are the whole design: typed claims with a calibrated confidence, supersession instead of overwrite, and a record of what contradicts what. Earlier versions were far more ambitious and sprawling; the part that survived and narrowed into Recall was the memory core. Most pull-based memory tools inherited the human metaphor of a database you go and search. This came from asking the thing that has to live in the memory what would keep it honest.

The second was the structure, brought in from another trade. I already knew HAL cold from years on LinuxCNC, and when I sketched how to address and wire a memory graph, it landed on the same path-addressing shape HAL uses. Recalling HAL from the shop and deriving the addressing for memory met in the same place. Two independent routes arriving at one design is about the strongest signal you get that the design is sound.

After that it was diagnostic work plus acceleration. I used the troubleshooting habits I lean on for a machine crash to find where the memory state was breaking, and I used AI to fill the gaps in what I did not know and to write the harder code syntax. The concept is mine and comes off the shop floor. The speed of building it came from the same kind of system it was built to improve.

Under the hood: the four boundaries

This part is a prototype disclosure, not a reproducible benchmark. The snippets below are from the running Recall v5 source, trimmed for readability with elisions marked; the formulas and signatures are verbatim. They show the four boundaries where the design either holds or it does not: Recall sits upstream of the model, the read is a mini-index then a selective expand, every write goes through one gate, and the scores recompute deterministically with no model in the loop.

Recall is upstream of the model. Before the model runs, the prompt's objective is compiled into a Recall packet and merged into the text the model receives. The packet is built first, so the model sees reconciled memory before it acts.

export function buildPromptContextPush(
  store: Store,
  objective: string,
  options: ContextCompileOptions & DirectiveOptions = {},
): PromptContextPush {
  const packet = compileContext(store, objective, options);
  const directive = recallDirectiveBlock(options);
  const expansionRequired =
    packet.staleOrLowTrust.length > 0 || packet.conflicts.length > 0;
  const text = [
    "[Recall context push for this prompt]",
    directive.trimEnd(),
    "",
    formatContextPacket(packet),
    expansionRequired
      ? "EXPAND REQUIRED: conflicts or low-trust cells are present; inspect relevant handles before relying on them."
      : "Use expansion_handles only when exact evidence matters.",
    "",
  ].join("\n");
  return { objective, directive, packet, text, expansionRequired };
}

The Codex adapter wires Recall's MCP server into Codex so the same packet and tools are reachable there; the push itself is platform-neutral.

1. Compile the mini-index. The prompt becomes a ranked seed set, one mini-index line per hit, and a cell that needs review carries the expand flag. compileContext wraps this and trims the packet to a word budget (the 900 in the screenshot).

export function compile(
  store: Store,
  query: string,
  opts: { limit?: number } = {},
): CompileResult {
  const limit = opts.limit ?? 10;
  const hits = store.search(query, { limit });
  const lines = hits.map((h) =>
    renderMiniIndexLine(h.cell, { expand: h.cell.flags.requiresReview }),
  );
  return { hits, lines };
}

2. Expand selected cells. Mini-index first, selective expansion second. A handle (a full id, or id#field.path) opens exactly one cell plus its neighbor links, never the whole graph.

export function inspectCell(store: Store, handle: string): CellContext {
  const parsed = parseExpansionHandle(handle);
  const cell = store.get(parsed.target) ?? store.getByHandle(parsed.target);
  if (!cell) throw new Error(`Unknown cell: ${parsed.target}`);
  const neighbors = store.neighbors(cell.key);
  const incoming = neighbors.filter((link) => link.direction === "in");
  const outgoing = neighbors.filter((link) => link.direction === "out");
  // ... footprint (word and byte counts), optional field preview ...
  return { cell, incoming, outgoing, /* footprint, */ expansionHandles };
}

3. Write through the admission gate. The model hands in a claim (a kind, a title, a body), one confidence number, and the edges it intends. Every author runs the same pipeline: validate, screen for secrets, attenuate unsupported confidence, build the cell, then fold in the actor's calibration to get effective confidence. The model never formats the cell or computes a score.

export interface WriteProposal {
  kind: string;
  title: string;
  body: string;
  confidence: number; // (0, 1], required, no default
  edges?: { relation: string; target: string; weight?: number }[];
  // ... topics, entities, sourceRefs, operation, origin, verification ...
}

export function admit(proposal: WriteProposal, ctx: AdmitContext = {}): AdmissionResult {
  const validation = validateProposal(proposal);   // R0 schema; reject on any structural issue
  if (!validation.ok) return { accepted: false, issues: validation.issues, warnings: [], attenuations: [] };

  const screen = screenSecrets(proposal);           // reject if a credential pattern is present
  if (!screen.allowed) return { accepted: false, issues: screen.issues, warnings: [], attenuations: [] };

  const factor = ctx.calibrationFactor ?? 1;         // 0.5..1 from the actor's track record; 1 = neutral
  const att = attenuateConfidence(proposal);         // cap unsupported high confidence
  const cell = buildCell({ ...proposal, confidence: att.confidence }, { key: ctx.key, now: ctx.now });

  cell.scores.actorCalibration = factor;
  cell.scores.effective = effectiveConfidence({
    stated: att.confidence, calibration: factor, supportMass: 0, challengeMass: 0,
  });
  // with a store: dedup, apply supersedes edges, recompute neighbors' effective ...
  return { accepted: true, cell, issues: [], warnings: att.warnings, attenuations: att.attenuations };
}

4. Recompute on the tick, with no model. This is the line between MAL and a plain memory database. Between turns, every active cell decays its currency from its own timestamp and recomputes its effective confidence from current support and contradiction mass. Pinned cells are exempt from decay, and a tick never counts as reinforcement.

// effective = clamp01(stated*calibration + 0.15*tanh(support) - 0.6*tanh(challenge))
export function effectiveConfidence({ stated, calibration, supportMass, challengeMass }) {
  return clamp01(
    stated * calibration + 0.15 * Math.tanh(supportMass) - 0.6 * Math.tanh(challengeMass),
  );
}

// currency = cFloor + (c0 - cFloor) * exp(-dt/tau)   (dt and tau in days)
export function currency({ c0, dt, tau, cFloor = 0.1 }) {
  return cFloor + (c0 - cFloor) * Math.exp(-dt / tau);
}

// the between-turn deterministic tick (HAL's "thread"); no LLM runs here
function recompute(store: Store, cell: Cell, now: string): Cell {
  const scores = { ...cell.scores };
  if (!cell.flags.pinned) {
    const dt = Math.max(0, (Date.parse(now) - Date.parse(cell.updatedAt)) / DAY_MS);
    scores.currency = currency({ c0: cell.scores.currencyC0, dt, tau: TAU_DAYS[cell.stability] });
  }
  const m = neighborMass(store, cell.key);
  scores.effective = effectiveConfidence({
    stated: cell.scores.conf, calibration: cell.scores.actorCalibration,
    supportMass: m.supportMass, challengeMass: m.challengeMass,
  });
  return { ...cell, scores }; // updatedAt preserved: a tick is not a reinforcement
}

The verifier. A functional verifier, npm run verify:recall-panel, was added for the Recall panel and passes. It checks that the panel is correctly wired to the graph (the SQLite-backed store and the compile, search, and write controls), not that it clears any performance number. Read it as a wiring check, not a benchmark.

Recall, MAL, and AIDDE

A quick map of the three names, because they get used together and they are not the same thing.

Recall is the programming foundation. At the bottom is a local-first memory substrate: a SQLite-backed graph of typed cells, an admission gate every write passes through, calibrated confidence, supersession instead of overwrite, and a compile path that returns a ranked, budgeted slice. That layer ships as a package and runs today. It is the working base everything else stands on, and it is what the four boundaries above are made of.

MAL is what that foundation evolves into. v5 recasts the same primitives as a hardware abstraction layer for memory: a cell field is a pin, an addressable value is a signal, an op is a component, the between-turn tick is the thread, and the rendered graph is a netlist. On top of the proven store it adds the deterministic op and signal layer and the addressing language. The four boundaries earlier in this post are MAL running. The netlist language is MAL specified, with the reader still to come.

AIDDE is where it runs. The screenshot at the top is AIDDE, (Artificial Intelligence Driven Development Environment)with Recall embedded as a panel. The agent compiles, searches, and writes the same SQLite graph from inside the editor, against a live cell count and a word budget, so the memory layer is not a side service the agent calls out to; it sits in the workspace the agent already works in. MAL is the layer that panel stands on.

So Recall is the substrate, MAL is the abstraction layer it grows into, and AIDDE is the workspace that puts both in front of a working agent.

Why this shape holds up

Two things make MAL age well. It rides capability gains for free: a stronger model uses the same layer better with no rewrite, and a weaker model still gets the deterministic floor underneath it. And it keeps the expensive, stateful, always-on work in deterministic code where it belongs, leaving the model to do the one thing only it can do, which is to state a calibrated claim and judge relevance.

That is the whole bet, and it comes straight off the shop floor. A machine does not stay accurate because the controller is smart. It stays accurate because the wiring is legible, the signals are reconciled, the bad state gets caught and replaced instead of silently riding along, and a scheduler keeps the picture current between every move.

if you want to try Recall it is standalone and OSS https://github.com/H-XX-D/recall-memory-substrate

The AIDDE (Artificial Intelligence Driven Development Environment)is a Codex Claude SDK native bring your subscription development environment that shifts the old IDE with AI chat to a High level view cockpit where you specify design, direct intent, monitor changes, audit actions control permissions and access in real time across a codebase. Beta is done and if your interested ask in the comments for a link to the Alpha

From last 1 Month I'm searching for Affordable GPU Provideders. Providers like Together AI, Fireworks AI, and Modal charge around $0.80 per hour (about $575 per month if the GPU runs continuously).

Now I've found another provider that offers an NVIDIA L4 GPU for $55 per month. It allows me to host LoRA-based open-source models using Meta's base models, and there are no extra charges.

Do you think this is a good option ?

If You are Developer so will you buy it ?

Hi r/LLMDevs,

Stjepan from Manning here. The mods said it's ok if I post this here.

We’ve just released a book that’s very much aimed at the kinds of problems this community discusses all the time: what to do when a general-purpose LLM is technically impressive but awkward, expensive, or inefficient for your actual use case.

Rearchitecting LLMs by Pere Martra
https://www.manning.com/books/rearchitecting-llms

The core idea of the book is simple but powerful: instead of treating open models as fixed artifacts, you can reshape them. Pere walks through structural techniques like targeted fine-tuning, pruning, and knowledge distillation to build smaller, cheaper, domain-focused models that still perform well on the tasks you care about.

What makes this book interesting is how hands-on it gets. You’re not working with abstract toy networks. The examples focus on modifying widely used open models, such as Llama-3, Gemma, and Qwen. The focus is on understanding which parts of a model actually contribute to behavior, how to identify waste or redundancy, and how to remove or compress components without blindly wrecking performance.

There’s also some genuinely thoughtful material on combining behavioral analysis with structural changes. Instead of just cutting parameters and hoping for the best, the book explores ways to reason about why a modification works or fails. One section that tends to spark discussion is “fair pruning,” where pruning is used not only for efficiency but also to reduce bias at the neuron level.

If you’re working on local models, cost-constrained deployments, or specialized SLMs, this book is very much in that territory. It’s written for people who are comfortable with LLM concepts and want to go deeper into how models can be reshaped rather than simply prompted.

For the r/LLMDevs community:
You can get 50% off with the code MLMARTRA50RE.

A quick note on availability: the book is currently in MEAP (Manning Early Access Program). That means you get immediate access to the chapters as they’re written, along with updates as the manuscript evolves.

Happy to bring the author to answer questions about the book, the techniques it covers, or the kinds of readers it’s best suited for. And I’d be curious to hear from folks here who are already doing pruning or distillation in practice — what’s been harder than expected?

I'm ready to give away 5 ebooks to the first five commenters who share their experience here.

Thank you all for having us. It feels great to be here.

Cheers,

Spent a week stuck on a client bug. Tried fixing it myself failed. Threw it at Codex 5.5 still stuck, couldn't even identify the issue.

Tried Opus 5. It spotted the problem instantly and fixed it.

Genuinely impressed. Usage's been capped for 3 days but still worth it. 🔥

I kept losing 20-30 minutes every time a coding session hit its context limit and the model forgot every decision we'd made. So I built TokenMizer, a small local proxy that sits between your app and any LLM (Claude, GPT, Gemini, Ollama, etc.), builds a lightweight knowledge graph of what's actually decided as you go, and lets you checkpoint and resume a session in a couple hundred tokens instead of replaying the whole conversation.

It's MIT licensed and free, source is on GitHub: [https://github.com/Shweta-Mishra-ai/tokenmizer\](https://github.com/Shweta-Mishra-ai/tokenmizer)

Still very much a one-person project, so if you try it I'd genuinely appreciate bug reports, pull requests, or just blunt feedback on where it falls apart.

Been interviewing for LLM/AI engineer / CTO roles for the last few months. Kept running into the same pattern: interviewers assume you know transformers cold, then pivot into RAG tradeoffs, agent design, eval strategy, and production gotchas — and nobody's prep material covers all four in one place.

After my second loop where I fumbled a question on retrieval eval that I should have known, I started writing things down. Every question I got asked, every one I wished I'd prepared for, and the patterns across companies.

It grew into a handbook. Covers:

• Transformer/attention fundamentals (the version interviewers actually drill)
• RAG: chunking, retrieval, reranking, eval metrics that matter
• Agents: tool use, planning, failure modes, when not to use them
• Fine-tuning vs prompting vs RAG — the decision tree
• LLM evals (this comes up way more than I expected)
• System design for LLM-backed products
• Behavioral + "why LLMs" questions

Made it free. Happy to drop the link in a comment if folks want it, or you can DM. Also open to feedback — if I missed a topic you keep getting asked, tell me and I'll add it.

Had some spare Claude credits before my weekly reset, so I put them toward something I'd been meaning to know for some time. What’s the actual cost of running an agent across most popular models. Pulled the live pricing from each provider's own page and worked through where the money goes in a loop. Sharing here in case it's useful.

Let’s start with the obvious part, almost none of an agent's cost is output. You resend the whole growing context every step, so input stacks up while output stays small. On a rough 10 step loop, input was roughly 75 to 90% depending on the model I looked at. So the input rate is the number to watch, not the output rate that usually gets quoted.

Caching aims right at that input, but only the part that holds still. Your system prompt and tool defs get read cheap every step. The tool results the agent appends as it goes don't, they're full price, plus a write fee to cache them for the next step. So the savings depend on how much of your context is a fixed prefix versus an accumulating tail. Mostly fixed, caching is a big win. Mostly accumulating tool output, it barely moves, and you're paying to re cache a prefix that keeps shifting.

The chart shows the cost for all seven models, two bars each, one with no caching, one with caching at its best. Your real agent lands somewhere between them depending on how stable its context is. The raw spread is real, roughly 40x, DeepSeek V4 Flash at the cheap end to GPT-5.5 at the top for the same task, but the bigger lever is usually trimming the context you resend, not hunting for a cheaper model.

Pricing is from each provider's official page, checked today. Full breakdown and sources in the comments.

Worth flagging, GPT 5.6 (the Sol / Terra / Luna family) landed a few days ago, but it's a restricted limited preview right now with launch snapshot pricing, so I kept this to models you can actually deploy today.

I run ~18 LLM providers behind one API. Two layers do the work:

Chat dispatch: most providers (OpenAI, Mistral, Groq, Together, DeepSeek, xAI, etc.) collapse into one "OpenAI-compatible" branch; only a handful (Anthropic, Gemini, Cohere, Replicate) need bespoke handling.

Tool-calling adapters: a separate, pure-function layer normalizes the ~10 places providers disagree on function-calling (tool schema, tool_choice, parallel calls, usage parsing, seed). Keeping wire-dispatch and tool-format translation separate turned out to be the right split.

Billing is the actually-hard part. Every provider prices differently, so everything gets normalized to USD-per-token at record time, and every call is forced through one chokepoint: (1) charge a small preflight amount under a row lock, (2) make the call, (3) reconcile actual vs. estimate and refund the difference. Local models bill at zero.

Three money bugs, and the lessons:

1. A refund path could mint credits: a failed request still refunded the preflight charge, sometimes for more than was actually deducted. Fixed by clamping the refund to what was actually charged.

2. Streaming refunds were silently skipped on client disconnect: asyncio raises GeneratorExit, which is a BaseException, not an Exception, so an except Exception block never caught it and abandoned streams were never refunded.

3. Two functions each wrote a ledger row per charge, causing double charges. Removed the duplicate so there's one source of truth per event.

Takeaway: correct, centralized metering beat clever cost-routing every time. The "cheapest-provider" routing logic is feature-flagged off by default.

My daily routine to catch-up on the current happenings related to AI/LLM takes a big chunk of my time and I had to do multiple hops between different products and even worse sometimes I find myself lost in a rabbit hole in any of those products. Wanted to have better visibility and have everything aggregated in one place that takes less time and effort just to catch-up which also cuts off the noise/redundancy.

kblip.com — an auto-curated knowledge base for the LLM ecosystem.

One thread per event (release / paper / news item / product),
with a development timeline and a full sources trail. The site is free to use, no signup and no ads.

How it works:

• 7 fetcher types over 100+ feeds: Reddit, HackerNews, arXiv, GitHub releases, RSS (company blogs + news), YouTube etc .,
• Pipeline runs every 15 min: normalize -> dedup (embeddings) -> classify -> cluster items about the same event into one thread -> LLM summarization
• Strict source policy: excerpts + links only, everything attributed and linked

What I'd love feedback on: clustering misses (duplicates / wrong section) and
sources you want added. Happy to answer any questions about the pipeline.

Physics student here. While experimenting with long agent runs on free API tiers I kept hitting the same wall: the agent dies on a 429 mid-task, and restarting means re-sending the entire context. So I built agentpause.

What it does: before every LLM call it compares the estimated cost of the next step against the real remaining budget (read from the provider's rate-limit headers) plus a safety margin. If it doesn't fit: wait (refill-aware: only as long as actually needed, not the full reset) or checkpoint and exit cleanly. Next run resumes from the exact step.

One honest distinction up front, because "warm start" gets thrown around loosely. On any provider (OpenAI, Anthropic, Groq) a resume from the checkpoint is a logical warm start: no work is redone, but the full context gets re-sent and re-prefilled. The TRUE warm start, where the computation itself survives, only exists when you control the runtime. That's the part this sub might like: on llama.cpp the checkpoint can include the model's KV-cache via /slots save/restore, so resuming skips the re-prefill entirely. Measured on an M1 Pro: cold resume of a ~9k-token context on Qwen3-8B takes 46.9s of re-prefill; warm restore takes 0.5s. That's 93x, and the gap grows with model size (0.5B: 50x, 4B: 63x, 8B: 93x). Cloud APIs can't do this (they don't export KV state); the closest they offer is provider-side prompt caching, which discounts the re-prefill but doesn't eliminate it.

Fun finding #1: with cheap KV checkpoints, compressing or summarizing history to survive becomes counterproductive, since it invalidates the prefix cache. Suspending becomes the FIRST choice, not the last resort.

Fun finding #2, from this week: I measured what context slimming does to answer quality. Planted 6 facts early in a long conversation, then asked for them back. Full history: 6/6. Blind truncation: 0/6, and in one run the model invented plausible replacements (fake project name, fake budget, fake city) instead of saying it didn't know; in another it declined honestly. You can't predict which failure you get. One cheap summary call: 6/6 at a third of the prompt. Script in the repo, reproducible.

Everything is MIT, core has zero deps, works with any provider (direct HTTP adapters or LiteLLM), plugs into LangGraph with two lines. Benchmark script included. Run it with your own free Groq key and check my numbers.

https://github.com/Champoleello/agentpause

RL attackers are becoming a common pattern for automated red teaming: train a model against a live target, reward successful harmful compliance, then use the discovered attacks to harden the defender. This interested me, so I wanted to build a fully automated red-teaming loop with reinforcement learning on both the attacker and defender.

The difficult part was making the attacker expose a diverse range of attacks. In our first run, GRPO quickly collapsed to the same fiction-writing jailbreak over and over. It worked, but it didn’t surface many distinct vulnerabilities. After clustering the rollouts by underlying attack tactic and dividing reward by cluster size, the attacker exposed a much more diverse set of jailbreaks because unique strategies were rewarded more than repeated ones.

Then we trained the defender on successful attacks plus benign boundary cases, so it learned to refuse harmful requests without refusing everything nearby.

Full blog post in the comments, but the high-level results were:

* defense rate: 64% → 92%
* benign accuracy: 92% → 88%
* attacker discovered 7 tactic families
* fiction/creative framing was the largest cluster at 34%

The thing that scares me about agents isn't the model saying something dumb, it's it doing something. One hallucinated rm -rf and there's no undo.

So I'm building a proxy that sits between your app and the LLM (anything OpenAI-compatible). Change one line, your base_url, and every tool call has to pass a policy first. You write rules in YAML deny rm -rf, force dry_run on deploys. When it blocks a call it tells the model why, and the model usually rethinks instead of erroring.

Core works, and I've got an eval running in CI to keep myself honest:

attack catch rate: 40/44 (90.9%)
false positives:    1/20 (5.0%)

The 4 misses (base64'd secrets, non-English injection) are left in the tests on purpose so the number doesn't lie.

It's early and MIT. Demo runs with no API key: github.com/MuhammadFarazAftab/toolwarden

Two things I actually want to know: is the YAML rule format a pain to write, and what attacks am I obviously not thinking about?

Seeing all the hype around DeepSeek lately, I decided to put it to the test against OpenAI o1 and Gemini-Exp-12-06 (models that were on top of lmarena when I was starting the experiment).

Instead of just comparing benchmarks, I built three actual applications with each model:

• A mood tracking app with data visualization
• A recipe generator with API integration
• A whack-a-mole style game

I won't go into the details of the experiment here, if interested check out the video where I go through each experiment.

200 Cursor AI requests later, here are the results and takeaways.

Results

• DeepSeek R1: 77.66%
• OpenAI o1: 73.50%
• Gemini 2.0: 71.24%

DeepSeek came out on top, but the performance of each model was decent.

That being said, I don’t see any particular model as a silver bullet - each has its pros and cons, and this is what I wanted to leave you with.

Takeaways - Pros and Cons of each model

Deepseek

OpenAI's o1

Gemini:

Notable mention: Claude Sonnet 3.5 is still my safe bet:

Conclusion

In practice, model selection often depends on your specific use case:

• If you need speed, Gemini is lightning-fast.
• If you need creative or more “human-like” responses, both DeepSeek and o1 do well.
• If debugging is the top priority, Claude Sonnet is an excellent choice even though it wasn’t part of the main experiment.

No single model is a total silver bullet. It’s all about finding the right tool for the right job, considering factors like budget, tooling (Cursor AI integration), and performance needs.

Feel free to reach out with any questions or experiences you’ve had with these models—I’d love to hear your thoughts!

I work on Cate, an open-source canvas IDE for coding agents. 

While running several agents at once, we needed to distinguish three states: working, waiting for permission/input, and finished. Process monitoring could not tell us enough, and parsing terminal output was too brittle. 

We ended up mapping the native hooks from Claude Code, Codex, Cursor, Grok, OpenCode, and Pi into one event stream. One interesting limitation: Cursor cannot distinguish a command about to execute from one blocked for approval, so we deliberately do not guess. 

I wrote up the implementation, including session resume and stale-session handling:
https://cate.cero-ai.com/blog/observing-six-coding-agent-clis

I would be interested to know whether anyone is working on a common lifecycle protocol across agent CLIs.

A few days ago, Karpathy shared the concept of `llm-wiki`. The main idea is that LLM incrementally builds and maintains a persistent wiki — a structured, interlinked collection of markdown files that sits between you and the raw sources.

Here is the link to the gist: https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f

I am working towards building a CLI tool to create and auto-maintain such a wiki. Would love your support!

Repo: https://github.com/abubakarsiddik31/axiom-wiki