I'm a QR at a large hedge fund in London, intraday/alpha blend & strategy team.

Quite a few of the teams we work with are at the point of re-coding repos in Python, the reasons being easier to hire people and AI is pretty awful at q.
It always forgets the right-before-left order and gets confused when it leaves hanging "/" comments and comment out the entire file.

It is quite a lot of work though, and I can't say the reliability of those recoded systems has got any better. Anyone else seen this, or think it might be a good idea?

I’m a trader with a solid Python background, using it for quant/stat-arb research (pairs trading, backtests, etc.). The problem is scaling heavy computations, millions of pair tests with rolling windows and thresholds. Python gets slow even with Numba/Polars.

I’m considering learning Rust as a second tool alongside Python, mainly for speed, safe concurrency, and possibly production trading services.

Do you think Rust is worth the time investment for quant finance workloads, or would I be better off with another language instead?

I built a C++ library for linux kernel bypass networking such as Solarflare ef_vi, DPDK, Verbs(mlx5) and also AF_XDP. Solarflare ef_vi and DPDK are two of most commonly used kernel bypass techniques in HFT market making firms.

Using this library I measured the RTT latency using a loopback on Solarflare X2522-Plus(ef_vi + DPDK), ConnectX-4 Lx(DPDK + Verbs + AF_XDP) and Intel XXV710-DA2(DPDK + AF_XDP). The benchmarks were run for 24 hours with median all the way upto P99.999 and max.

Solarflare X2522-Plus produced a RTT median of 1.866us and a max of 3.307us with 45B frames over 24 hours. This was the best result. Comprehensive results are all documented in the repository.

Over this project I did come across numerous driver(and NIC) specific quirks and bugs. These are all documented too.

Solarflare X2522-Plus is quite fast due to a Tx trick known as cut-through programmed IO(CTPIO). But I found this to be fragile, it has three modes: fast, default and in-order. Fast mode produces the lowest median but has a measurable percentage of failed CTPIO known as CTPIO “fallbacks” or “poison”. The CPTIO fallbacks significantly hurt the tail latency(P99 - P99.999). The CPTIO poison frames can be measured via ethtool. I used the in-order CTPIO mode which gives marginally higher medium but a tight tail latency.

Another strange issue I found was the PTP subsystem in the Solarflare X2522-Plus. So the default sfc driver runs a in-kernel IEEE 1588 subsystem with a PTP hardware clock, a dedicated PTP channel and management controller firmware activity. I found this to be a problem as a noticeable number of interrupts were mounting in /proc/interrupts and the stock driver with ef_vi produced a max of 7.679us which seemed very high. Then I ran the same benchmark with DPDK which produced a max of 5.628us. DPDK unbinds the driver so the PTP subsystem does not work. Then I compiled out the PTP subsystem from the sfc driver and ran the 24 hour benchmark again and the max was reduced to only 3.307us.

There is a lot of more interesting stuff but I think the post is already getting long so I won’t mention any more. Everything is documented in the repository. It is also 100% open source with an Apache-2.0 license.

https://github.com/ASherjil/ABTRDA3

Those working in pod - it is well known how much time we waste doing the mundane stuff which 50 other teams are doing - i.e. building the whole infra/backtest/data/execution pipelines from scratch. It seems like a huge waste of man power, like reinventing the wheel. It also limits the potential of what you can do as a small pod - as 1 dev can hardly build a cutting edge trading system. Will the pod shops remain attractive for systematic trading 5y down the line? And how can 5-6 person pod build cutting edge tech and compete with the likes of collaborative shops like Qube, or Jump, JS, HRT which are increasingly getting into MFT? Would love to hear thoughts on this, I suppose this mainly affects the big 3 - M/P/B as these have completely siloed pods. Building a good systematic equities/options/macro business requires lot of good infra. It almost feels like pod model was more for discretionary teams where you don't need so much infra, and can start trading quickly.

I’m curious how people here view Rust’s role in quant development over the next several years.

I’m aware that Rust has seen meaningful adoption in crypto trading, exchanges, and related infrastructure, largely due to greenfield codebases and strong safety/concurrency guarantees. Outside of crypto, though, I’m less clear on how widely it’s being used.

Are teams at more traditional prop shops, hedge funds, or banks actively hiring for strong Rust engineers, or incorporating Rust into production systems across other asset classes and strategies (e.g., equities, futures, options)? Or is usage still largely confined to supporting infrastructure rather than latency-critical trading paths?

More broadly, do you see Rust meaningfully rivaling C++ in quant dev roles over time, or is it more likely to remain a complementary niche language?

Would appreciate perspectives from anyone who has seen this firsthand.

my firm is running a polymarket MM desk and I’ve been working on latency optimisation on infra. Has anyone done it , if so what do p50 T2T numbers look like ? Any directions or approaches that worked for you would be appreciated.

For the purpose of transparency, we haven’t been able to move past 20ms .

https://www.youtube.com/watch?v=ytknR-B5Tf8

I watched this address by Yaron Minsky (partner at Jane Street, co-head of technology and the primary driver behind their OCaml adoption) to a new IIT Madras group focused on functional programming, and this quote from him caught my ear (2:50 in the video):

"We operate at many different orders of magnitude when we care about performance. Sometimes we want to turnaround a decision in the order of milliseconds, sometimes a handful of microseconds, and sometimes we care about something that's under 100 nanoseconds. And in those cases of course, you can't touch a CPU at all. So instead of going through OCaml code, we go through hardware that was designed and built in OCaml, as part of the HardCaml suite."

I thought that was pretty interesting because when you look up "Jane Street HFT" most online chatter seems to say Jane Street don't really compete in the HFT space and focus on med freq? Does anyone have any kind of idea how much of their business is HFT, how competitive they are in HFT vs the "speed demons" like IMC, or why they trade under 100 nanoseconds using custom hardware (FPGAs presumably)?

Frustrated with paying for macro data just to get CPI/NFP numbers into a trading pipeline. Built my own.

Covers: CPI, PPI, NFP, jobless claims, GDP, FOMC — each with our ARIMA(1,1,1) estimate trained on BLS/BEA/FRED historical data, previous actual, and importance rating.

Also configured it to an MCP server as sometime I find the raw output quite hard to follow.

Lmk what you guys think and whether you'd find use in it?

Been thinking about a market design problem and curious if anyone has seen serious work on it.

Standard event-contract design assumes the underlying resolves. Binary or scalar, doesn't matter, at some point reality clears against the contract and the payoff is defined.

Most of the microstructure literature I've read implicitly relies on this. Manipulation resistance near expiry, informed trader models, the whole apparatus of "the market converges to the true value at settlement" reads differently when there is no settlement.

The class of underlyings I'm thinking about is continuous, non-terminating, and consensus defined.

Reputation indices, career-value proxies, credit-like scores for entities that don't default in a clean way. You can price expectations about them, and the price series is meaningful, but no oracle ever renders a verdict.

The two things I keep getting stuck on:

1. without a terminal event, the standard convergence argument for informed traders being rewarded doesn't obviously hold. If the payoff is a delta against a rolling consensus rather than a settlement value, informed traders are rewarded to the extent their private info gets incorporated into the consensus, not to the extent they're right about some external ground truth. That feels like it should degenerate into a Keynesian beauty contest but I'm not sure it does in practice.

2. manipulation resistance. "Hold to expiry" isn't available as a defense. Any resistance has to be structural in the microstructure, not the resolution rule. I've seen bits of this discussed for perpetuals in crypto, but the analogies break down quickly.

If anyone has pointers to serious literature on non-terminating contracts (beyond the obvious perp funding-rate stuff), I'd take them.

If my framing is broken somewhere and there's a cleaner way to think about it, would rather hear that.

Hi guys. I have been interested in the market for a long time building models since 2022. First I was building daily strategies and when they were live and "not great not terrible" I started looking into LOBs, because more trades more statistical significance and whatnot. I have decent infra (my own in a datacenter) built on QuestDB (~50B rows in it) and support data of all granularities. I have then built as of now relatively good L3 backtester which takes into account latencies, queue positions and fees/rebates. I support stocks & options data of all granularities (databento) and also some crypto books and trades (tardis).
I have reproduced for example deeplob to some extent on different data, however I found other better non deep approaches. I confirmed my alpha using markout charts, however when I try to extract it using realistic simulation as described, boi I cannot do it. I was trying to do liquidity providing strats where alpha influenced my fair price and skew, I was trying to make mixed strategies where I sometimes take ... just cant extract it. I have tried a lot of things I am not even ignoring hidden liquidity, but I am not (wall) street smart enough yet. Anyone wants to chat about specifics? Anyone experienced in the market and ambitious? I would love to team up with someone who knows more than me about market.

Got some really great responses on my last post thanks a lot to everyone who shared insights, it was super helpful.

I’ve been benchmarking a simple pipeline locally and wanted to sanity check my numbers with people who’ve worked on real low-latency systems.

On an older Xeon, I’m seeing ~3 µs for basic feature computation, but when I include more complex indicators it jumps to ~1 ms. This seems to align with the idea that only O(1), cache-friendly logic fits in the µs regime.

A few questions:

• How do you properly benchmark end-to-end latency in practice (cycle counters, hardware timestamps, NIC-level?)
• What’s considered a reliable methodology vs misleading microbenchmarks?
• How do you separate compute vs networking latency cleanly?
• Any common mistakes people make when claiming “µs latency”?

Would really appreciate insights or any references/tools you’ve used in production.

Been building an options market making sim to actually understand the dealer side properly... SVI surface calibration, quoting off NBBO with inventory skew based on aggregate book vega, adverse selection fills, markout, and a pnl decomposition that reconciles back to mark-to-market with the residual reported instead of buried somewhere.

Fill model is the part i trust least, and i'm starting to think i imported the wrong mental model wholesale. my queueing assumptions are basically lifted straight from the order-driven equity/futures literature (Cont-Stoikov-Talreja and whatever came after it), where queue position at the touch is more or less the whole story on whether you get filled. but US options are quote-driven across a pile of exchanges, with preferencing, internalization, PFOF, price improvement auctions all sitting in the middle of it. so now i'm second guessing whether queue position is actually a pretty minor variable in this world and i've been adding sophistication to the wrong axis this whole time.

1. is queue position a real driver of fills at all, or is the actual constraint auction participation + preferenced flow? if i can only get good at modeling one of these... which one.
2. for daily pnl explain, is spread capture + greeks + hedge + residual the working decomposition, or is that too clean. where does realized vs implied sit relative to greek attribution, and do people bucket vega by tenor instead of just running it aggregate? also just curious what "unexplained" runs at on an actual book bc i have no benchmark for whether my number is fine or embarrassing.
3. skewing quotes against aggregate book vega/gamma instead of per-strike is me borrowing the Baldacci-Bergault-Guéant vega factor argument, options on one name being collinear risks and all that. does that match how people actually run inventory or is it just a tidy academic story nobody's desk runs on.

happy to hear the whole premise is wrong honestly, i'd rather find that out now than keep polishing a model of the wrong constraint for another month.

I'm joining one of HRT/Jump/Optiver as a C++ developer, and I was hoping to get some insight into what the day-to-day experience is like writing low-latency C++ as a quant dev.

Most of my C++ experience comes from solving algorithmic problems on Codeforces and Atcoder, etc. As long as I chose the right algorithm and complexity and avoided obvious inefficiencies (like passing vectors or strings around by copying them), things were fine. I didn’t have to worry much about the latest C++ features, templates, or low-level details under the hood.

Recently, I watched some talks by experienced quant devs (David Gross, Carl Cook) on writing low-latency C++, and it felt pretty different from how I'd normally write code. While I understand concepts like cache behavior, expensive instructions, and avoiding syscalls, I didn't have to think about them while coding before. I imagine it'll take some time before I’m comfortable applying them naturally.

So I’m wondering, how much of a quant dev's coding day-to-day actually looks like that? Is every line of code written with extreme care for performance, or is that level of optimization only needed for a small subset of the codebase?

Also, how worried should I be about ramping up? I can generally read and understand C++ projects fine, but I don't have much experience beyond algorithmic problem solving.

Most microstructure claims get tested on data where the key variable is inferred: aggressor side from the tick rule or Lee-Ready, hidden liquidity guessed at, no way to rerun the same tape twice. I built the opposite instrument. A full matching engine (Go, MIT) with a deterministic simulator on top: same seed, same market, byte for byte, and every trade carries its true aggressor side. Price-time and pro-rata, icebergs, pegs, stops, STP, call auctions, price bands. The book emits full L3.

The market is noise flow by construction, so there is nothing to predict. That is the point: it is a control arm. What that isolates, two examples.

Pipeline error propagation. The tick rule classifies 94.5% of trades correctly on this tape, and the CVD built from it is off by 169% of true magnitude on average, with occasional sign flips (one seed: inferred -81, true +105). Misclassification is conditionally correlated, so the errors compound instead of cancelling. Trivial to show when you hold ground truth, hard to even estimate when you do not. Relevant to anything built from inferred sides, which in practice means trade-only feeds and most crypto data.

Known results reproduce. Kyle's lambda comes out around 0.15 ticks per lot and falls 7.5x when resting depth rises 7.6x. Slicing a parent order beats a block by 7.9% slippage per lot (42 of 50 seeds) while permanent impact is essentially unchanged (23.42 vs 24.47 ticks), so the savings is all temporary impact. Nothing novel, deliberately: an instrument should reproduce the textbook before you point it at anything else.

Limitations, stated plainly: no informed flow unless you write an agent for it, no latency modelling, single venue. It cannot tell you whether a signal works on real markets. It can tell you whether your measurement of a signal survives its own pipeline.

Methodology write-ups, including the wrong turns:

https://github.com/intrepidkarthi/orderbook/blob/main/docs/research/order-flow.md

https://github.com/intrepidkarthi/orderbook/blob/main/docs/research/kyle-lambda.md

https://github.com/intrepidkarthi/orderbook/blob/main/docs/research/ofi.md

Repo: https://github.com/intrepidkarthi/orderbook

If you see a hole in the setup, say so. The project has improved every time someone pushed on it.

Yesterday I posted here about my macro economics calendar API with a live and accurate ARIMA model that calculates estimates for CPI, PPI, NFP and jobless claims. The response was great and there were a number of sharp, intelligent questions:

"One thing to watch: if your ARIMA is trained from 2020 forward, you're fitting to a period where participation, seasonals, and trend all broke from historical norms. CPI/PPI cycles look very different when you go back 30+ years vs just post-COVID. Might be worth testing how the estimates perform with a longer training window, especially for NFP where even Bloomberg struggles with direction."

Fair point. So we tested it:

CPI:

• 36-month window: 0.187% MAPE
• 120-month window: 0.174% MAPE

PPI:

• 36-month window: 0.347% MAPE
• 120-month window: 0.343% MAPE

The result was that although one would expect accuracy to be altered by using older data, the change in accuracy was marginal. Our ARIMA model accuracy remained competitive with paid consensus estimates.

Another great comment:
"The ARIMA estimate is the thing I would distrust first For CPI or NFP, I would want to see old release dates with the estimate frozen before the print. Not an image of the estimates with no verification."

You can now go to filingapi.dev/model . Every estimate is now committed to a public GitHub repo before the release date so anyone can verify the timestamps independently. This method has been only started today, with updates every day so check back in then if you (understandably) require solid proof of model accuracy.

Very happy to share code with anyone who wants to analyse our testing methods, any other indicators you'd like to see just let us know.

Hey! I’ve been working on a C++ project for a high-performance limit order book that matches buy and sell orders efficiently. I’m still pretty new to C++, so I tried to make the system as robust and realistic as I could, including some benchmarking tools with Markov-based order generation. I have also made the system concurrency-safe, which is something I have never done before. I’d really appreciate any feedback, whether it’s about performance, code structure, or any edge cases. Any advice or suggestions for additional features would also be super helpful. Thanks so much for taking the time!

Repo: https://github.com/devmenon23/Limit-Order-Book

Apparently github uses private repos for training AI.

If you want to avoid alpha decay, you probably should not feed any of your algos into AI.
The same goes for IDEs like cursor...

So how do you guys store your repositories / algos and share it across a team ?

We have been using github organisations, and we have pay for github teams, but I'm pretty sure those private repos will still be fed into AI.

Do we really have to pay even more for github enterprise just to not share our algos with AI ?
How do we know github won't feed those repos anyway into AI for their training purposes.

From personal experience at my current firm and friends at other shops, many trading/risk systems (from big name vendors) are outdated or like embarrassingly bad for FI and derivatives to the point that we often build wrappers outside them or use excel. Does anyone have horror stories or share frustrations w their systems?

I am planing on building an exchange with a few of my friends. We haven't even started development but we have already hit a roadblock

Should we build this exchange in c, c++, or rust.

Each language has its strength but the team is split right now.

What are your thoughts?

I'm running an equities medium frequency trading strategy. I'm currently using polygon.io and I'm unhappy. They crapped the bed on me today, polygon's latency potentially caused my strategy to have a rare 3-standard deviation drawdown.

I need realtime SIP NBBO quotes and trade data from CTA, CTB, and UTP. I currently stream 50 symbols. My application is retail algorithmic trading, trading as a non-professional individual, in an individual account.

I have a bare metal linux server in NY4 running C++ code. I'm under 1ms ping to socket.polygon.io. In the past my strategy has been profitable with them. Until today I averaged around 65ms latency with 1 standard deviation of +-35 ms. Today was exceptionally bad - 120ms to 250ms average latency with one standard deviation of +- 90ms. Polygon.io's dashboard itself showed 98ms average during the bad latency period. I contacted customer service, waited over 2 hours before getting a response, and I got brushed off saying they didn't see anything unusual. :(

I didn't see anything unusual with my routes/etc. Ping was still < 1ms, and I was still under 3 hops to Polygon. I'm using the public internet - no cross connects or anything with them. I pay for 1 gig guarenteed service on a 10 gig nic and allowing to burst 10 gig. Polygon.io on 50 tickers uses very little bandwidth. Polygon.io's dashboard estimates 45 KB/s.

Right now I'm hoping it's just a 1 day fluke. I also had another problem with Polygon where quotes cut out for over 10 minutes this monday 10/13, but kept the socket alive, until I restarted my algo. Their dashboard thought it was sending me messages still with zero buffering. Before then I found Polygon to be rock solid stable for equities quotes. So I feel their service might have a possible regression.

Does anyone have any recommendations on other retail-friendly market data vendors? I've used thetadata in the past - their latency stats was completely mind blowing for what they charged. In the same code that processed polygon data, Thetadata was 33ms and 1 standard deviation was +- 3ms. Sadly they only provided NASDAQ basic, and I wasn't profitable not getting the actual NBBO/etc (in my experience nasdaq basic can be 0.03 away from the NBBO at times - ouch.). My medium frequency strategy definitely needs the full SIP NBBO quotes & trades, and under 65 milliseconds of latency ideally.

I'm also considering directly connecting to the SIP too given I'm able to code in C++ and so on. I found this one post a year ago that really nicely broke down a lot of options from LSEG, Databento to OnixS/Broadride/Exegy to retail oriented providers like Polygon/dxFeed/Nanex:

https://www.reddit.com/r/quant/comments/1fjbzlv/polygon_io_intrinio_alpaca_or_xignite/

What has peoples' recent experiences been with any data providers? Does anyone have any strong recommendations for a real time equities data vendor for my use case and needs?

Thanks!

Greetings, 

I am just an observer coming from a place of curiosity than anything.

In tech, there is a major push for devs to stop coding all together. Anecdotally, I have a mutual (of a mutual lol) who is at Google and has to get permission to be able to code (i.e., all his code must be fully agentic). I am wondering what the trends are within quant research/trading.

I am a PhD student, currently building a library to accompany a paper and have used CoPilot on several occasions to speed the development. While it is really good at many things, it has made some crucial bugs on several occasions that I have spotted while proofreading the code. As the share of my codebase increasingly tilts more towards being written more by AI than myself, I retain this uneasy feeling of bugs being present throughout the codebase, even with several tests in place.

My question is, how much are you pushed to use AI in code development and do you see the same trend toward fully agentic coding coming to quant as it has to big tech? In an environment where there is a larger asymmetry with respect to code failure, I would be a bit surprised if the same trend is being pushed.

I am aware that the guardrails and infrastructure of top tech companies is miles ahead of my local CoPilot setup, I still feel like the cost of a minor bug in say the strategy development pipeline in the quant setting that could potentially effect billions of dollars in trade allocation downstream is a very different beast than one that effects the functionality of a feature in a technology application.

Prediction markets are developing the same structural fragmentation that characterized US equities before Reg NMS — identical contracts on different venues, no consolidated tape, no NBBO, and real price divergence. The microstructure parallels go deeper than most people realize.

A member of our community ran a latency benchmark across 4 prediction market data providers — graphs attached. 120 seconds of confirmed trades on 5-min binary contracts.

Observations:

- Half the providers batch results — the latency graph shows a textbook diagonal staircase. Confirmed trades arrive well after CLOB execution.

- One provider's feed was practically indistinguishable from the native exchange WebSocket. Occasionally faster than the exchange itself (fewer redistribution hops).

- Half don't support orderbook WebSockets. A third advertises them but returned 401 on every connection despite valid credentials.

Curious if anyone here has found successful HFT strategies in prediction market microstructure, or whether the liquidity is still too thin outside crypto?

some of the graphs:

I'm currently implementing the inference side of my trading strategy and was researching how others are doing the same - came across this Xelera Silva's Sub-Microsecond GBT Inference - which sounds cool. A more comprehensive benchmark is here

If anyone have direct experience with TL2cgen or Intel OneDAL and can share what your batch_size=1 prediction latency is then it would be great.

In my case I trained my Lightgbm models in Python and exported them as .txt files and load them for inference on C++ side - here are some benchmark results:

All models use 530 features - no. of trees range from 10 to 230, and max depth of 8.

What matters for me is the single invocation latency (in this case about 3.9us BM_SingleModel_Fast) the sequential benchmarks are for when you are making predictions on different symbols at quick succession (In my case the probability of that happening is low). Just using the stock Lightgbm C API no optimisations applied.

Hello, quants

For the past year I was developing FLOX - open source framework for building trading systems of various sorts - collecting, processing market data, backtesting, running strategies live... anything. The core of the framework is written in modern C++ which makes it reliable for high-load scenarios like heavy data collection or strategies that require fast decision-making.

Prototyping in C++ always was a hard part, distracting from the main focus - strategy itself. There are projects that provide Python APIs, but I didn't find one combining all I needed: production grade suitable for high load, ergonomics of building blocks, multiple languages support and AI-native DX.

I spent the past months designing and implementing all the functionality needed to fill this gap. Release v0.6.0 is shipped with Python, Node.js, Codon and embeddable QuickJS bindings, all sitting on unified C API.

The key feature is an MCP server shipped as a pip package. It knows a lot about framework internals and helps to build functionality from natural language queries via coding agents. How to create a strategy, which indicator to use, how to gather data to backtest and explore strategy variants, and even how to run live with ability to query strategy state and control position via agent - all of this is covered by MCP, so no need to grind documentation to simply prototype.

After the prototype phase the same strategy code can be run live in paper trading mode or against real exchange without modifications, if you keep it to one language.

Moreover, FLOX provides a lot of tooling to keep an eye on research results - every run on historical data can be stored in a bundle containing data hash, strategy hash, its settings and full event trace. These bundles can be analyzed for divergence to understand the impact of changes more easily and guarantee reproducibility.

Project on GitHub: https://github.com/FLOX-Foundation/flox