Trading algorithms have been a key part of investment firms’ toolkits for more than two decades. And for almost as long, the biggest sell-side firms have offered suites of algorithms to their tier-two bank clients and buy-side firms, with Credit Suisse’s AES business and Deutsche Bank’s Autobahn platform being early pioneers in this space.

But now, smaller firms and startup brokers are driving the biggest innovative leaps in algo development—and the latest innovation is algos that encompass more than one trading strategy, and can dynamically adapt to changing liquidity environments.

Though it is the oldest of the firms mentioned in this article, agency broker and algorithmic technology shop Quantitative Brokers (QB) didn’t even exist when Credit Suisse began selling algos to customers. Founded in 2008 by CEO Christian Hauff and Robert Almgren—co-author of the 2000 “Almgren–Chriss” paper detailing a model for optimal execution that became the market impact framework for much of Wall Street—the firm recently announced a two-in-one algo that it claims is a first for trading options on futures.

The algo, Striker 2.0, is an evolution of its original Striker tool for outright options trading, which was launched in April last year. The new algo supports trading in separate liquidity pools—the listed markets, and in a request-for-quote (RFQ) mechanism operated by CME Group, dubbed User-Defined Spread (UDS).

“Think of it as two distinct markets for the same instrument. You can overlay one over the other in real time to create a co-mingled book … so you know that some levels of pricing are available via one mechanism, and other levels are available via the other mechanism,” says QB’s Hauff. “We can make Striker 2.0 interact with both listed liquidity and opportunities and pricing information from those, but also implied pricing from listed outrights or implied options. But then you get a unique opportunity for price improvement by leveraging this user-defined pricing that exists on CME in the form of covered options and listed spread order types—so you can also interact with those UDS options.”

He says the ability to do this is a direct result of QB’s pre-trade transaction cost analysis (TCA) capabilities, which can perform TCA in real time on all Striker orders.

“What’s groundbreaking is that this is our first algorithm that encapsulates what we’ve been working hard to mechanically perform what we’ve been doing for a long time here at QB,” Hauff says. “Our vision is to be able to trade all request-for-quote or request-for-stream liquidity across the spectrum, and this is the first step.”

The result is improvements in performance and price, and also in productivity. Specifically, Hauff says Striker 2.0 has saved clients $5.50 per lot, which he says is a significant amount for those executing large options trades.

The productivity savings are harder to quantify, but easy to see. Instead of humans manually retrieving quotes, and performing calculations in spreadsheets, “we’ve automated all of that to make it more precise by systematizing the actions and decisions more efficiently,” and cutting down the number of people required, Hauff adds. “Where multiple brokers and traders used to be involved, we have automated and simplified the whole process from quoting to executing.”

Reinventing the ‘wheel’

But QB isn’t the only game in town—at least not in equities markets, where others are evolving their algos to deliver better returns. Driving this is increased adoption of “algo wheels” at buy-side firms that split orders between different broker algos, says Hitesh Mittal, founder and CEO of BestEx Research, a specialist algo development firm created in 2017.

“This is a new way of doing things, and I’m sure others are starting to do it. So as more buy-side firms start building algo wheels, sell-side firms will have to react to that and make systematic improvements to their systems,” Mittal says.

The firm has also recently embraced the idea of a two-in-one algo for its equities algorithm (BestEx Research also provides algos for futures and foreign exchange markets), combining a scheduled algo and a liquidity-seeking component, and claims it has yielded significant improvements in execution performance.

“In the equities markets, there are schedule-based algos like VWap (volume-weighted average price) and liquidity-seeking algos. Scheduled algos don’t do a good job of executing liquidity in dark pools, whereas liquidity-seeking algos do a good job in dark pools but not on-exchange. So the typical approach is to start an order using a liquidity-seeking algo, then if that doesn’t finish the order, switch to a scheduled algo,” Mittal says.

BestEx Research’s approach was to create a schedule-based algo with a built-in liquidity-seeking element for accessing off-exchange liquidity. “This liquidity-seeking layer can be following a VWap schedule but also access liquidity in dark pools. It reduces market impact and doesn’t show your whole hand to exchanges, because you can execute 50% in dark pools without violating client instructions,” Mittal says.

Of course, these improvements don’t come without a lot of effort, specifically around testing and simulating how an algo would perform in real-world scenarios, using real market data.

“If I’m being honest, not every change you make results in an improvement, so we don’t make a splash with something new until we can show that it makes a significant improvement,” Mittal says. “We’ve built an A/B test framework, and over the course of a couple of months, we can determine if a change has made enough of a difference.”

Testing times for timing

But there are challenges to building a platform to accurately back-test algos and how the market would react in real life, rather than in a closed environment. One challenge is replicating the exchanges’ platforms and rules. Another is adjusting timing to allow you to test days’ or years’ worth of data in minutes or days.

“Algos are very dependent on time. For example, an algo may be set up to wait 20 seconds between placing orders, so you have to make sure your algos, your trading simulator, and your timing infrastructure are all set up to run at the same speed. All those things need to be perfectly synchronized, and not still running at normal clock speeds while your test data is speeding past,” Mittal says.

It was a challenge he’d tried to solve in previous roles at trading firms. At BestEx Research, his solution was that no systems use their operating systems’ timers at all, but rather use an extraction on top of the OS—a “timer library”—that pulls timing information from the dataset being tested, so that all systems involved are pulling definitive timing information from the same source, without changing any major code. “It’s a simple trick, but if you can’t do it on day one, you won’t solve it,” Mittal says.

And even that’s not the be-all and end-all. Testing can be a lengthy as well as complex process. “Simulations don’t tell you everything,” Mittal says. “You can test how much of the bid-ask spread you can capture and adverse selection—how much you get picked off—in a simulator. But you can’t test market impact in a simulator. So you have to do this third step after a testing simulator. An all-day VWap may take months before you know if a new algo is better than your previous algo.”

There’s good reason why combining a dedicated liquidity-seeking capability with scheduled algorithms makes sense—to protect some algos from their own shortfalls in certain circumstances, says Joseph Urban, product manager for electronic execution services at Clear Street, a three-year-old agency brokerage startup.

For example, VWap does contain some liquidity-seeking capabilities, but that’s not always a good thing. “If you fall behind your fill rate, VWap will cross the spread to catch up, but will add costs. To stay on schedule, any scheduled algo will go and get a bad fill if it can’t get a good fill,” Urban says.

Not only that, but once other market participants realize you’re playing catch-up, they can take advantage of that and can drive up market prices, destroying any chance of minimizing market impact.

“You have to very carefully navigate the order books. We’re focused on expertise that allows us to execute opportunistically—for example, passive fills where you get the rebate and don’t pay the spread. You can soak up liquidity during favorable conditions and avoid it during unfavorable ones, especially when you can identify exactly where every trade is going off within the spread and when you have the capabilities to place order anywhere within that spread. That allows you to add a liquidity-seeking element in any circumstance,” Urban says. “I do a lot of work looking at fills. You have to remain very sharp and build out the discipline so you can navigate those pools fairly anonymously and meet those fills.”

Urban says opportunistic trading will become embedded in all Clear Street’s algos, and that the firm plans to build a “menu” of algorithms so that clients can take the outcome they’re seeking to achieve and map it to that menu, based on their requirements and mandates, which can be a gating factor to allowing more opportunistic execution.

“Some clients have very strict mandates or may be benchmarked to something, so they have to execute a certain way,” he says. “Clients need to be flexible to be able to take advantage of the kind of opportunistic activity I’m talking about, and they need to let us take the full parent order and maneuver it between datacenters to find liquidity. You have to know the market, the regulation, and the datacenters [that host liquidity pools] … and you have to be able to show clients the data until they trust you.”

‘A whole field of work that’s going to explode’

Daniel Aisen, CEO of startup broker Proof Trading, says the concept of “dynamic” algos isn’t new—some firms had algos that could switch between strategies automatically while he and other IEX founders were still working at RBC, long before he started Proof. The difference is that it’s becoming more important as clients’ trading strategies evolve and become more sophisticated.

“Algorithms are tools. Every trader has their own toolbox and approach. Maybe they use one algo for 10 minutes then switch strategy—and they want an algo that can do that for them. So you want to be able to offer a tool for their workflow that delivers their goal,” Aisen says.

Thus, the firm’s second algorithm, which it has been testing and plans to launch next month, later than originally planned, will be a hybrid liquidity-seeker and impact minimizer, developed in-house based on the research of Proof’s president Allison Bishop, rather than based on previous models (you can read the detailed 57-page whitepaper explaining the new algo here). The algo first seeks out large blocks of liquidity to execute large chunks of an order against. Once it has picked off large blocks—or if it can’t find any—the algorithm steadily executes trades under the radar to avoid any large impact on price.

“Basically, every impact-minimization strategy on the street is a front-loaded VWap based on Almgren–Chriss. Ours is a new approach that stitches together different strategies to achieve optimal execution,” Aisen says.

Sam Clapp, executive director and head of electronic trading sales at Mizuho in New York, agrees that the concept of dynamic algos isn’t new, but says new technologies such as artificial intelligence are making it a reality.

“It’s been talked about for years. Fifteen years ago, there were conversations about algos that could change strategies based on market conditions, but the technology wasn’t there yet. A decade ago, an algo may be able to change speed, but now you’re talking about changing strategy completely,” Clapp says. “We’ve been talking a lot about AI and what we do, and how we can use AI to make our algos more intelligent.”

For example, the bank developed a tool to predict volume changes within the next five-minute time period. Its clients take the data output of those predictions and incorporate it directly into their trading models. Originally, Mizuho used 20-day average volume to generate its predictions. But after introducing its new, AI-driven model, its tracking error improved by 20%, leading to a corresponding improvement in the quality of its predictions.

Clapp says most clients aren’t asking about strategy-switching algorithms yet, because Mizuho’s main algo business is centered on the Japanese market, where clients are more concerned about finding and accessing liquidity, and where trades are generally tied to VWap rather than arrival price.

He says Mizuho does intend to build its own platform for the US market, to replace a white-labeled third-party offering that it currently uses, leveraging and customizing the logic from its Japanese algo platform—but adds that any such initiative is still a couple of years away because of the client volume required to justify the investment in an algo trading platform.

“The cost of building an e-trading platform has increased over the last 20 years, so it’s not something you want to think about lightly. If you’re not among the top 15 brokers, you don’t want to spend that money just to have a ‘me-too’ product. And we know we’re not there yet,” Clapp says. “We have other areas of the business we want to develop first. But as Mizuho moves into the higher rankings with buy-side firms, we’ll be in a position to build our own US product—but that’s probably still a couple of years out.”

Doubtless during that timeframe there will be more new approaches as players across the industry grow and evolve their offerings. These efforts are just the start of new initiatives in this space, says Hauff at QB, which has developed a series of “regimes” to define different market environments and serve as the basis for trading decisions. “Using VWap versus liquidity-seeking is our ‘regimes’ work. That will be a whole field of work that’s going to explode,” he says.