In 2008, a team of quants at JP Morgan set out to automate the hedging of one of the firm’s derivatives portfolios. The effort was quickly abandoned. The hedging strategy—which relied on computing risk sensitivities, known as Greeks—required constant manual adjustments to account for transaction costs and other market frictions, which are not captured in classical quantitative models.

Some years later, they tried again, this time using machine learning. The new system eschews conventional modeling techniques such as Black-Scholes and replication—essentially, everything quants have been doing for the past 45 years—in favor of a purely data-driven approach.

The results will surprise no-one familiar with recent advances in the field of artificial intelligence. The machine learning algorithm far outperformed hedging strategies derived from existing models.

JP Morgan began using the new technology to hedge its vanilla index options books last year and plans to roll it out for single stocks, baskets and light exotics next year. Bank of America, Societe Generale and Standard Chartered are exploring similar applications of the technology.

Hans Buehler, global head of equities analytics, automation and optimization at JP Morgan, was one of the co-authors of a recently published paper on deep hedging. The research is part of an ambitious project at the bank aimed at using machine learning to hedge positions multiple time-steps ahead. Buehler has discussed the research in a series of talks, and in a podcast with Risk.net, which have captured the imagination of quants.

“I listened to [a talk by Buehler]… and I got pretty excited about it,” says Mark Higgins, chief operating officer and co-founder of Beacon Platform, a New York-based financial technology company. “The right way to think about it is that it’s addressing some of the weaknesses in the current way people approach derivatives pricing. Rethinking it from first principles, really.”

Higgins was co-head of JP Morgan’s quantitative research team until 2014.

Deep hedging is already being heralded by some as a breakthrough in quantitative finance, one that could mark the end of the Black-Scholes era and usher in a future of “model-free pricing”.

“This is something I have argued for several years now,” says Alexei Kondratyev, head of the data analytics group at Standard Chartered. “The future is working with data directly, with empirical distributions directly, rather than trying to invent a new parametric model every time and fit it to historical data.”

I listened to [a talk by Buehler]… and I got pretty excited about it. It’s addressing some of the weaknesses in the current way people approach derivatives pricing. Rethinking it from first principles, really

Mark Higgins, Beacon Platform

At JP Morgan, that future has already arrived. Several other financial firms, including Bank of America and Societe Generale, are getting there.

“We are growing in this direction," says Daniel Giamouridis, global head of scientific implementation in the data and innovation group at Bank of America. "The plan is to engage as much as possible within the business with these techniques."

The bank began testing the application of machine-learning techniques for hedging complex, cross-asset portfolios a few years ago. The benefits were deemed to be “economically material enough” to justify its use. “The trade-off between complexity and advanced modeling, and the benefit obtained, makes it worth pursuing this methodology versus something that is more linear,” Giamouridis says.

Societe Generale is already using machine learning to pick stocks for its quantitative investment strategies. “Machine learning is a big topic for us,” says Sandrine Ungari, the bank’s head of cross-asset quantitative research. “We have looked at it quite a lot from an underlying perspective. We do have a machine learning algorithm that runs for picking stocks. We are looking at a wide range of fundamental data on corporates, like balance sheet data, earnings data and analyst sentiment data.”

“The next step,” she says, “is in applying machine learning for having a better hedging strategy when selling options systematically.”

Technology firms, both old and new, are also getting involved. IBM is working with a group of banks, hedge funds and pension funds to test a machine learning system for hedging equity portfolios using options. Donna Dillenberger, a fellow at IBM in New York, says clients involved in the project saw accuracy improvements of 25–30% compared to classical models, while the reduction in hedging costs was in the double digits.

Beacon Platform is trying to apply deep hedging to commodities and variable annuities. The company estimates the technique could reduce the cost of hedging some commodity exposures by as much as 80%.

The use of machine learning for pricing and hedging has big implications for the way markets and traders operate. A senior structurer at a European bank says the concept of a mid-price—the average of the bids and offers in the market—is “dead” if banks adopt data-driven pricing. Each bank would quote its own price, based on the information available to it. And that price could vary considerably if banks use proprietary or alternative data sources.

Traders will also need to learn new skills. “People who today spend their time adjusting for the deficiencies in classical Greek-type models now need to understand how the statistics work,” Buehler said in the Risk.net podcast. “If the machine proposes a trade that may not be intuitive, the question is why? The traders need to be able to understand how the machine comes up with a particular answer, and if they believe the answer is wrong, how do they adjust this?”

JP Morgan is training its staff to use Python and has made Jupyter notebooks—a web application that is used to create and share data analysis—available to trading desks.

How Deep is Your Hedge?

The Black-Scholes model—developed in 1973—has been the de facto standard for pricing options and other financial derivatives for nearly half a century. Traders value their puts and calls by entering five variables—the price and volatility of the underlying stock, the strike and expiry of the option, and the risk-free rate—into the model.

The formula can be used to calculate the so-called Greeks, an option’s sensitivities to various risk factors. With this information, it is theoretically possible to create a perfect hedging strategy that eliminates all risk in a portfolio of options.

For all its elegance, Black-Scholes is far from perfect. It assumes price moves are random and normally distributed, and that volatility remains constant over the life of an option. Market frictions such as transaction costs are ignored.

These simplifying assumptions, which are inherent in classical models like Black-Scholes and the Heston model—another popular stochastic volatility model—have long troubled quants.

“As a quant for investments, I really care about the real dynamics of prices,” says Societe Generale’s Ungari. “We have seen in the past these classical models being challenged by market participants. There have been cases where you had huge market disruptions, huge volatility in the market, and classical models failed to provide accurate hedging of books of options. We have seen environments where classical models such as Black-Scholes and Heston failed to prescribe the right hedging strategy. This is something as a quant investor you are very much aware of.”

We have seen environments where classical models such as Black-Scholes and Heston failed to prescribe the right hedging strategy. This is something as a quant investor you are very much aware of

Sandrine Ungari, Societe Generale

The failure of classical models to fully explain the behavior of asset prices has inspired a huge body of academic literature on ‘incomplete markets’, which explicitly accounts for market frictions and other real-world constraints. However, the sheer amount of data required, and the number of competing explanations for empirical asset price moves, made it difficult to put these theories into practice in a standard way.

Buehler has described deep hedging as an application of the theoretical understanding of incomplete markets with machine learning. The idea is that by relying exclusively on empirical, data-driven analysis, rather than simplified assumptions and approximations, it is possible to create more robust and realistic simulations of markets that evolve over time.

“Machine learning allows us to, first of all, consider a broader set of possible risks or possible factors that can formulate the overall risk of these portfolios,” says Bank of America’s Giamouridis. “It allows for the modeling of more complex interactions, and enables us to generalize better, resulting in potentially better accuracy on unseen data, out of sample. They are also more robust against situations where the factors happen to be closely related.”

In their paper, Buehler and his colleagues modeled hedging strategies using neural networks, a type of artificial intelligence that can learn to perform complex tasks by identifying patterns and relationships in large volumes of data.

The system is first programmed to recognize information relevant to hedging. These so-called feature sets include not only the prices of hedging instruments but also trading signals, news analytics and past hedging decisions—the sort of information a human trader might use to formulate a hedging strategy.

[The machine] knows the payoffs from derivatives. But it knows nothing about Black-Scholes, it knows nothing about the deltas, gammas, vegas, and so on. It just does trial and error

John Hull, University of Toronto

The algorithm teaches itself to hedge by studying this information. It runs statistical regressions to find patterns and relationships between different variables, and extracts rules and strategies from these observations.

“The respective algorithms are entirely model-free,” the authors write. “This means we can include market frictions such as transaction costs, liquidity constraints, bid/ask spreads, market impact, etc.”

Finally, a technique called reinforcement learning is used to train the machine, through a large number of simulated trades, to find the best possible hedge in any given scenario or market environment.

“If you have something you want to hedge, you give the machine a lot of data and it learns,” says John Hull, professor of derivatives and risk management at the University of Toronto’s Rotman School of Management. “It knows the payoffs from derivatives. But it knows nothing about Black-Scholes, it knows nothing about the deltas, gammas, vegas, and so on. It just does trial and error. The way this learning works is, you develop a strategy and then you improve on it and improve on it.”

Beacon Platform is applying these techniques to find optimal hedges for commodities and variable annuities. For instance, a company that uses natural gas storage facilities could hedge its exposure to commodity prices at the storage location, where costs might be high, or take some basis risk and hedge at a more liquid location. Deep hedging could be used to figure out the optimal distribution of hedges between these locations.

The research is still at an early stage, but Higgins says it is already attracting interest from commodity firms. He estimates that for natural gas storage, deep hedging could lower transaction costs by 50–80%, depending on the market structure, while reducing hedging errors by 50–90%.

Higgins says the same techniques could also be used to more accurately hedge variable annuities sold by insurance companies, which carry hard-to-hedge risks such as mortality and early redemption.

Monte Carlo, Fast

Deep hedging is one of several applications of machine learning in derivatives markets. Another promising use-case is pricing more complex instruments that typically require cumbersome Monte Carlo simulations, where a product is valued thousands of times under various scenarios to arrive at an average price.

Standard Chartered’s Kodratyev gives American options as an example. Unlike European options, which have a fixed expiry and can be priced using Black-Scholes, an American option can be exercised early. This means its value is dependent on a number of factors that evolve over time, such as the price and volatility of the underlying, and when the option gets exercised. To price these options, a bank would run thousands of scenarios in a Monte Carlo simulation, which can be time consuming and computationally intensive.

With machine learning, a neural network can be trained to do most of this work ahead of time. It can then be used to price derivatives in real time when the products are being traded or risk-managed.

“Normally, when we run a Monte Carlo simulation, we need to first generate new scenarios and then we would need to revalue payoffs on these scenarios [which] can be quite time consuming,” says Kondratyev. “[With machine learning], once the network is trained, any subsequent pricing function call is almost instant. So instead of building a model, calibrating the model, and then using the model for pricing, we can spend some time learning the approximation, but then we have pricing effectively for free.”

Once the network is trained, any subsequent pricing function call is almost instant. So instead of building a model, calibrating the model, and then using the model for pricing, we can spend some time learning the approximation, but then we have pricing effectively for free

Alexei Kondratyev, Standard Chartered

The University of Toronto’s Hull says machine learning may be able to approximate Monte Carlo results one to five thousand times faster than current methods.

Standard Chartered is currently researching the use of machine learning for this purpose.

Kondratyev says machine learning could also be used to improve value-at-risk calculations, which provide an estimate of how much a portfolio might lose with a given confidence level over a certain period of time. VAR modeling is hampered by sparse data. At a 99% confidence level, only 1% of losses over an observation period should exceed VAR—so in a year with 250 trading days, there will only be 2.5 relevant observations, which is not enough for an accurate reading.

“When the 99th percentile of the portfolio value change is calculated, the error in VAR numbers can be huge,” Kondratyev says. “It can easily be 20% higher, or 30% lower because there is only one particular realization of history and really a small number of data points.”

He says machine learning can solve for this problem by generating ‘synthetic’ data. An algorithm could be trained to study the available data and then create new distributions with similar features. This way, there would be more data points to model VAR.

“If a machine can learn an empirical distribution and generate more samples from this distribution, then you can have as many samples as you wish,” says Kondratyev.

The same principle can be applied to approximate sensitive client data. If the synthetic dataset cannot be linked to a particular client or portfolio, it will no longer be considered sensitive, and can be used by quant teams for modeling purposes.

“We can’t share client data at all, even if we try to anonymize it,” says Kondratyev. “But if we can generate synthetic data—data that comes from the same distribution, but is not real—then this becomes a dataset we can share. We can perform analysis on this data without breaching any restrictions or compromising on privacy.”

Data drought

Synthetic data could remove one of the main obstacles to letting the robots loose—the need for large datasets to train the algorithms.

Machine learning works best for liquid, listed products, like stock options. Even then, banks spend a considerable amount of time collecting and organizing large sets of trade and market data so it can be used to train new algorithms. One quant says his firm has been collecting data for years to use in machine learning. But the approach struggles with sparse data and illiquid products, which tend to be the hardest to hedge.

“One of the challenges for this technique is that the amount of instruments we have in the market—vanilla options, forwards, discount factors—compared to the actual time series of data which are relevant isn’t that large,” Buehler said in the Risk.net podcast. The answer, he said, “is to build market simulations, ways of simulating the distributions such that you take care of sparsity.”

While JP Morgan is already using machine learning to hedge vanilla options, “over-the-counter derivatives is still something we work on because you need much longer time series for the underlying statistical measure,” Buehler said.

The machines have other limitations. Unlike classical models, they cannot extrapolate beyond the data available at the time they were trained. “It will only work within the range of the data that you trained it on. So you train your model with a certain range of volatility parameters, and range of asset prices, and all the other inputs for whatever product you’re talking about.

But if you move outside that range, you can’t extrapolate,” Hull says. “It’s not like a mathematical model that often extrapolates reasonably well.”

This means the algorithms would have to be retrained every time a portfolio changes, which can be cumbersome. However, the quants at JP Morgan believe this issue can be overcome. “We’re trying to come up with a formulation where you can train it—it takes two to three weeks—and once it is done, you can use a fixed model,” Buehler said in the podcast. “That’s classic reinforcement learning. Unless the market changes fundamentally, you should be able to reuse the model.”

The inability of machine learning models to extrapolate is one reason why classical models like Black-Scholes are unlikely to be done away with any time soon. For instance, while JP Morgan has adopted deep hedging for options, it still uses Greeks to set risk limits.

The other big hurdle is interpretability. Machine learning algorithms make decisions by crunching through millions of data points, which makes it hard to pinpoint exactly how it comes up with specific answers, or explain why something went wrong.

“It gets very complicated to know why the model is prescribing you to buy that amount of underlying stock, let’s say, to hedge your options,” says Societe Generale’s Ungari. “Traders who are managing the book need to understand why that delta is changing so much from one day to another.”

At some banks, machine learning experts are working with experienced traders and risk managers to better understand and trace the inner workings of these algorithms.

“Domain expertise is vital in the early stages of collecting the important variables and in making sure the analysis has financial grounds, while machine learning expertise is needed in the iteration loop of understanding the results and coming up with improvements on the models to achieve higher accuracy,” says Giamouridis. “If what the model tells you is not aligned with economic intuition, that means there may be a problem. There needs to be a proper implementation protocol justifying why insights should hold out of sample.”

Supervised machine learning techniques, such as deep hedging, tend to be easier to explain. The training data is filtered by human experts, which sets some limits on the machine’s potential outputs. “At the moment, everything we do is feature based. We don’t do full black box in the sense that you show it a data stream and ask the machine to find the important features. We tell the machine what features we think are important. That alleviates the problem somewhat,” Buehler said, adding: “Maybe one day we might be able to go to real deep learning.”