Vanguard cautiously explores neural networks for alt data analysis

John Ameriks has seen significant movement and change in the data landscape since joining the asset manager in 2003, and he’s done his best to ensure Vanguard’s Quantitative Equity Group has moved and changed with it. 

QEG has 30 members, including portfolio managers, analysts, and QEG’s quantitative development and innovation team, which is in charge of sitting in between the portfolio managers and analysts and integrating new technologies into the trading process. QEG’s alpha side looks at analyst reports in order to determine which characteristics make datasets attractive, as well as which data vendors to buy the attractive datasets from.

“We look at tech firms within the tech space, we look at pharma firms within a pharma category, and we’re really trying to figure out what firms have a set of attractive characteristics that we think will outperform over the long term,” says Ameriks, global head of QEG. 

Alternative data, which refers to non-traditional sources of information beyond market data and reference data, has been prized by buy-side firms for years due to the unique benefits it can yield compared to traditional datasets. When hunting for new datasets, asset managers like Vanguard want options that give them an edge over their competition, but that also have enough history to be usable over a longer period. 

“We stay away from—and probably don’t have a lot of utility for—things that are very, very short-horizon,” Ameriks says. “We don’t pay attention to information that has a very, very short half-life in and of itself; we want to try to get to things that are going to help us suss out fundamental differences in the attractiveness of firms over longer periods of time.”

Ghost in the machine

One of QEG’s largest innovations in its alpha generation process was the introduction more than a year ago of a neural network-driven process into its alpha generation engine. Ameriks says this works as an overlay across the traditional investing process and aims to increase the portfolio managers’ efficiency. However, the decision to use AI at Vanguard was met with skepticism internally, not least from Ameriks himself. He describes the process of incorporating AI within QEG as “restrained.” 

“I have just as much skepticism about machine learning and AI as anyone in the industry has,” he says. “I mean, these things were designed to fit data [inside them]. It’s very easy to go back [and analyze] a historical dataset and there’s no uncertainty in a historical dataset. All the uncertainty is in going forward. And it is just a matter of an algorithm trying to fit a historical curve. The big question is, how well is that going to fit the world that arises in, where uncertainty is important going forward?”

Ameriks says that within QEG, there was equal focus on working out how to continuously re-examine and intervene in the AI’s work and on observing the kind of work it would do if it were free of human interaction. Attempting to interpret the AI’s role in QEG’s performance was difficult, so the team created an internal “interpretability engine,” which helps QEG parse what the signals emitted by the neural network mean.

Ameriks is aware that the use of AI in quant shops is controversial. Analyzing risks is difficult when an AI tries to fit extraneous data points onto a curve. 

“The machine is able to do so much more work computationally than what we would traditionally try to do with that kind of analysis, that you end up with way too many degrees of freedom,” he says. “You end up with something that is just either data mining or curve fitting—whatever you want to call it—and I think that’s always been a danger for quants.”

Ameriks says that for now, human oversight of AI tools is key. 

“People have asked before, ‘Do you worry about AI and ML as these machines get smarter? Will there continue to be a role for humans?’ I guess that’s the most frustrating thing,” he says. “I mean, do we have enough time in the day for the humans? I think we do, to look at this need to really assess the validity of what’s coming out of the machines.”