AI in sheep’s clothing? Wolfe Research develops finance chatbot

Wolfe Research has engineered an AI chatbot for financial markets. Designed in the same vein as ChatGPT—the large language model (LLM) chatbot launched by OpenAI in November of last year—the Wolfe chatbot is specifically trained on financial and environmental, social and governance (ESG) data.

And for certain questions, the bot’s biggest benefit is saving time for both clients and the firm, says Yin Luo, vice-chairman at Wolfe.

“It’s definitely a lot faster to ask the chatbot than asking us. We will take anywhere from a few hours to a few days depending on the complexity of the question.”

But the technology won’t make human analysts redundant—just yet at least, adds Luo.

Interest in the use of computer models to read text has ballooned in finance, with uses that range from assessing company sensitivity to one-off events, through to building virtual analysts to parse internal documents. Training the models to read industry-specific material, however, takes time and expertise.

“The goal is to train the model on the questions so that it can provide informative answers,” says Luo.

“The most common question is 'What kind of investment styles perform the best, for example, in a recession environment or as the economic cycle changes?' People are asking very similar questions about investment styles, asset classes, and the major risk factors in different economic cycles,” he adds.

Other typical questions that the chatbot can answer are on the fundamental side, such as “What is the target price for company ABC?” or “What are the implications of central bank policies like rate hikes on the company?”

In fact, the many client questions the firm has received over the years about which investment styles perform well in different cycles was one of the easiest things to automate, says Luo.

There are some drawbacks, however. The foremost, says Luo, is if a user asks a very specific or highly technical question about a given company or scenario that Wolfe Research hasn’t analyzed before, and for which no training documents exist.

“The large language model has shown a good ability to answer questions when we don't have any training documents for the question. In many cases, it will still give you a reasonable answer,” he says. “But, of course, if a question is very specific, very technical, and something that has not really been seen before, then you probably won't get a good answer.”

The model is also unable to answer questions like what techniques will help avoid a market crash—that’s because every single market crash is different, so what worked in the past may not work as well in the future.

There are other well-known problems with LLMs. ChatGPT, for example, has been known to provide quotes, citations, and references that are non-existent.

“Our language model has suffered similar kinds of issues, but we have implemented techniques to prevent it from attributing research and publications that do not exist,” says Luo.

Only human

Wolfe launched its QES GFT [generative fine-tuned transformer] LLM in a series of beta phases over the last few months. The technology underlying the chatbot consists of three key components. The first is the LLM, an artificial intelligence model that uses neural networks to consume vast reams of text data to understand context, syntax, and semantics in language, and answer questions.

We can probably replace a lot of customer support roles in the shorter term, where people are asking simple questions and looking for simple answers

Yin Luo

For its second component, instruction training, Wolfe Research fed the LLM over 100,000 training datasets, based on questions from clients received by the firm’s systematic/quantitative, fundamental analyst, and sales teams through various formats, including email.

The third component was fine-tuning the LLM for finance and ESG purposes—one of the last layers of the neural network. The model was fed a large collection of documents, predominantly related to finance, public policy and ESG. The documents cover a wide gamut within these topics, including news and social media, company transcripts, documents on monetary policy, fiscal policy, trade policy—or industrial policies and regulations—government environmental policies, UN climate change conferences, white papers and companies’ proprietary corporate social responsibility recordings.

Having built the chatbot and witnessed its strengths and weaknesses for himself, Luo believes the notion that AI will replace human workers remains a fantasy, especially when it comes to human analysts.

“We can probably replace a lot of customer support roles in the shorter term, where people are asking simple questions and looking for simple answers,” he says. “The language models in today's world can provide reasonable answers, and they are powerful tools. But it’s not necessarily replacing human analysts just yet.”