For Effective AI, Domain Knowledge often Trumps Analytics

Just deploying analytics and artificial intelligence to make business and trading decisions is not the end goal, as companies must make sure the data actually aligns with reality, said technology executives at the Waters USA conference, held in New York on December 3.

Therefore, domain knowledge, rather than observations coming specifically from numbers, may be more important.

Victor Tewari, senior technology officer for global capital markets and corporate banking technology at Bank of Montreal (BMO) Financial Group, said analytics should also align with the user’s knowledge of the markets.

“You validate the model efficiencies and verifications to a level that the results you’re seeing match your innate business knowledge. If you have domain knowledge for 15 or 20 years and you know bond demand is in a certain pattern based on demand, and your data is telling you something different then your knowledge is probably better,” Tewari said. “If there’s an outlier, you verify that outlier.”

He added BMO has had large success with working with its data, to get to a point where what he calls ‘black-and-white outcomes’ are generated. This information can be easily verified and be in line with inherent market knowledge.

And it is not enough for firms to start using analytics to inform business decisions, either. Companies have to make sure they understand how to read conclusions from the analytics, and to figure out a way to make their grasp of the data stand out.

WorldQuant chief technology officer David Rushkin said people are an intrinsic part of understanding outcomes from data analysis.

“One of the ways you can stand out is to create your own unique data, and the other is around people,” he says. “People have different ways of looking at the data and this generates different approaches to the outcome. Of course, you need to offer some sort of limit, so it’s not a completely random outcome.”

Rushkin warned, however, that firms must make sure their people have a fundamental understanding of statistics. He said not having statistical rigor can lead to faulty outcomes, like not knowing the difference between causation and correlation. Therefore, it is not enough to simply run a tool against data to get results, people have to interpret it correctly as well.

After the analytics have been cleaned and verified, most firms would immediately set it for use by an AI system. But Julia Bardmesser, senior vice president and head of data, architecture, and analytics at Voya Financial, warned it is not that easy to set up AI programs, and crucially, to keep them running.

“If you listen to vendors and consultants, they will say putting up AI is easy,” she says. “But it’s like an Ikea bookcase. It looks easy but when you get it home you realize you need an expert to build it. Robotics is the same because scripts can get complicated so you need a team to do it, you need a team to maintain it.”

Bardmesser adds companies need to understand what business processes they want the AI to sit on top of, to justify the cost of putting a bot on top of it.

All the speakers in the panel noted at the end of the day, analytics and robotics become of greater value to a firm if a business process is made better because of the technology, or if the data or AI provides a much better way of reaching business goals.