AI, Machine Learning Can Tackle Dirty Data

Data is described as a "sea," a "firehose," and a "tsunami." There's a lot of data and as the ability to take in and store information has improved, the ability to analyze the growing inflow of data is the greatest challenge faces firms, whether for finding trading opportunities, managing risk or for regulatory reporting.

While everyone wants clean data, a lot of value exists in dirty, unstructured data. That's where Marc Alvarez, chief data officer at Mizuho Securities, believes that artificial intelligence—and, by extension, machine learning—can help. Rather than worrying so much about clean and dirty, use AI to sift through that sea of/firehose of/tsunami of information to direct the user to useful data hiding in the mud.

"I don't talk about clean data," said Alvarez, speaking at the Waters USA conference in Manhattan. "We talk about data in the terms of control and not in control. We don't prevent data from going anywhere. The reality is that today, as the business evolves we are becoming a very quantitative- and derivative-driven world. So we end up with these relatively sophisticated methods of moving money in and scaling it up for our purposes. By definition, all the data can't be right, nor should you under any circumstances attempt to make it all perfectly shiny and new."

He said it's best to leave it up to the business to decide what is good or bad data. Where firms should focus their investment is in skills around quantitative and statistical analyses.

"The reality of statistics is that we've been interpolating missing values and time series forever," he said. "We use statistics to measure the distribution and probabilities of populations—by definition it is not perfect, and this is where the strength of AI, in particular, is useful because it leverages those abilities."

Establishing new controls and understanding what the data is being used for is also important.

"But artificial intelligence is coming—let's be frank about it," Alvarez said. "If it's a better mousetrap, it's going to find its place and scale up. So what that means is, what are the controls in the organization? These are going to be controls that we haven't had before; they're going to be very different—moving to real-time evaluation of buy and sell, real-time profitability analysis, maybe even predictive profitability analysis. So it's not just about the technology and it's not just about the data. It's actually going to drive more of a systemic organizational change."

Getting Buy-In

Alvarez said that at Mizuho, senior management on the business side, working with IT and operations, has driven the use of AI in the firm. The business poses certain questions and asks for ideas to solve for.

"People talk about how financial firms are turning into technology companies. Well guess what the core competency of a technology company is: telling your management what to invest in and what they're going to get for their investment."

Charles Fiori, a consultant with 30 years of experience in the finance space, said that since it's an organizational shift, "the impetus has to come from the top; there has to be sponsorship from the upper levels of the organization and it has to be made clear that making this work is a priority for the company," Fiori said.

Still Concerns

Joseph Lodato, global head of compliance technology and surveillance at Guggenheim Securities, said getting that buy-in can have challenges.

"The problem with something coming from the top is that you have to go and climb that ladder," he said. He added that where firms have to be careful is that when pitching too many options, the process can become much more complicated than necessary. "It's like being in an app store and there are 10,000 apps," Lodato said.

Alvarez added an extra caveat: AI and machine learning is not going to solve all problems.

"Throwing automation at this stuff—especially the big datasets and datasets that are intermingled with relatively structured/unstructured datasets—it has a habit of creating an awful lot of false positives," he said. "So the business is going to approach this in a very conservative fashion."