FSB Warns Artificial Intelligence Could Present Systemic Risks

AI and its various subsets, including machine learning, robotics process automation and natural-language processing, have been among the most widely deployed of a number of emerging technologies that are gaining traction in the financial industry at present.

Programs that employ AI are being actively researched by nearly all major institutions—a recent report from Greenwich Associates estimated that three quarters of banks and financial services firms will be using AI in some form within their businesses over the next 12 months—and have seen widespread deployment in areas such as robo advice, portfolio construction, customer support, risk modeling and trade surveillance.

However, the Financial Stability Board (FSB), in a wide-ranging report released on November 1, expressed concern that the difficulty inherent in determining how a complex AI system arrived at a particular decision may prove to be problematic in crisis periods.

“Many of the models that result from the use of AI or machine learning techniques are difficult or impossible to interpret,” the report said. “The lack of interpretability may be overlooked in various situations, including, for example, if the model’s performance exceeds that of more interpretable models. Yet the lack of interpretability will make it even more difficult to determine potential effects beyond the firms’ balance sheet, for example during a systemic shock.”

The report also raised a number of concerns in other areas, such as the fact that many models are being “trained” during quiet periods, and as such, could behave unpredictably during market stress. The FSB also reiterated its concerns over model complexity, saying this made internal audit and risk management processes difficult, given the skillsets needed in order to effectively analyze the technologies at work. There are further worries that such models, if they become widespread, could also interact in unpredictable ways.

“This problem could be serious since we often don’t know what models will do outside these quiet periods, and the boundary locations between normal and extreme events are unknown,” says Elliot Noma, managing director at Garrett Asset Management. “In addition, the availability of easy-to-use packages such as scikit-learn and tensorflow encourage analysts to gravitate toward a common set of models. These models could act synchronously once market conditions are outside their trained conditions.”

Noma adds that an “exclusive emphasis on performance rather than understanding puts the industry at risk in stress events.”

Finally, the FSB was also concerned that a widespread use of AI could give rise to third-party dependencies—and result in the creation of monopolies that may not only fall outside of regulatory scrutiny, but which also “could be translated into financial stability risks if and when such technology firms have a large market share in specific financial market segments.”

“These third-party dependencies and interconnections could have systemic effects if such a large firm were to face a major disruption or insolvency,” the report continued. “Many current providers of AI and machine learning tools in financial services may fall outside the regulatory perimeter or may not be familiar with applicable law and regulation.”

The FSB, which issued a similar report on the financial stability risks associated with fintech in June, recommended further study into how AI is used.