Datactics Takes 'Human-in-the-Loop' Approach to Machine Learning

Belfast-based Datactics is producing an entity resolution model it has developed using a “human-in-the-loop” approach. The new feature will be embedded within the Datactics Entity Match Engine, which is used for tasks such as client onboarding, anti-money laundering (AML) and know your customer (KYC) processes.

As the name suggests, human-in-the-loop is a concept in machine learning where humans are directly involved in training and testing models for a machine learning algorithm.

Fiona Browne, head of artificial intelligence (AI) at Datactics, says the offering is currently at the semi-launch stage. In the next couple of months, the firm will move into production and monitoring of the model.

Browne says that one of the benefits of the human-in-the-loop approach is having greater AI transparency and explainability—the ability to explain the technical processes of an AI system and the related human decisions.

“I think what is often underestimated in this whole machine learning landscape is the effort required to produce, firstly, a high-quality dataset to feed into your model, and secondly, high-quality labels to feed into your model,” Browne says. “So the whole explainability side was very important to us.”

Browne says that AI explainability requires explaining decisions such as which datasets were employed, their labeling, and providing the rationale behind why the datasets were labeled in a certain way. 

She adds that entity resolution is an important part of the KYC process. It can involve validating the identity of a client who is being onboarded and making sure, for example, they are not on a sanctions list. That often involves the comparison of client information to both internal data stores and external ones, such as Companies House and Dun & Bradstreet. 

“Often these data are quite messy datasets: they contain duplications, and they may be outdated and so on,” she says. “A lot of time is spent by banks or other businesses on the manual review stage of determining whether or not this person is who they say they are, [and whether] they match on this list or not.” 

In a study using the model last year, the firm used Refinitiv’s PermID datasets, which contain information on legal entities and financial instruments. It then matched this data against entities in the global Legal Entity Identifier datasets.

By using high-confidence predictions, which had scored highly on the match rules, the firm was able to reduce the manual review process by over 45%. 

Browne says the extra time will allow people to work on more complex cases, and free up their time for other tasks. “We also find by using this type of approach that the data quality itself improves and becomes more consistent,” she says.  

A report on machine learning in UK financial services by the Financial Conduct Authority (FCA) and the Bank of England (BoE) last year mentioned alert systems and human-in-the-loop mechanisms as common safeguards used by many firms they had surveyed. The report notes that such methods are useful when a model does not work as intended; for example, in the case of model drift, which can occur when machine learning applications are continuously updated and make decisions outside their originally intended parameters. 

Nasdaq also uses human-in-the-loop training in its AI-powered market surveillance tools. By having their US surveillance team sift through actual examples of alerts, Nasdaq’s human- and machine-powered model can identify whether an alert is interesting, and if it is of interest for a certain time frame.