Solid Data Lineage: The Foundation for Automation

As they deploy automation to solve regulatory problems, Asian firms are paying closer attention to data’s journey.

Regulators globally are requesting more accurate, timely, and detailed data from market participants. Asia is no different, except many firms operating within the Asia-Pacific (APAC) region are not dealing with one regulator, but several. Consequently, banks are turning their attention to the journey of their data, concentrating on data lineage as an essential part of establishing more sophisticated methods for data management and governance.

Abraham Teo, global head of tax products and head of product management for the Asia-Pacific region at risk and regulatory solutions provider AxiomSL, says Asia’s industry landscape is prohibitively fractured. “There are all these different regulators asking for different things and they don’t really coordinate or talk with each other. If you look at the US, there are a few regulators, but they’re essentially nationwide. As for the EU, the European Banking Authority are standardizing regulations across the region,” Teo says.

As a result, APAC firms are looking for solutions that allow them to focus on their core business instead of allocating additional resources towards dealing with confusing or conflicting reporting obligations. 

In June, AxiomSL surveyed 91 executives based in New York, Singapore and Sydney, and found that adapting to regulatory changes remains a top challenge for risk and regulatory management. APAC executives ranked highest in overall concern at 69 percent, compared with 43 percent of American executives chiefly concerned about regulatory change. 

Lineage Logic

Gary Goldberg, CDO at Mizuho International, says organizations have woken up to the fact that data powers business processes. “An IT system doesn’t function without data and a business process relies on that data to do what it needs to do. The accuracy and quality of the data has a material impact on the efficiency of the operations and quality of service for our customers. There’s a realization that it’s absolutely critical,” he says.

Tim Versteeg, general manager at NeoXam APAC (ex-China) notes that data lineage has been a part of all discussions the company has been having with banks in Singapore, Hong Kong and elsewhere in Asia. There is a consensus that it is an area of concern globally, according to the AxiomSL survey, which found 63 percent of APAC executives and 64 percent of Americas executives have indicated a need for improvement in data aggregation, lineage and reporting. 

Although the term ‘data lineage’ has been used since the 1990s, regulation such as BCBS 239, which evaluates data in terms of usage and allocation of cost, pushed the topic of data lineage to the forefront. 

This piece of regulation was introduced in response to the credit crisis. The problem at that time was that institutions on both the buy side and the sell side could not always tell what risk they held and who held it, plus not all institutions held accurate data to clearly identify the risks. 

“For example, an organization that had risk with Lehman Brothers or Bear Stearns and didn’t have accurate data, might not have known if that risk was with Lehman New York or Lehman in London. If the systems merely identified Lehman Brothers, then the organization would have been unable to tell which legal entity the risk was actually against. Those differences have a significant impact on how the business is managed,” says Goldberg. 

Global financial institutions learned from the crisis that they needed a better understanding of their data. 

“When they encountered a stress scenario where decisions needed to be made quickly and intra-day, not every organization was able to respond the way they wanted,” he adds. 

The premise of BCBS 239 is a set of principles outlining that banks essentially need to know where data is, how good it is, its purpose, et cetera—the regulation reiterates best practice. “Like so many things, it’s easy to say, but implementation is a different order of magnitude,” says Goldberg. 

Another aspect to consider is data licensing, says a market data manager at a global securities services provider. “If you don’t have lineage then you don’t know who owns it, and license permission is necessary to use proprietary data. It’s a very painful and unsexy element of data management. Having to do this and also putting it into legacy systems is not easy,” he says.

In Asia, he adds, regulators such as the Monetary Authority of Singapore (MAS) have stepped up financial institutions’ reporting obligations

“When regulators have prescriptive reporting requirements, they require that the company knows where the data is from. In other words, knowing the provenance—the beginning of existence of the data,” the manager says.

While many banks already have existing regulatory reporting, monitoring and testing programs, most of them have been established to comply with a single regulation. This is changing, as regulators increasingly demand that banks continually carry out transaction testing programs to identify any data quality issues that may arise, and to provide ongoing monitoring over reported regulatory data. 

Versteeg notes that financial institutions with a prudent risk management setup require running historical and ad hoc what-if scenarios. Regulators such as the MAS require banks to conduct stress tests, he adds. 

“All this implies flexibility to generate, manage and provide multiple ‘scenario data sets’ to a risk system. Having the right lineage mechanisms in place enables the (re)creation at any time and management of these sets in parallel and over time. Risk engines cannot cope with this easily,” he says.

To improve data lineage, Teo says firms need controlled environments, which can be achieved through automation. “Once you put a human into the process, there’ll always be a chance for human error, when someone enters the wrong information or clicks on the wrong button. But if you’re doing true end-to-end automation, the only way to get it wrong is to have wrong logic encoded, or some new functionalities that need updating,” he says.

Automation actually requires better data, explains Mizuho International’s Goldberg. “There are a lot of initiatives in the financial industry around robotic process automation. Robotics require better data, because when you no longer have a human in the process, you lose the ability for that person to intervene and raise questions. A robot takes the inputs you give it, runs a process according to a set of rules and provides an output. The more you automate, the more important data quality and data controls become,” he says. 

The lower the quality of the data, the more exceptions a firm would need to have in process, and this means somebody will need to step in to intervene, investigate and understand the problem. For all these reasons, data is becoming an important consideration for many financial companies. This is particularly true when firms are trying to use data analytics and gain value from it. 

“If you look at what companies are trying to do with data science and analytics, the focus is to obtain insights to serve customers better and to improve regulatory reporting,” says Goldberg. “Those analytics rely on data. The increased importance of machine learning and data science is leading to a greater understanding about the need for data management. There’s a symbiotic relationship between data management and data science. 80 percent of the work in data science is in preparing, cleaning and wrangling the data. Clearly the more data quality is improved, the greater the likelihood that the 80 percent can be reduced.” 

As demand for quality data increases, banks are challenged with disparate and legacy systems. To be fair, the existing infrastructure at these firms was not made to serve regulatory reporting needs. As a result, many of them require significant overhauls to be able to improve data quality. 

According to AxiomSL’s survey, executives in the Americas and in APAC plan to increase their investment in risk and regulatory technologies this year, to meet compliance challenges. The Americas recorded an uptick from 48 percent in 2017 to 52 percent of respondents planning to increase investments in 2018. In comparison, the results from APAC respondents was higher, at 61 percent versus 44 percent the previous year. 

Easier Said Than Done

Although regulation such as BCBS 239 can be a burden to some institutions, not many would disagree with its principles. The compliance challenge for banks is remediating their legacy infrastructures to better deal with regulations. 

Cultural change within the organization should also be laid on the table. Having the best systems in place is not enough if staff do not realize the importance of their tasks.

“Large organizations that hadn’t considered data as part of the core operating model needed to educate staff. It’s one thing getting an understanding at the executive level, but it needs to be understood by the staff on the ground,” Goldberg says. 

Good data lineage drives data quality, adds the market data manager. “I think it comes down to each firm having a dedicated data governance function where there are people responsible for information gathered and embedded in the business process so that it’s living and breathing and it’s kept up to date and adapted to what it’s needed for,” the manager says.

Outsourcing Options

Of course, outsourcing key data functions is a popular option for firms overwhelmed by projects and short on budget or staff, but financial institutions in Asia often have fewer vendor solutions in place compared with similar financial institutions in Europe and America, Versteeg says.

“Quite often [firms] rely on spreadsheets or proprietary solutions which tend to act as a black box to manage their data,” he says, but those methods show a fractured picture of lineage. “Trying to add that in the system is quite difficult, while not having it is a huge operational risk. Regulators want to know how you receive the data—where it is sourced from. They want the data flow to be governable and transparent.” 

From the perspective of data vendors, data lineage is necessary for vendors to know if they have entitlement on how the data is used, and if it is being used by clients for the purpose it was bought. 

Versteeg says NeoXam has found that even within banks where the data procurement is centrally owned, often it is not known who used the data and how it is used. 

Determining where and how data is sourced from and where it is used benefits not only the institutions in tracking their data’s journey, but also makes the regulatory reporting process easier. 

“Having appropriate and systematic data lineage is needed for validation of data. It will also prevent any unnecessary overlapping of data, and therefore help with cost and control of data,” he says. “Firms have to pay for static and market data, to which the latter is more expensive. If the same market data is bought by, for example, two different departments to be used for different calculations, there is already a duplication of that market data. Market data is expensive and having duplicates of the same thing is just not cost efficient.” 

Now, whether automation is easily achievable depends on the bank. Some of them could be outposts of a global bank that already has some processes automated that could be replicated for its Asian operations. Others might have more or less complex processes to automate. 

Regardless, Teo says, there is always room for improvement and more investment into automating data processes, especially after an automated system has been put into action.

“A lot of banks focus on the initial implementation, but it’s the maintenance that’s important. Otherwise, there could be breakdowns and banks risk falling back to manual processes, and that’s what they want to prevent after making such a big investment,” he says. 

Ending ‘End-User Computing’

Banks are trying to put a lid on what Teo calls ‘end-user computing.’ “Spreadsheets, macros, created by guys on the line, although well-meaning attempts to cut down the amount of work to do, in the longer term, they could [create] other problems,” he says. 

For example, end-user computing could result in a scenario where no-one remembers how to update the macro, or the person who initially created the macro has left the bank and without passing key maintenance information to colleagues. 

Teo explains AxiomSL tries to integrate such workflows into its solutions so the maintenance cost is low and that process is supported going forward. 

“Our solution is not object-based, we’re basically adding additional objects that replicate the logic this macro was intended to do, and then it’s just run daily or monthly, etc, as part of the workflow. That part is quite easy. The hard part is understanding why they are using this. What is this macro trying to achieve? Once we understand that, putting it into our solution is easy,” he says. 

This process is unique to every financial institution. In some cases, the tolerance for risk could be higher where a process is informational to the business but does not drive key decision making. 

“But if it’s a key process, for example, invoicing or information security, then clearly your consideration for data quality does go up. Most institutions will be aware of the context of the data. In general, the more important the process, the more important it is for the data to be correct,” says Goldberg. 

Even when processes are driven strictly by regulations, as the rules keep stacking up, managing data will become increasingly difficult, which means the bare minimum is not good enough. 

“Once BCBS 239 came in, regulators assumed organizations had an infrastructure that can deliver high-quality data. Every subsequent regulation is raising the bar and building upon those expectations,” says Goldberg. 

Firms that don’t prioritize data lineage and quality may risk their own survival. 

“Organizations that don’t address their data strategically may find they don’t exist in a decade. Data is that fundamental to an organization’s ability to compete,” he notes. 

Only users who have a paid subscription or are part of a corporate subscription are able to print or copy content.

To access these options, along with all other subscription benefits, please contact info@waterstechnology.com or view our subscription options here: http://subscriptions.waterstechnology.com/subscribe

You are currently unable to copy this content. Please contact info@waterstechnology.com to find out more.

‘Feature, not a bug’: Bloomberg makes the case for Figi

Bloomberg created the Figi identifier, but ceded all its rights to the Object Management Group 10 years ago. Here, Bloomberg’s Richard Robinson and Steve Meizanis write to dispel what they believe to be misconceptions about Figi and the FDTA.

Where have all the exchange platform providers gone?

The IMD Wrap: Running an exchange is a profitable business. The margins on market data sales alone can be staggering. And since every exchange needs a reliable and efficient exchange technology stack, Max asks why more vendors aren’t diving into this space.

Most read articles loading...

You need to sign in to use this feature. If you don’t have a WatersTechnology account, please register for a trial.

Sign in
You are currently on corporate access.

To use this feature you will need an individual account. If you have one already please sign in.

Sign in.

Alternatively you can request an individual account here