“Oh, I’m sorry. I must have missed your email.” It happens all the time. But usually a missed message doesn’t incur financial penalties or unexpectedly halt the flow of market data into trading systems. However, missing a data notification message from a vendor or exchange about a change in the data available, the format in which data is delivered, or the fees that apply to specific datasets can cause exactly that.

“This is one of the least interesting aspects of market data management, but it is one of the most critical,” says Ron Troy, an industry veteran who has spent decades in market data roles at a variety of Wall Street firms and now serves as an independent consultant.

Data notifications are a growing challenge for capital markets firms, which must ensure that they capture every notification and that every change is noted and incorporated into any affected systems. As the number of marketplaces and complexity of securities increases, the volume of notifications being generated—including technical notices and system status updates—is rapidly increasing, placing an added burden on the teams responsible for making sure those changes are addressed.

Specialist market data and management consultancy Jordan & Jordan (J&J) conducted a survey of almost 40 financial firms over the summer, focusing on data notifications. One finding was that there are simply too many notifications hitting data managers’ inboxes or group emails to be handled with the attention they deserve. Some respondents said there were up to 200 notifications coming in per day on peak days, with others estimating that total number of notifications has grown tenfold over the past 20 years. For each notification, data professionals need to determine what the change is, whether it affects the firm’s systems and data consumers, and then take appropriate action to account for the change—often within a timeframe of just two weeks.

Unfortunately, this is rarely straightforward. Notifications may not be relevant, though this may not be immediately apparent; they may have subtle nuances, such as referring to a separate policy statement that clients need to be aware of; or, their text may be so complex that it requires interpretation, says Barry Raskin, managing director at J&J.

“They’re voluminous, complicated, arcane, and interpreting these things is boring but necessary. Notifications from developing markets may have been translated into English and hard to understand, and even those written in English may need a lawyer to translate what they actually mean, because especially in small asset managers with a small data group that might be rolled into their technology function—these people aren’t lawyers,” Raskin says. “But there are a bevy of potential issues if you miss these notifications. You could miss a fee change, a policy change, a format change in a feed that means all of a sudden data is not flowing, or a change in trading hours that means you miss opportunities to have your network available.”

And if not handled properly, firms can find themselves facing unexpected fees (and possibly non-compliance penalty charges) and the prospect of data being unceremoniously cut off.

For example, data notifications can cover increases in prices for data. If a firm does not apply price changes to its user profiles or to its inventory management system in a timely manner, it would receive a higher-than-expected bill and would need to fork out cash to pay the increase, and update its systems to recognize the new amounts.

Meanwhile, if a data notification covers a change to a security code, or what dataset an individual security is part of, or if a code is being retired, data professionals need to ensure that those changes are accurately reflected in the systems that consume data—as well as the entitlement and inventory management systems that govern them—to prevent instances where data may appear to shut off. This could mean that those systems can no longer perform their designated functions properly because they don’t have access to all the data inputs they require.

“If I miss a price change, someone on the business side will be mad at me, and they’ll have to find additional budget, but it doesn’t prevent them from doing their job,” Troy says. “Other changes can have more extreme impacts.”

For example, he cites an instance of arriving at work on a Monday morning to find that a bunch of treasury codes that should have been decommissioned years ago—and which it turned out that the firm had been notified about years ago—had finally been turned off by its data provider without warning. “If you lose access to those, you can’t trade, your spreadsheets won’t update, you can’t perform analysis, and you don’t know what your portfolio is worth.”

‘Miss a Change, the Data Stops’

“Very simply put, if you miss a change, the data stops coming in. Or if a data format changes, you have to regression test your applications,” says Jeremy Green, head of market data and trade services at DBS Bank in Singapore.

In some cases, firms are willing to take the risk to avoid the headaches, says Marc Rolfe, director at London-based process management software vendor Ropnoy. “Vendors issue data notifications, but clients have no way of knowing whether it affects them, so they download a spreadsheet to see what securities are affected, then run reports in Refinitiv’s Data Access Control System (DACS) entitlements tool to see if anyone is using that … which can take anywhere from one day to one week,” Rolfe says. “For some, it’s so arduous that people just aren’t doing it at all. They’d rather take the risk that, for example, a [Reuters] instrument code goes away, and when a firm’s calculation engine calls it, it results in a null input into their portfolio.”

There’s another, non-monetary and non-operational impact of not getting these right: Errors reduce a data department’s credibility when trying to influence its counterparts on the business side or in management roles, says Aaron Garforth, co-founder and CIO of exchange data policy information provider Axon Financial Systems, and chief content officer at parent company TRG Screen.

“The problem is the volume of data notifications and the amount of work involved for people to figure out how it impacts them,” Garforth says. “Within that, the biggest problem is inconsistencies between the format of attachments to the notifications. So the list of affected securities might be in a PDF or a spreadsheet. You could parse them, but I’m not aware of anybody who has done that.”

On top of this, each notification can include changes that impact thousands of instruments, which is hard enough for large firms that trade all securities, but presents meaningless work for smaller firms such as hedge funds that have a much more limited universe of securities that would be affected by the changes, but who must nevertheless comb through all the changes to find those that affect the securities they trade.

Those charged with handling data notifications must also understand the impact on different user types. “A single change for one exchange affecting fee liability can have multiple impacts, depending on the vendor and the user type. For each, the details can be different,” Troy says. And in some cases, data teams must prioritize who they notify first to best manage the workflow involved and deliver the most appropriate level of involvement from different parties affected. “If I have a short time to get something done, first I would inform those people who actually use the data, but ideally we would also tell everybody who is permissioned for it and could use it,” he adds.

DBS’s Green also warns that data teams must be discerning about whether or not to treat every notification in the same way, and to judge which notifications are absolutely necessary to specific applications, compared to what one would do in an ideal world.

“You might be the application owner of a large system that captures reference data. Do you really care if an exchange is publishing data 10 minutes later or changing its cost structure? There’s a lot of noise that not everyone cares about, and it’s coming from all the exchanges. If you action every one of those notifications, you’re going to bombard that application with thousands of updates per year,” Green says. “Yes, ideally people would look at every one of those. But people have never been busier.”

So they have to choose how best to spend their time when dealing with these notifications, based on which areas present the greatest risk.

Ch-Ch-Changes

The data notifications challenge is not new, but it now presents a more urgent issue than in the past because the notifications have grown drastically in volume, frequency, and sophistication.

Green says that in the past, vendors used to highlight any upcoming changes in their systems that they would monitor, but account representatives would also call to verify that the user understood that a change was coming. From there, the user would figure out how the change would impact the organization and the developers, and figure out who needed to make those changes. Today, there are simply too many notifications and they are complex for a one-on-one experience to be reasonable.

“The underlying process has not changed too much. What has changed is the changes themselves—the volume, the product complexity, and that the notifications now include technical changes and now contractual changes, too—and a big difference is that all of a sudden, account reps don’t phone anymore,” he says. “Vendors have pushed the responsibility for managing this onto the consumers, and you as a consumer are expected to interpret these changes. So the hand-holding is gone, and we’re being bombarded by changes.”

Beyond consumers of the affected data, those who need to be aware of data notifications include anybody within a firm who would be involved in addressing the notification or working on the data or systems required to fix any changes, Troy adds. “Software developers need to know, database maintenance staff need to know, and the bosses need to know. So you need to create a list [of everyone affected]—and that itself involves work.”

How much work? Green estimates that handling data notifications alone would consume around half of someone’s time, before tackling any other market data-related tasks.

“It’s a half-person, full-time job. It takes about two weeks per month. Even though there are tools available, there is still a lot of volume to deal with. It’s not difficult to do—you scrape the changes, and compare them to what you are permissioned for in DACS—what’s difficult is connecting very disparate systems,” Green says. “For example, the market data team will not be the development expert who needs to code the changes into those systems … and those developers may have their own assumptions about the data coming in.”

Help at Hand

Even with a knowledgeable understanding and a methodical approach to handling notifications, the workload alone presents a challenge when many data departments have been stripped to the barest bones. That’s where a new generation of systems can help, with several vendors offering solutions specifically designed to lighten the burden of managing data notifications.

Ropnoy, for example, has several solutions for addressing data notifications. Its full-service offering, Data Notifications Analyzer (DNA) has been available for several years, and provides a comprehensive view into all notifications affecting a firm, with full audit control so data managers can see exactly how each notification is being handled at any point in time.

Smaller clients with more modest universes of securities to monitor requested that Ropnoy take on more of that heavy lifting to reduce the overhead of running a large platform. So, more recently, the vendor released a more lightweight suite of solutions, collectively dubbed DN Bureau, which comprises three options: Essential, Premier, and Prime.

Essential is a “self-service” option where Ropnoy performs cross-checks on notifications, and the client decides who will be affected and assesses the potential impact. For Premier, Ropnoy profiles a firm and its users to know which people and systems are affected by a specific change, and can manage the process in an automated manner, with the ability to run unlimited reports. Prime is a recently released hybrid of the other two options that identifies low-, medium-, or high-impact notifications, and allows a limited number of reports.

Which product a firm chooses would depend on the size of a firm’s data department, the volume of notifications they typically need to manage, and which aspects of managing notifications they need help with, says Lisa Nicholson, director at Ropnoy.

New York-based West Highland Support Services—a former partner of Ropnoy on the Evo-One data notifications utility—has its own offering, Data Notification Manager (DNM), which identifies notifications with RIC lists attached, then flags whether there is “usage found” corresponding to those lists, and where within a firm that usage exists.

It also shows the status of how a notification is being handled, the revision history for that security, who the response has been assigned to, and has the ability to add notes. So, for example, the person assigned to handling a particular notification can update on their progress—and provides a calendar view, which data managers can use to search historically for a notification if a data consumer reports a problem, or to plan resources ahead of busy periods or vacation dates. The tool also has tools that allow users to filter results by each key factor, and has the ability to search by key words, such as by application name so as to see all notifications affecting that app.

“Data Notification Manager frees up time and reduces errors associated with data notifications,” says Scott Breslin, senior vice president of sales and business development at West Highland. “Clients who use this tool don’t miss data notifications, period. We don’t have a client using it who has missed a data notification.”

West Highland has had DNM for around seven years, covering notifications from Refinitiv, Bloomberg, Activ Financial, and ICE Data Services. It is also talking to FactSet about adding support for notifications from the vendor. West Highland has just unveiled a free service, Data Notification Subscription Service, which provides a basic notifications-alerting service for changes from Refinitiv and Bloomberg. Users sign up with basic contact details, and can choose what types of notifications they want alerts for, such as a price change, or content being removed.

“We’re hoping to create greater awareness around data notifications,” Breslin says, and to give potential users a taste of what West Highland can provide. “Since we were already capturing all those notices from vendors, we came up with the idea for a free service around a month ago, and the developer of DNM was able to build it quickly.”

Meanwhile, in response to the survey it conducted over the summer, J&J developed its own solution. The firm currently has an alpha version of the as-yet unnamed product, which it has tested internally.

Based on the survey results, J&J targeted a more bespoke data notifications handling process that accurately pre-filters notifications so data professionals need only spend time processing changes that they are already certain will affect their systems and staff. And in addition to filtering out “noise,” J&J will add value to the notifications it pushes to data managers. Examples include having its licensing attorneys review any notification to translate or interpret it, by assigning each an interpretive headline that makes it crystal clear what it contains and whether a data manager needs to read it, and by presenting the remaining relevant notifications in a way that doesn’t overwhelm data staff.

“We got the same feedback again and again from clients: They are overwhelmed by all these emails. They want notifications to be bespoke and pre-sorted, presented with summaries, in a timely manner,” Raskin says.

The next step, Raskin says, is to have some of the firms who participated in the survey test the product and provide feedback, ahead of making it more widely available before year-end.

Axon also has an offering in this space, though Garforth refers to it as a “light workflow” that allows market data managers to see which applications are affected by a change, and to assign responsibility for implementing changes to individuals, and to annotate when a change is completed, rather than the kind of full end-to-end solution being touted by others.

“This was always a natural next step for us. Originally, data notifications were one of the things we monitored to keep our Policies, Explanations, Analytics, Repository (PEAR) exchange policies database up to date. So we were monitoring exchange notifications—focusing on policy changes—for our own purposes, but we didn’t have vendor notifications. So we started adding Bloomberg and Refinitiv notifications around 18 months ago,” Garforth says.

Human Touch

Despite the advantages these tools can bring, parts of the process are likely to remain manual—and some aspects may even always require human involvement and oversight, Troy says.

“It’s helpful if you can get a tool or service to analyze permission codes and usage (from DACS or other perm tools) against the data notifications that you receive from your data providers,” he says. “If you don’t have a tool or service, come up with your own process to run the notifications list against the exchanges, services and symbols that each business and or user is following. You may find permissioning changes (like DACS code changes), items going fee-liable, symbology changes, and price changes. In the case of something going fee-liable, you might compare popular symbols that could be included in the data notification to your detailed usage reports (by symbol). Done properly, you’ll be a lot less likely to miss notifications that impact your firm. You then review your findings, figure out what needs doing, and who needs to be notified. Usually, you get at least a few weeks’ notice, but not always.”

And some practitioners are wary of relying too heavily on software to manage the data notifications process.

“I don’t think these new systems are quite ready to handle the complexity of notifications,” Green says. “You still need that human touch to understand the impact and how to filter that information … and to decide whether an exchange or vendor releases a change that affects the filters you apply, and what you’re filtering out. But one positive of these systems is that they target specific individuals responsible for managing notifications, rather than blasting changes to everyone. But getting the most out of these will require changes to the working processes within financial firms.”

For those not ready to make a major change, or who need that extra human touch to augment whatever systems they currently have in place, Ropnoy also plans to soon offer another option, dubbed When In Short, Employ (WISE), an on-demand service where the vendor can step in to fill short-term staffing gaps or sudden increases in notifications, run its analysis offsite, and deliver reports to the client by email. Ropnoy is still fleshing out full details of the service, Nicholson says, but envisages that clients would sign a service agreement in advance—basically putting Ropnoy on retainer—and provide usage files, then use the service on an ad-hoc basis as needed.

“You may only need it once a year. It’s like an insurance policy,” she says.

But even the vendors recognize that the automation that underlies their competitive advantage relies on other factors falling into place first, such as standardizing the formats used to deliver notifications.

“Consistency in the way notifications data is delivered would go a long way toward improving this. Because at the end of the day, you can’t start to automate anything if the system doesn’t know what information is coming in,” Garforth says.

The way firms handle data notifications is crying out for change—not just because of the volume they need to process, or the complexity, or the financial and operational risks associated with failing, but because the current model relies on a single point of failure, Raskin says, which could be mitigated with better processes and tools.

At some point, “I’m sorry, I missed your email,” will be replaced by “I missed your email, but my system caught it, acknowledged it, and forwarded it to the right person who has already handled it.” Until then, grab the Tylenol, because handling data notifications may still be a headache for some time to come.