The IMD Wrap: Market data budgeting for 8-year-olds

As I get older, I find myself referencing things that just get blank stares in response, like 1980s rock music, old British TV comedies, and GCE and O-Level exams: things I grew up with at school and at home. One thing from school was a class called Home Economics that taught useful household lessons such as wiring an electrical plug correctly, cooking and baking, and shopping for the right ingredients on a budget. Maybe there are market data professionals out there who remember this. Or maybe they skipped that class to go smoke behind the bicycle sheds.

Either way, budgets prove to be one of the most challenging elements of market data management, probably because market data professionals at sell-side and buy-side firms are caught in the middle between supply and demand, with little control over either: the business side demands data, from management to traders, portfolio managers, risk and compliance, and the back office. Not only that; they also don’t have control over the purse strings: they may submit a planned budget for the year, but that may get overruled or sent back by management with instructions to reduce spend. So, almost every aspect of what they’re expected to do is decided by someone else, and they’re told to just smile and get on with it.

Over recent years, those smiles have worn thin as data professionals have found themselves overworked, underfunded, underappreciated, and struggling to make ends meet for their firms at a time when data costs are rising, the volume of datasets—including new data types like alternative data—is rising, and consumption and usage are rising. 

If you think prices are rising faster than usual, you’d be correct, and there are several reasons for that. First, many providers paused increases during the Covid pandemic, or allowed users to access data while working from home that would under normal circumstances have attracted additional fees, such as Bloomberg, which to its credit gave users access to its terminals via a disaster recovery provision during lockdown and beyond. Once that period was over, the vendor then had to convert those who had taken advantage of the provision to properly licensed services.

Another factor is more macroeconomic: we’ve seen a recent spike in inflation, which—while it’s now falling—coincided with the time when vendors were adjusting their pricing. Thus the “X-percent above inflation” increases turned out to be much larger than they would have been during more normal cycles. 

And besides going over budget, there’s another danger of rising costs and ever-tightening market data budgets: the rise of “shadow” spend; if a trader or data scientist has their own budget, they can go out and buy datasets themselves. Yes, it gets them what they want, but buying ad hoc without any central controls means that spend goes largely unmanaged. One trader can duplicate what another is already buying. Because those datasets are not managed on the firm’s central inventory, they’re not being negotiated properly, or rolled up into bigger deals, and their usage isn’t being tracked to save money if it turns out to be under-utilized. 

And it’s not being analyzed and compared by those who have expertise in data and suppliers, so cheaper alternatives may be ignored altogether. The result? By trying to keep data spend low, spend actually goes up. And, because that spend isn’t being properly managed, there’s a greater risk of users not fully understanding the terms of their contracts and not complying properly, which can of course result in fines and penalties.

So, to review, firms want to lower their data costs, and get more data, while vendors and exchanges want to sell more data and raise prices. And even if you understand and respect both sides, no matter how you do the math, that’s not going to work.

When my kids want to buy something but don’t have enough pocket money, the simplest solution is to take items out of your shopping basket until you can afford what’s left. (Pat on the back for being a good, miserly parent.) But while that’s possible for financial firms to a degree, there are core data costs that simply can’t be touched. You might be able to ditch certain services if they’re “nice-to-have” rather than “must-have,” but the truth is, there isn’t a lot of data spend that its users would deem discretionary. There’s the core data, then services that provide more insight, or better and faster decisions to help generate alpha or perform risk checks more accurately. Generally speaking, simply discarding data or services isn’t a viable option.

So, kids, instead of simply putting something back on the shelf, why not look for a cheaper alternative? (Another pat on the back for encouraging them to be equally miserly parents when they grow up).

Data professionals have used this strategy where they can. For instance, they can replace the most expensive terminals and data sources with cheaper alternatives for those who don’t need the all-singing, all-dancing model, but really need, say, a bunch of prices that could come from a workstation displaying delayed data rather than a low-latency datafeed or even a real-time display. 

However, that can lead to a mish-mash of products from different providers that may not integrate well, and where inconsistencies may exist in the data. If 60% of data consumers in an organization use Vendor A and the remainder use a mix of Vendors B, C, and D, there’s always the risk that the data from one doesn’t precisely match that of the others. And if you’re using a price from one vendor to place a trade and what should be the same price from a different vendor, and those don’t match, your trade either fails or takes longer to work out.

Also, while there may be thousands of companies providing different types of data, there are only a few that can provide all the data needed by financial firms. Thus, as a recent report by UK-based Substantive Research concludes, firms are “negotiating from a position of weakness given the lack of viable alternatives in many areas”.

Another strategy I try to teach my kids (being such a good, miserly parent) is to ask whether there are any discounts, or ask for a deal when buying more than one item: “Hey, if I’m buying 6 of these $5 packs of Pokemon cards, would you take $25?” That tactic rarely works in supermarkets with fixed prices and POS scanners. But sometimes the nice lady at the checkout will throw in a lollipop for them instead. And sometimes the mom-and-pop greengrocers will let me throw a couple of limes in my bag of 5-for-$3 lemons.

Certainly you can get better deals when buying in bulk. And data, just like avocados and bananas, is a perishable item. The fresher it is—or, the lower latency it is—the more it costs. Once it starts to go soft and brown, it goes in the $1 bin, just as how after only 15 minutes, data is considered almost worthless by some. Vendors and exchanges want to sell as much data as possible while it’s still fresh, rather than having people pick over the rotten delayed stuff. An enterprise license can help lower per-user fees, but at the cost of a larger and more expensive overall contract. Maybe a discount will encourage you to buy more. 

Problem is, discounts are just as inconsistent as the price lists themselves. According to the Substantive Research report, prices charged as well as discounts offered can vary wildly on a client-by-client basis. For example, according to the study, one index provider’s pricing varied by up to 1,410% between different clients, while among ratings data providers, the level of discount offered averaged 39% based on list prices that can vary by as much as 200%.

One reason cited by vendors to justify fee increases is that firms have been receiving such generous discounts that they need to be brought back into line with the normal fees paid by their peers. Yet, if discounting is so endemic and the “price lists” vary so wildly, then what exactly is normal pricing, and who is really paying it? 

Firms often feel compelled to renew contracts at seemingly high price increases that vendors claim are already discounted over what that actual price increase would be. As the Substantive Research report notes in conclusion, “this is a cycle that continues in perpetuity”—the list price goes up so firms never catch up.

So, are prices set to keep rising uncontrolled? Not if regulators have anything to do about it. European regulators, for example, introduced the concept of a “reasonable commercial basis” for fees based on a reasonable markup over and above the cost of producing the data. One problem is that “reasonable” was never fully defined, leaving it much like US Supreme Court Justice Potter Stewart’s definition of obscenity: “I know it when I see it.”

Last year, French regulator the Autorité des Marchés Financiers found “significant shortcomings” after conducting spot checks on four data providers and exchanges to test their implementation of the Markets in Financial Instruments Regulation rules around data. The AMF found data policies to be lacking, and found that half of the data providers surveyed could not demonstrate a link between cost and pricing. Also, while three of the four disclosed that their prices added a margin, they could not explain the basis for calculating that margin.

This March, in the UK, the Financial Conduct Authority is set to publish the findings of its year-long wholesale market data study to identify competition issues around benchmarks, credit ratings, and market data. And meanwhile in the US, the Securities and Exchange Commission’s attempts to rein in exchanges’ data pricing appear to have been stifled by the courts, at least for now.

I guess as I get older, aside from all my 80s references, I’ve come to realize that the only certainties in life are death, taxes, and market data fee increases.