For market data management professionals, it’s the Holy Grail: being able to buy the data you want, when you want it, and pay for what you use, rather than paying a fortune for always-on, voluminous datafeeds or bundled services of which a firm uses a fraction.

While pay-as-you-go pricing is not yet a reality, some data providers—notably, among brokers and exchanges—are laying the foundation by providing “self-service” portals where clients can subscribe to content and services without having to contact a sales rep, streamlining previously manual permissioning and onboarding processes.

Industry observers say this approach represents a shift away from old-style commercial agreements, and is more geared toward clients’ needs, creating greater flexibility for consumers.

For example, in April, inter-dealer broker Tullett Prebon combined its data and analytics, post-trade services, and investment analytics (acquired from Liquidnet) offerings under a new brand, Parameta Solutions. Part of the reasoning for this was that data is at the heart of many of the broker’s non-data offerings, which in turn feed information into the data business.

Along with that, the broker rolled out a self-service portal for the unified services, allowing clients to choose from its services online, and providing a choice of delivery mechanism as firms increasingly move to the cloud. The offering is not intended to replace existing ordering and delivery methods, but to adapt to clients’ changing needs.

“Clients can still access our data via our data partners, but we also want to meet new requirements and be able to deliver data to clients where they are,” whether on-premise, in offsite datacenters, or in the cloud, says Roland Anderson, CIO of Parameta. “It’s all about efficiency and flexibility as companies develop their technology stack.”

The portal allows users to browse the broker’s content, obtain samples of datasets for testing, and execute a contract online. Parameta then delivers the data directly into firms’ Amazon Web Services S3 cloud storage. Anderson says that from conception, Parameta envisaged the portal as being cloud-native.

“We’re producing data in the cloud, and we wanted to make it available in the cloud,” he says. “We’re trying to recreate an Amazon-style shopping experience.”

The project was something of a technical departure for the broker. To create the portal, Parameta set up a new internal team to work on the user interface and enlisted a group of “external data experts” with experience in building e-commerce platforms for financial firms. But it also employed technologies more associated with Facebook and Airbnb than capital markets, including the analytics query language Presto, Airflow for reporting, and the Apache Kafka open-source event and data streaming platform.

Parameta isn’t the only broker data arm making use of self-service portals. StoneX Group, formed by the merger of Intl FCStone and Gain Capital, recently rolled out its MyStoneX self-service portal, aimed at the firm’s commodities trading clients. The portal provides a single point for clients to access “market intelligence, risk measurement, scenario analysis, and market execution” for futures and options trading.

All the firm’s over-the-counter (OTC) clients receive access to proprietary content on the portal at no extra charge. To place orders through the platform, traders place a request, then must submit to a compliance and risk process. But in keeping with the aim of streamlining access, “It’s a quick turnaround,” says Karrissa Allyn, vice president of business strategy at StoneX Markets.

“We’re empowering the customer, giving them access to pricing and educational tools—from the basics, like learning what a call option is, to more sophisticated options strategies,” Allyn says. “And with execution, which we recently rolled out, we’re giving customers the ability to place their own executions, and choose strategies, from conservative to aggressive.”

The tools available via MyStoneX allow its brokers to have more meaningful conversations with clients about how to customize trading strategies and enable the firm’s clients to more easily track commodities and prices, provide a better service to their customers among grain and other commodity producers, and ultimately to win more business and trade greater volumes.

“This was a way for us to do something new and stand out against our competitors. The biggest thing is the ability to get that information quickly because when a producer calls, they want to make decisions quickly,” says Matt Zeman, vice president of grain at Iowa-based Cooperative Farmers Elevator, which uses the StoneX platform.

Though MyStoneX is new, it’s been in the works for around four years and is part of a larger digitization project at the broker. “We’re in the process of digitizing our entire product line—from vanilla swaps through to more complex products. I would say we’re now 70% along that process,” says Mark Mauer, CEO and president of StoneX Markets, who says the portal will establish StoneX at the forefront of a commodities industry that is often slow to adopt efficiencies.

Part of that timeframe was spent reassuring the firm’s brokers—who were initially skeptical—that the platform was not intended to remove brokers from the process, but rather to free them up and give them more time to focus on relationship-building with clients. “This was driven by demand from our customers and brokers. After the initial build, our brokers realized the value this could add to their workflow, and so following that, enthusiasm for more features and functionality grew exponentially,” Mauer says.

But self-service portals aren’t totally new: CME Group has been operating its self-service data portal for five years, though it was able to get a head-start over others by employing third parties—and, like Parameta, making extensive use of the cloud—to provide specific components, rather than building everything in-house.

CME’s initiative began as a more efficient way of distributing data, but now allows the exchange more time to focus on product development than on sales and distribution.

Traditionally, to obtain historical data from CME, a client would need to phone a salesperson to license and buy the data. Then the exchange would load data onto a hard drive and physically ship it to the client.

“A few years ago, we set out to figure out what’s in the best interests, not only of CME, but of our clients, and how could we deliver services quickly and easily to clients,” says Trey Berre, global head of CME Data Services at CME Group.

In 2016, the exchange enlisted Canadian data technology vendor TickSmith to revamp DataMine and provide a more modern and—key to its expansion plans—scalable platform. TickSmith deployed its Enterprise Data Web Store and its Gold (Gather-Organize-Leverage-Distribute) sales platform. Now, existing clients can get their hands on data within minutes, while new clients can get access within a couple of days, after completing CME’s know-your-customer (KYC) requirements.

The following year, CME tapped DataBP, which usually builds marketplaces for organizations to automate data sales activity, to build a web-based data licensing portal to automate the manual processes of signing and returning contract documents, drastically reducing the time it takes for clients to license data. And in 2018, it added the ability to pay for data using a credit card.

“For large banks, their data requirements are complex and take more time. But for smaller institutions, this has been very useful. And if they want to add other products, they just need to go through a maintenance procedure that can be completed in hours, if not minutes,” Berre says. “Making data available on a self-service basis helps us focus more on building better products for clients. So we feel it’s a win for everyone.”

Since then, CME has used DataMine to expand the range of datasets it offers via self service, including 28 proprietary datasets—including top of book and market depth data, time and sales data, US Treasury data from Brokertec, as well as liquidity tools—and data from 17 partner organizations ranging from exchanges to alternative data vendors. The exchange also offers a free 30-day trial of the datasets available.

Berre says one of the aims was to provide more variety in how clients could access CME data, and also to appeal to a wider variety of potential clients. The exchange initially envisaged that the portal would appeal to sophisticated retail traders. And while it has seen some takeup from that market, it has also seen demand from crypto trading firms wanting futures and options data—not just on bitcoin futures, for example, but also on gold futures and CME’s Dow Jones Indexes products because of the interplay between these markets—as well as from firms wanting to cut costs by leveraging cloud delivery to reduce their legacy infrastructure footprint.

“We are definitely seeing more of these data marketplaces, but I do feel we are a way off from seeing the true impact,” says Bernardo Santiago, founder of market data consultancy S4 Market Data. “These self-service portals are seeking to add a level of ease to the data acquisition process,” which is evident for the data providers, though Santiago says he is yet to see that translate to ease for consumers, as the data onboarding process must still be managed by consumers outside of the vendor’s self-service portal.

Cautionary Notes

While data professionals say the portal-based approach offers potential benefits in terms of efficiencies and time savings, they note that it also raises some red flags from a data management perspective. First, depending on who has access to the portals, firms could see unchecked rises in spend on data—for example, if end users on trading desks or data science teams are allowed to sign up for content without adhering to established processes governing how a firm buys data.

“If you open this up to end users, it potentially conflicts with the vision of the market data group around controlling costs. If business users want to buy data to feed into applications, the market data group could get blindsided by costs and by where data is being used, and possibly get tripped up during an audit process,” says one market data manager.

Ensuring that the process is properly set up contractually and can be accounted for in a firm’s inventory management system will be key, the market data manager adds. “If you only put it into the hands of market data folks, they should be privy to the terms and conditions of any given agreement,” he says.

Typically, end users request data via a central procurement or market data management team, which ensures users are properly entitled and licensed, that the contract is managed properly, and usage and costs are tracked via the firm’s inventory management system. This process, in particular, would at best be manual when acquiring data via self-service portals, requiring data managers to update their inventory systems, and at worst might end up being inaccurate if end users buy data without enlisting market data management and properly recording spend and usage.

“In the software-as-a-service (SaaS) world, you hear about ‘shadow IT’ where people spend money on technology themselves. It always happens, and market data teams would have to scoop that up and manage it centrally. Plus, end users have been well trained to not do that. We don’t see these portals as ways for vendors to work around data teams, but rather that they realize that ordering data in person or by phone is not the most efficient way to work anymore,” says Richard Mundell, chief product officer at inventory management software vendor TRG Screen.

Mundell draws another comparison to the SaaS world, noting that providers ranging from Microsoft to Salesforce to cloud operators allow clients to provision access to services using API calls, whereas client portals remain a manual link in the purchasing chain.

“Self-service portals do make life easier because they remove the need for phone calls and emails, so it’s better for those selling data because they can automate the sales process. But it’s not necessarily better for consumers, because while they do want automation, they also want straight-through processing. And with these portals, you still need to have humans involved. Clients still need to re-key everything that they would have called their rep about in the past,” Mundell says.

For example, if a bank uses TRG’s Optimize Spend platform, a business user can log into the platform, see what services the firm has available, and request any service available in its current catalog of services. That request is routed for approval, and to the market data team to acquire the data. But that’s where the process hits a manual roadblock.

“When it reaches the market data team, they still have to go to these portals or phone or email the vendor. So the fulfillment remains manual. That’s the piece that could be automated,” Mundell says. “But the vendors don’t have APIs set up to handle those requests. The thing is, all these portals have APIs on the back end—they just don’t expose them to customers.”

In addition, Mundell says that by not leveraging APIs, data providers are potentially “leaving money on the table,” because APIs would allow them to raise their profile and expose their services to potential users who may be unaware of their existence. “If vendors provided APIs, in addition to automating fulfillment, it would allow end users to browse not only what their firm has already licensed, but the entire catalog of all services available from all vendors who expose their APIs,” he says.

But if brokers and exchanges can allay firms’ data management concerns, the market data manager says firms should test the portals. “You would definitely need to trial it. You would need technical people, representatives from the user community, finance and admin people, and information security staff, and bring them all to the table because no single person will be able to cover all the bases and anticipate any potential pitfalls,” he says.

At the end of the day, self-service portals aren’t the holy grail of market data provision. Not yet, anyway. But they’re certainly a step toward making that a reality.