JPX Mulls Pooling Content into Data Lakes

As exchanges expand the amount of data they collect and distribute—which increasingly involves new data types that don’t conform to “traditional” market data structures—they are revisiting the potential of data lakes to deliver the breadth and depth of capabilities to store and surface both structured and unstructured data at any scale in the most efficient manner.

One such potential exchange user, the Japan Exchange Group (JPX), is looking at the potential of data lakes to support its strategy of finding new ways to add value to existing proprietary data.

Data lakes are centralized repositories that store data in an ungoverned and unformatted way, allowing firms to deploy tools to run different types of analytics and machine-learning applications without having to move the data. Used and implemented correctly, they could help organizations yield significant improvements in their ability to store and manage certain datasets, particularly when it comes to experimenting and building out potential new products.

It’s this aspect that particularly appeals to JPX CIO Ryusuke Yokoyama, who says that—while not yet set in stone—the exchange is currently thinking about using a data lake to help achieve this.

“At this moment we only have a data warehouse. We don’t have a data lake or data pool where we can inject all the data. We are thinking about it but we haven’t realized it,” Yokoyama says.

A data warehouse stores structured data differently from a data lake, which organizes data while accounting for business processes and determines how data sources are analyzed. When a purpose for the data has been identified, only then will it be loaded into the warehouse.

According to advisory firm Deloitte, data lakes should be divided into three zones: data loading, defining user access and security, and creating a more user-friendly environment. The first zone consists of the raw and untransformed data direct from the source. Zone two is the data sandbox, where the data can be lightly processed, cleansed and combined for exploration and analysis. The third zone consists of the refined data that is ready to be used by the data warehouse or manipulated for analysis.

JPX is currently replacing its internal systems, including the system that accumulates data, Yokoyama says. “We are constructing a system considering the convenience of data usage and the smoothness of cooperation with other systems,” he says. 

Like other organizations, JPX is trying to gain more value out of the data it already has in order to better cater to its customers and market participants. Adding to this, artificial intelligence (AI) and machine-learning technologies are also becoming more sophisticated. “So we believe that data utilization will expand more in the future. JPX holds various data, so I think that what kind of value we should add to raw data is the issue for us,” he says. 

But while the exchange sees the opportunity presented by data and is considering ways it can expand its data business, exactly how it will do so is still uncertain at this point. 

“Everybody is scrambling to the data business. I think it has huge potential in JPX’s future growth and expansion. What we’re doing right now is providing the plain and simple raw data, like stock prices and volumes. What we have to think about going forward is how we can add value to that data and sell it to customers, and how we can work out their needs and appetite for data,” Yokoyama says. 

That may even mean that JPX will need to direct some efforts towards development of AI technology “to filter and pick out necessary data,” he adds.  

Though deciding how JPX can add value to its data and repackage its data as new products is a business decision, rather than one that will be determined by the data and IT teams, the IT team still has its work cut out. “On the IT side, we have to think about how we can handle the massive data. How can we build the data handling platform that can respond to the business needs?” Yokoyama says.