Big Data Is Big Stuff

When and how the concept of Big Data found its way into the financial services nomenclature is anyone’s guess. And while it is fair to say that its appearance was really just a matter of time, this is not just a financial services phenomenon—the gradual digitalization of almost every business vertical, including social networks, has rendered the Big Data challenge a global one.

Ask a dozen industry participants what exactly constitutes Big Data and you’re likely to receive a dozen different answers. Suffice to say that Big Data presents CIOs with a variety of technology and operational challenges, given that its sheer volume, married with the velocity and frequency of how those volumes are produced, and the variety of sources and formats of those datasets, means that traditional databases and data management tools simply cannot cope with the added demands placed on them.

On Valentine’s Day last month, Waters hosted a Big Data webcast, sponsored by the Intel Corp., Platform Computing, and Sybase. Speakers included Dennis Smith, managing director, advanced engineering group at BNY Mellon; Ed Dabagian-Paul, vice president at Credit Suisse; Daryan Dehghanpisheh, global director, financial services segment at the Intel Corp.; Scott Campbell, director of product management enterprise analytics, at Platform Computing, an IBM company; and Neil McGovern, senior director of strategy, FSI, at Sybase, an SAP company. Victor Anderson, editor-in-chief of Waters, moderated the event.

Challenges
A significant amount of ground was covered during the 75-minute event, which included four poll questions responded to by an average of 180 audience members, representing about 40 percent of the event’s total audience. The issue most central to the interests of the majority of the audience centered on the operational and technology challenges facing market participants when dealing with exceptionally large data volumes.

Smith, of BNY Mellon, says he believes financial firms need to rethink their approach when dealing with Big Data, especially from a business perspective. “I think on the business side, it’s always been about business insight,” Smith says. “But I think the environment has changed—from a business standpoint, you cannot use existing data or think patterns in anticipating future events based on past ones. On the technical side, there are challenges around the overall information infrastructure area, in that we cannot tackle this area with a physical orientation—it needs to be more of a logical orientation where you can move data to the processing or vice-versa.”

“We’re in a declining-margins business, and that goes for the industry as a whole, and I don’t believe we can continue to support the solutions we have on existing platforms. There is no reason that we couldn’t use a data warehouse to handle our volumes, but the cost per gigabyte for storing that data over time is what’s pushing us toward cheaper NoSQL Big-Data-type solutions.” —Ed Dabagian-Paul, Credit Suisse

According to Smith, the challenge facing financial firms can be boiled down to two key issues: from a business perspective, understanding the type of data firms have in their environment and the specific use-cases of that data; and on the technical side, developing the appropriate information infrastructure to deal with Big Data, key components of which are agility and extensibility.

“This is more than just a pure volume issue, and I think there are tremendous cost benefits with this from a scale standpoint, particularly when you’re looking at volume use-cases,” Smith says. “But I think the ability to handle various different data types depends on agility and having an extensible services framework. To my mind this problem is much more complicated than just volume.”

Dabagian-Paul, of Credit Suisse, says he disagrees somewhat with Smith’s assessment, explaining that from his firm’s perspective, for the immediate future Credit Suisse’s existing data management infrastructure is sufficiently robust to deal with its weighty data challenge.

“I’m going to disagree slightly with Dennis. I feel that we have very large data requirements, but from a technical standpoint, there is no reason that for the next few years existing data analytics and very large databases couldn’t address those demands from a pure technology standpoint,” Dabagian-Paul says.

“However, we’re in a declining-margins business, and that goes for the industry as a whole, and I don’t believe we can continue to support the solutions we have on existing platforms,” Dabagian-Paul says. “There is no reason that we couldn’t use a data warehouse to handle our volumes, but the cost per gigabyte for storing that data over time is what’s pushing us toward cheaper not-only-SQL (NoSQL) Big-Data-type solutions. As far as business problems that don’t map to relational databases are concerned, we’ve done some pilots and we have some edge use-cases, but that is not the bulk of our data and that is not where our problems sit. The current technology will suffice for the next few years, but it’s unsustainable from a cost perspective.”

Common Pitfalls
On the issue of what financial firms tend to overlook when embarking on Big Data initiatives, both Dabagian-Paul and Smith cite unforeseen complexities as the most common pitfall. “One of the things most people overlook is the skills needed to write to and develop a Big Data solution,” Dabagian-Paul says. “The majority of our developers are used to working in transactional databases and are used to having database administrators there to assist them, and moving that development burden to them is something they do not fully appreciate.”

Smith concurs with Dabagian-Paul, suggesting that the answer lies in developing a specialist data management team in-house, headed up by a data scientist, a title the financial services industry is likely to become increasingly familiar with as the Big Data challenge intensifies. “Most of the Hadoop deployments I’m aware of are custom-built and that isn’t trivial,” Smith says, referring to the Apache Hadoop free-license software designed to support data-intensive distributed applications. It enables applications to work with large numbers of nodes and extremely large data volumes.

“You need to have the appropriate talent in-house, and as time goes on, in order to address some of these problems, you’re going to need data scientists—people who can do some of the predictive analysis sitting with the technical staff in order to craft these solutions,” Smith says.

Dehghanpisheh, of Intel, says he agrees with both Smith and Dabagian-Paul about the operational-technology trade-off, saying that merely deploying cutting-edge technology is a pointless exercise unless firms have the appropriate specialists in the right positions to ensure that the infrastructure is optimally aligned with the business.

“I think it’s an operational and also a technology issue,” Dehghanpisheh says. “The operational challenge is something that is going to be vexing. You can have the best technology on the shelf, but you also need the operational capacity and the human capital to deal with it. And that is where I think one of the most interesting challenges for financial institutions and market participants is going to come from in the future. The last decade has been about hiring the best possible quants; the next decade will likely be about hiring the best possible data scientists.”

Aggregation Aggravation
McGovern, of Sybase, says he would add data aggregation to the mix of issues that financial firms need to address in order to satisfactorily negotiate the Big Data challenge, a factor that crops up more often than not whenever discussing data management predicaments.

“When we talk to our customers, in addition to what’s already been said about addressing the technology challenges and ensuring that you understand the data, one issue seems to come up time and again: data aggregation,” McGovern says. “A lot of our customers underestimate the amount of work it takes to collate the data—we are talking about Big Data here, so typically we’re talking about the collation from multiple sources and different types of data.”

Campbell, of Platform Computing, says he agrees with McGovern’s aggregation sentiments, but broadens his take on the challenge facing financial firms, citing operations, resources, and firms’ infrastructures as areas that need to be addressed as a matter of course.

There are three areas where people are struggling to tackle Big Data, according to Campbell. “One area is the resources themselves and the skills sets needed in this area. The second is the operational management and trying to avoid [developing] a single-purpose infrastructure,” he says. “Finally, data aggregation is probably the one area where we see more and more effort, not only in terms of the requirements, but we’re seeing some of the bigger use-cases in the Hadoop space tied to data aggregation as a mechanism for dealing with the [Big Data] problem.”