How Goldman Sachs Looked to Google To Reinvent its Data Management Strategy

One of the biggest challenges facing financial services firms today is not necessarily the amount of data they have to manage and store, but how much data they end up throwing away, given the fact that over the last couple of years they have produced increasingly large data volumes. 

At this year’s WatersTechnology Innovation Summit in London, Goldman Sachs’ head of technology, Joanne Hannaford, said that in order to tackle this challenge and create better data flows, organizations need to start thinking about their data in a more innovative, entrepreneurial way.

Hannaford explained that Goldman Sachs had to rethink its data strategy, given that the firm employs around 32,000 people globally, of which around 8,000 are engineers/computer scientists that work with and produce enormous datasets. “We have all the issues companies face in terms of dealing with data at scale,” she said. “We produce billions of lines of data and have large numbers of datasets in an extremely distributed way.”

Hannaford said that one of the biggest issues facing large corporations is how organizational boundaries and different divisions within them make it difficult to ask questions about data broadly, and then access the data without having to find it, put it into a database, and create ownership of it. 

“Historically, what we’ve done with data is put it into databases and these spaces required you to define your data upfront,” Hannaford said. “We had to understand how to deal with that data before we even got to the point of finding it.”

Creating Databases

According to Hannaford, Goldman Sachs resorted to creating new databases for the sake of flexibility, which resulted in the bank ultimately running 75,000 databases, in which it was impossible to cross-reference that data and introduce a common identifier for its business trades and transactions. The bank found the solution to that challenge by learning from a pair of trailblazers in consumer industries: Google and Amazon. 

“If you think about Google or Amazon, they have created database technologies, which make us completely rethink how we store data,” Hannaford said. “They have built database technologies that allow you to quickly assemble data; it’s more like a directory structure, where you can copy a file from one directory to another.”  She explained that this strategy makes it easier to ingest data into a database.

“Think about how Amazon reinvented itself,” she said. “It took its engineering department and made it from an expense into a revenue [stream], by adapting the technology it was developing for itself and applying it [to address its data challenges].”

Another challenge that needed to be addressed, according to Hannaford, was to solve the bank’s data query issue. Goldman Sachs opted to build a data lake underpinned by artificial intelligence technology, borrowed from Google’s semantic web, which sits underneath the firm’s Chrome web browser. 

“That allowed us to take data from all different sources and perform queries and ask questions [of it],” she said. “We defined the data upfront and we linked it to our sources, which gave us the ability to make sense and understand different questions from our data.”

Hannaford is firm in her belief that data tends to be an undervalued commodity across the capital markets, with large numbers of firms resorting to “giving it away,” at the expense of managing it efficiently and logically and treating it as one of its most valued possessions. “The lessons I’ve learned is that data is a first-class asset,” she explained. “It is the most valuable thing companies have. If you really value something, you understand who owns it and you understand how good it is. That’s why in financial services data is not really valued. For years we have been giving it away and we haven’t commoditized it.”