The European Commission (EC) has awarded Dublin-domiciled consultancy Accenture the contract to provide the European Securities and Markets Authority with big data capabilities that will operate on Microsoft Azure. The EU legislator officially signed off on the deal in the first week of July, a spokesperson for Esma tells WatersTechnology.

“All tenderers were evaluated on both technical quality and financial proposal by an evaluation committee composed of Esma and the commission staff,” the spokesperson says. “Accenture won as they presented the best quality proposal overall.”

In a typical tendering process, a company publishes a specific technical requirement and vendors respond with an offer on their services. With the help of the EC, however, Esma was able to use an electronic public procurement tool called the Dynamic Purchasing System, in which vendors must qualify to join a marketplace of suppliers from which authorities can choose. 

The Esma spokesperson says that Accenture and Esma have now commenced work on the big data analytics project and will spend the next few weeks mapping out the development plans.

In June, WatersTechnology spoke to Paul Hussein, team lead of supervision and analysis systems at Esma, about the new implementation, as the regulator began to see cracks emerging in its legacy systems. At the time, Hussein said the regulator’s old Oracle relational database was straining to ingest the growing volumes of data being reported to it by financial firms following the rollout of expansive regimes such as the Markets in Financial Instruments Regulation. The legacy system was unable to meet the urgent need for scalable computing power.

“We’re pushing the limits of what we can do with a relatively simple Oracle,” he said. “The data that is sitting within the Oracle databases—we have to query it, we have to analyze it, and we’re finding that querying the data sitting in this old tech is taking too long.”

Once fully implemented, Esma will be able to leverage big data tools built by Accenture, such as self-service analytics, search functions, data integration services, and a data dictionary. Additionally, its data scientists will be able to code new capabilities on top of the self-service analytics. ­­

Esma completed its cloud migration program of shifting data and operations to Microsoft’s Azure cloud platform more than a year ago. This move facilitates the big data capabilities, which require the storage of terabytes of data and heavy processing power to run the analytics. The regulator currently uses vendor Talend’s extract, transform, and load capabilities to push reporting data into Oracle, but in the new­­ implementation its data science team can transfer data to the Azure cloud environment where they can use Accenture’s tools to perform more intense calculations in Python.

“It means that when we come to manipulate the data, we’re not held back by the capabilities of the platform in terms of computing, and we can easily distribute the analysis,” Hussein said in June.

The big data project consists of two phases: integrating Accenture’s tools with Azure and then applying them to a real use case. Gartner, an IT advisory firm, helped Esma to write the technical specifications for the procurement process. Rather than asking for a specific out-of-the-box solution, the regulator wanted individual third-party functions that can be spun up on demand on Azure in a similar way to a pay-as-you-go-model, a type of service that is necessary for organizations with limited tech budgets.

“There are capabilities that you can switch on and pay per month, and some are per user, some per gigabyte or terabyte. So, there are different cost models for different functions, or what Gartner calls [control] towers,” Hussein said.

The EU regulator needed to update its technology stack as it prepares to take on more responsibilities. Following a review in 2019, the European Supervisory Authorities, of which Esma is one, decided to grant it more power. Starting next year, Esma will authorize and supervise the Data Reporting Service Providers, which include Authorized Reporting Mechanisms and Approved Publication Arrangements.

In the second leg of the big data project, Esma will apply the Accenture tools to the use case involved in manipulating reported data collected from DRSPs, including comparing trade data published by APAs with transaction data reported to the National Competent Authorities by Arms and trading venues.

Esma employs 50 data scientists and analysts across its various departments. Alexandru Dincov heads the information and communications technology (ICT) unit, which is made up of 38 IT and data specialists. The unit is split into three teams, one of which is the supervision and data analysis systems (SAS) group, led by Hussein. The SAS team provides support for the ICT systems, along with data engineering and data science work. Hussein recently finished building out his SAS team of nine people, including a project manager, service manager, architect, support engineer, business analyst, data engineer, data scientist, and team leader.

In the coming months, now that the contract has been signed off, Hussein’s team will be tasked with learning and testing the regulator’s Accenture’s big data tools. Upon completion of their training, they will then apply the capabilities to the DRSP datasets. If the project proves successful, the new vendor solution could be used to tackle other regulatory reporting data.

However, there is no rush to do this right now, Hussein told WatersTechnology. “The first thing everybody asks is, ‘Are you going to run the existing 20 other systems on Azure [and apply the new vendor capabilities]?’ And the answer is, ‘No, not in the beginning,’” he said. “It’s a complex technology and there’s a lot of learning to do, so we’ll run this first system, make sure that it is successful, and then we’ll see.”