Trading firms are notoriously secretive about their use of any technology that they believe delivers an advantage, so gauging actual usage of hardware components such as field-programmable gate array (FPGA) processors can be hard for those outside the firms themselves. But one can gain an idea of demand by looking at recent mergers and acquisitions activity, and the willingness of major chipmakers to invest in these technologies.

Last year saw three significant acquisitions in the space, with two major hardware providers bolstering their FPGA offerings, and one FPGA provider looking to expand its base of services.

In April last year, Intel acquired Omnitek, a Basingstoke, UK-based provider of FPGAs for video systems. The same month, FPGA maker Xilinx acquired Solarflare, which provides ultra-low-latency network interface cards. Then in December, Cisco Systems announced that it was acquiring FPGA and hardware switch provider Exablaze. These follow Intel’s 2015 purchase of Altera Corp.

Faster, stronger, better—that’s how people typically characterize FPGAs. In the capital markets, this piece of hardware found a home in high-frequency trading (HFT) shops, where speed is its main bargaining chip. And as time has progressed, FPGAs have become “table stakes” for banks, asset managers, and vendors trying to supercharge everything from elements of trading and risk management to deep analytics. But while speed is still certainly the hardware’s main calling card in the capital markets, FPGAs are becoming more democratized within the financial markets and are also helping when it comes to artificial intelligence (AI) development.

Although FPGAs are already widely adopted—Prasanna Sundararajan, CEO and co-founder of Reniac, a data engine provider that accelerates data-centric applications, calls them the “worst kept secret” in terms of computing performance and cost advantage—there are still some factors holding them back from achieving greater success, such as certain forms of programmability (which is also a strength), challenges of interoperability with new analytics techniques, and an overall industry drive toward software.

Nevertheless, market research and strategy consulting firm Global Market Insights expects the market size of FPGAs to grow from $5 billion recorded in 2018 to $13 billion by 2026.

“The FPGA industry is moderately competitive owing to the presence of limited strong and leading market leaders worldwide with their broad distribution network. These companies are focusing on R&D activities to develop new FPGA devices with enhanced features and capabilities,” said the research firm in a recent report on FPGAs, citing applications such as infotainment systems, speech recognition systems, and machine vision as catalysts for the increasing demand for these devices.

As FPGAs become easier to access and use, a natural vertical to expand into would be artificial intelligence and, specifically, various forms of machine learning (ML) that require massive datasets for training and analytics. As FPGA providers look to stay relevant in the world of capital markets, they’ll have to expand beyond HFT, which is becoming something of a zero-sum game. Machine learning might just be the future.

Magic Tool for AI?

There’s a reason why AI and ML are increasingly butting into the FPGA conversation. Today, AI and ML calculations and workloads generally use central processing units (CPUs) and graphics processing units (GPUs). But due to some of the limitations of these, FPGAs are gaining traction.

Assuming that firms are at the point where they know how to build an ML model, and have the data available to build that model and train it into a high-quality model with minimal biases, the next challenge is deploying that model.

According to Ludovic Larzul, founder and CEO of Mipsology, a startup focused on acceleration for deep-learning inference, if the deployment of an ML model is internal for small applications, the challenges are small. “However, if the application is for the service of the larger public, then it requires considerable computing power. The challenge is to have a reliable hardware infrastructure that can sustain the loads without failing. FPGAs are better positioned for that task,” he says.

Even though GPUs were adopted early on for ML training, they haven’t been widely used in industrial or professional applications, whereas deploying ML at scale is extremely important to banks and financial institutions, Larzul says. “Their design is still driven mainly by gamer needs, which are not as sensitive to quality as financial services. Furthermore, GPUs are a very poor fit for the inference/deployment stage of AI,” he says.

The inference stage of AI refers to the timeframe during which you can derive insights from the model. According to a report by IBM, the inference phase is the sum of all prior parts—data and training. “If your data was bad or the training was inaccurate, inference will suffer. Without proper inference, all prior efforts are for naught,” IBM stated in the report. 

Meanwhile, Reniac’s Sundararajan says one of the biggest bottlenecks when running AI and ML models is around handling data in a real-time manner. “We have figured out how to collect and store more data than ever, but the issue is with operationalizing the data, particularly as it is being collected,” he says. “The key to that operationalization is an extremely fast database, and database software alone has been hitting a ceiling of optimization.”

That’s where combining FPGAs with software can work to provide the speed and scale required for complex and data-intensive applications like AI and ML. Two of the key advantages of FPGAs are their programmability (which can also, conversely, create challenges) and flexibility: They can be reprogrammed even after the circuit has been designed and implemented, meaning they can be updated to perform a completely different task down the road. Also, FPGAs have additional gates and wiring, which allow them to be flexible.

“Machine learning can use thousands of nodes to execute training on traditional servers and database nodes, but that is really due to CPUs, which communicate in a less efficient manner,” Sundararajan says. “FPGAs communicate with one another and other parts of the tech stack at a very fast rate, which provides that speed and scale that AI and ML require. There are other solutions, like adding more traditional hardware to the mix or adding database nodes, but the solution is akin to attempting to support the data of tomorrow with yesterday’s datacenter—inefficient, cost-heavy, and complex.”

Therefore, he says, if financial institutions can realize that the shape of the datacenter is changing, implementing FPGAs can become a critical competitive advantage. “The giants and hyperscalers are already taking advantage of this technology, and it truly is the worst kept secret in performance and cost advantages right now. Microsoft, Alibaba, and Amazon all have instances with FPGAs under the covers—and there is no reason a company of any size couldn’t benefit from their use,” he says.

FPGAs have the flexibility to integrate more features at the hardware level post-installation, and more specifically for AI/ML, they provide parallel compute capability with a large density of memory and high bandwidth, says Mipsology’s Larzul.

Sumeet Puri, global head of field technology at Solace, which provides messaging technology, concurs that FPGAs are very good at parallel processing. “In our case, publish, subscribe, message streaming input/output (I/O) is a very parallel problem. So that’s how we leverage FPGAs, and that’s how we get the performance that they provide. Similarly, whether it’s cryptosystems or AI, machine-learning systems, it depends on how you use the FPGA, but they are great for parallelism,” he says.

Puri further explains parallel calculations this way: “When you search for something in the Google search engine, say you look for movies [showing] tonight, or search for the best restaurants. These lookups are completely parallel, and their learning is also parallel. So, Google is trying to learn about me as a person. Okay, I like food, and for further personalization, the machine-learning engines are watching my patterns, and they’re saying ‘Okay, Sumeet likes food.’ So next time, they will suggest more food-related options or anything related to food or cuisine. While in your case, it’s learning that you like movies. So that’s the machine-learning part, and it’s also parallel. Google was able to scale their search because of these capabilities themselves, and so did a few others. And now, with the advent of TensorFlow, with Google’s open source, a lot of other trading applications are using some of these capabilities from various cloud providers.”

For their programmability and flexibility, FPGAs are being presented as a magic tool for AI and ML workloads. However, according to Henry Harrison, co-founder and CTO of Garrison, a London-based cybersecurity solutions provider that uses FPGAs in an approach called Hardsec, nine out of 10 users are better off focusing on improving their software or finding more efficient algorithms—though he also notes that there are some scenarios where FPGAs can perform significantly better than CPUs. (Click here for more on the cost-benefits of FPGAs versus software.)

“It is entirely feasible that someone will discover some key AI/ML techniques that lend themselves perfectly to FPGA acceleration—but I’m not personally aware of any to date. There will, of course, be case studies, but rarely do those case studies compare the FPGA implementation with a software implementation that’s had the same level of engineering attention. Very often, they are comparing the performance of heavily optimized FPGA implementations with non-optimized software,” Harrison says.

Getting Harder

On the FPGA-vs.-GPU argument, Jim Handy, a specialist in semiconductors, memory, and solid-state drives at Objective Analysis, a semiconductor market research firm, says that for a few years, GPUs were the preferred way to embody machine-learning or training systems. “Computer architects recently decided that they could do better by converting to FPGAs instead of GPUs. With the Altera acquisition, Intel has the opportunity to win back some of these designs,” he says.

Handy explains that GPUs are designed to solve matrix manipulations for matrixes of fixed dimensions, but AI algorithms use varying matrix dimensions. “FPGAs support this flexibility better than GPUs do,” he says.

Back in 2015, Intel bought Altera, an FPGA technology provider, specifically to build next-generation semiconductors. At the time of the acquisition, Intel’s then-CEO, Brian Krzanich, said, “We will apply Moore’s Law to grow today’s FPGA business, and we’ll invent new products and make amazing experiences of the future possible—experiences like autonomous driving and machine learning.”

This February, Intel announced its first next-generation structured ASIC—short for application-specific integrated circuit—named “Diamond Mesa,” aimed at 5G wireless applications. Structured ASIC devices are an intermediary technology between FPGAs and standard-cell ASICs, and allow for faster time-to-market and lower design cost.

Diamond Mesa is designed to complement Intel’s portfolio of processors and FPGAs, to deliver the high performance and low latency required for 5G networks. In a statement, Intel said, “Structured ASICs like Diamond Mesa provide a minimum-risk optimization path for workloads that do not require the full programmability of FPGAs, targeting double the performance efficiency versus the prior generation, and uniquely position Intel as the only provider delivering a full silicon platform foundation for network infrastructure.”

Early access to Diamond Mesa is now open to some Intel customers, while full production is expected to start in 2022.

The Future

Xilinx’s acquisition of Solarflare is a different side of the same coin, compared to Cisco and Exablaze (see “Case Study” below). Xilinx is already a major FPGA provider, but Solarflare could help the company expand its machine-learning capabilities.

Xilinx said the acquisition would help it accelerate its “datacenter-first” strategy and transition to a platform company. It will combine its FPGAs, multiprocessor system-on-chip (MPSoC) and adaptive compute acceleration platform (ACAP) solutions with Solarflare’s ultra-low-latency network interface card (NIC) technology and Onload application acceleration software to enable new converged SmartNIC solutions.

Xilinx and Solarflare have been working together on advanced networking technology since Xilinx first became a strategic investor in 2017. Its first joint solution—a single-chip FPGA-based 100G SmartNIC—can process 100 million packets per-second receive and transmit, at less than 75 watts.

But an analyst who covers Xilinx sees the acquisition as a move to strengthen the company’s AI and ML capabilities. “I think there’s a trend toward not just machine learning on a single machine, but fabrics of connected machine-learning components,” he says.

As machine learning takes on greater importance in the capital markets for risk management and investment analytics, speed will also become more important. While the FPGA space is not yet ML-obvious, reading the M&A tea leaves suggests that to meet the future needs of banks and asset managers, this could be the next major battlefront in the hardware wars.