FactSet Close to Debuting Cloud Ticker Plant

FactSet will begin migrating its physical ticker plant to AWS’s Amazon Elastic Compute Cloud later this year. The vendor says the move will enable it to offer faster onboarding times for new data sources and clients, while reducing latency and delivering greater capacity and flexibility.

The vendor began the project earlier this year, and expects to complete the rollout next year. It will initially run in parallel with its current ticker plant, which is hosted in multiple datacenters, including two major datacenters in the US, with local feed handlers deployed around the world to capture data locally from exchanges. Once the migration is complete, FactSet will decommission the old ticker plant.

“Cloud gives us features and functions that are cost-prohibitive in any other way—such as deploying infrastructure globally—and clients get lower latency and great quality regardless of market volumes. With ticker plants, you have to size for the highest market volumes, so you need a lot of capacity. And when you onboard new clients, it takes time to get infrastructure in place,” says Gene Fernandez, chief product and technology officer at FactSet. “This will allow us to onboard new exchanges very quickly and onboard new clients in a fraction of the time—and we’re going to gain a ton of operational efficiency.”

Though cloud environments are not known for delivering ultra-low latency, the gains come from a virtual cloud environment being equally accessible from anywhere in the world. In contrast, for example, if a client in Asia wants data from a local exchange, it has to be sent first to the US to be processed and normalized in real time by FactSet, which also adds reference data, before sending it back around the world to the client via leased lines or dedicated extranets.

“For regional clients requiring exchange data, we’re eliminating a hop for them,” Fernandez says.

FactSet chose to work with AWS because of its Nitro system—purpose-built tools for tuning general-purpose clouds to ensure they deliver the minimum service levels guaranteed by clients like FactSet. To do that using software, it would require 50% more servers in a general-purpose cloud, Fernandez says. FactSet can also anticipate demand and “front-run” higher compute requirements by adding capacity. “If we see the velocity of the market increasing, we’ll increase our front-run capacity and speed up provisioning,” he says.

However, none of the major cloud offerings are purpose-built for specialized functions such as data vendors’ ticker plants.

“Our engineers started out completely skeptical that it could even be done. But then they started to see statistics that showed it might be possible. And then they saw statistics that we might be able to materially improve things. So they went from being skeptics to being advocates,” Fernandez says.

‘We Tried to Break It’

The vendor spent the first half of this year planning and testing the cloud ticker. Much of that time was spent stress-testing it and deliberately trying to find ways to make it fail. This was so that FactSet could understand its limits and learn how it would react when faced with certain situations, from unexpected bursts of market volume to systems outages.

“We’ve become known for ‘gorilla testing’—like if a gorilla got loose in the datacenter and started ripping out cables and servers—so we can see how the system responds,” Fernandez says.

First, the vendor starts by trying to imagine what scenarios might occur. Then it assigns engineers to a “red team” and a “white team.” The red team spends its time doing all it can to take the system down, while the white team observes their efforts and chronicles the impact on the system and its environment without intervening or doing anything to defend against the red team’s efforts.

“Historically, and this year especially, we’ve seen some incredible volumes. We were able to simulate … close to 10 times the highest volumes we’ve ever seen. We could see the impact of that load, and we could see the environment being loaded differently from the average, but there was no impact on clients. You see a drop in terms of latency, but we were still achieving our SLAs with plenty of headroom,” Fernandez says. “Because of the flexibility of AWS Nitro, there is almost no gate—it’s almost infinitely scalable.”