As promised in the previous article, in this one you will find more information about such trading platform aspects like HFT, algorithmic trading and latency. Moreover, I will review main related technological aspects.
WHAT TECHNOLOGIES ARE USUALLY USED TO DEVELOP TRADING PLATFORMS?
Pretty much all the trading platforms I have seen were developed using .NET, Java or C++ technology stacks. Like for many other software products, quite typical solution is to divide a trading platform into backend (server or group of servers) and front-end (thin or thick client, web portal, mobile apps). Then you are not limited with single stack and you can choose different technologies for backend and front-end. For example, I was working with the platform where the backend was in Java and the front-end in .NET.
Moreover, backend part quite often consists of the following components:
- Price server (or cluster of price servers) is responsible for getting prices from feed(s) and delivering them to clients. It may change the price according the broker’s rules or apply some filters. Quite often instead of a single price server we have a cluster of servers. For example, a couple of servers in different locations might serve people from different countries to get prices with minimal latency. Another way of segregation is by the price feed, for example each node for each exchange. Price server usually need to have broad internet channel and enough memory to process the price stream.
- Trade server (or cluster of price servers) is responsible for executing trade requests. It needs to find a counter-agent locally or escalate the request further. Trade server needs to be optimized to support a lot of requests in parallel and process requests as fast as possible.
- History server is responsible for saving prices for the further analytics (most often to file system). It needs to have big amount of free space on a hard drive and some search engine (for instance, MongoDB).
- Permission server is responsible for authentication. It keeps account information and provides token used for authorization.
THE MOST IMPORTANT
Traders want stable, functional, fast, cheap software, available on Desktop, Web and Mobile. Most general requirements like correctness, availability, user-friendly interface are applicable for traders as well as for many other software users. However, while developing the trading platform, I have figured out several aspects which are important for traders.
Latency.
Latency is the amount of time between the moment when user sends a trading request and the moment when the trade server executes it. Imagine, the trader who wants to buy a contract for some price and while his/her request is being delivered, price increases. Then either trader spends more money than it was planned, or the request is cancelled. Since both options are unfavorable for trader, developers are working on minimizing this chance in trading platforms. This can be achieved by optimizing the part of trading platform, responsible for sending and processing trading requests.
Security procedures.
When banks are on the list of potential customers, platform has to implement some extra security levels. Banks (and not only banks) have a lot of strict rules about software to be used. For instance, SMS authentication for logging in, firewalls for back-end components, white-list of URLs between front-end and backend etc. To give an example of what banks may do: one of our customer hosted trade servers in his data center. The data center had a firewall between the cluster of trade servers and broker plus between each node of cluster. Servers “talked” to each other through a firewall and all of them were also covered by one more firewall.
Another customer implemented a white-list of allowed URLs, therefore each request not to the trade server was prohibited by default.
Multi-feed.
Quite often traders have their favorite feeds and brokers for trading. Therefore, you can attract more traders offering the variety of third-party liquidity providers available in your platform. It is like a shop: the more goods for sell you offer, the more clients you will get.
Public API.
A lot of traders use some patterns to decide when (and what) to trade. It is possible to program a script, which can do it automatically. Public API provides the users with the possibility to program an automated trading system (bot) to trade on behalf of a user. Many users will ask for that kind of functionality. It is almost a must-have feature for modern trading platforms. To give you some numbers: about 80% of market volume in USA is generated by algorithmic systems (when about 10 years ago it was around 50%). Probably the chart below very well illustrates the importance of this feature:
Performance.
It is very important for backend and thick clients. Trade server must process requests from traders as fast as possible to minimize the latency. At the same time, each user might send a lot of request using Public API. It means that server has to be ready to process a bunch of requests without drawdown. However, trade server is not the single component, which needs to be optimized. Thick clients usually offer different functionality to analyze the historical data looking for some patterns or trends. Thick client has to be ready to analyze big data volumes in a reasonable time.
HFT.
High frequency trading or high-speed trading is the quintessence of Latency and Public API features. This is not top tier level feature, however there are customers who are interested only in this type. HFT is a special type of algorithmic trading, when robot scans all the prices looking for an opportunity to make a trade and close it very soon with the profit. It’s important to mention, that HFT robot does not deal with domain knowledge, it deals just with numbers.
There was a broker, who was our customer. Once, one of his traders found, that sometimes in the morning you can see that buying price is lower than selling price. After around one second prices become as usual. If you are able to buy and sell within this window, you will get a profit. That trader was sitting around 1.5 hours daily before going to his office and looking for such prices. And guess what? He was usually earning from 1000 to 3000€ in this time gap.
For being able to offer the HFT feature, platform needs to provide prices to robot as fast as possible. Therefore, it is quite often when people try to host trading software closer to the exchange, preferably the same building, to minimize the network latency and be the first who can benefit from a new price. Another requirement is an ability to send prices as fast as they appear, without any caching or cumulation. Simply speaking: HFT needs minimal latency and no price caching.
Chart.
One of the most time-consuming details for developing a trading platform is trading chart. Except standard requirements for chart like zooming, scrolling and resizing, trading chart usually offers many functions to display additional objects on it, usually related with trend analysis and traded volume. Displaying all important for user information without performance drawdown is quite a challenging task. If you prefer not to buy a chart from the third party but to develop it on your own, you have to make sure that you have invested enough time into chart’s engine development.
MY CONTRIBUTION TO DEVELOPING A TRADING PLATFORM
Once we have got a customer who had the brilliant idea: they developed an exchange especially for cryptocurrency trading. Idea was super popular, and they had got many customers trading cryptocurrencies. However, there was one problem preventing them from attracting even more customers and making more money: their trade server was very simple and non-optimized, it was a “bottle neck”. They decided to buy some more professional service, but the problem was that none of any well-known trading platform was adapted for cryptocurrency trading. Moreover, they were too complicated for cryptocurrency traders, who aren’t really interested in existing variety of financial contracts. To be able to fulfil this customer’s needs we had to hide/remove a lot of functionality of our platform and at the same time extend by creating multicurrency accounts.
I do remember a lot of users wanted to test/analyze their trading strategies on a big amount of data. They wanted to upload csv files of 1-4 GB (25-100 millions of records) with historical prices. Such a big amount of data cannot be simply visualized by client because it will “blow up” computer memory. We implemented a solution, where this amount of data can be processed step by step. As soon as we need a new portion of data, it gets reloaded from file, while the old data gets swept. Basically, file system was used as swap. It helped us to allow processing very big data files, however, additionally to data visualizing we had to solve more challenges
One of them was that after implementing big amount of history data support, we immediately got very difficult and unexpected task. Previously all the tests were about 1k – 10k price records. Now users were are able to utilize millions. At this point we realized that our test engine was is capable of processing only 12-13k prices per second (pps) at max. At the same time our competitor’s engine can could process roughly 2mln pps at max. Of course, engines cannot be compared only by this criteria, because they have different functionality. However, the gap was too big. This fact had ended ends up after in months of continuous performance improvements until we have reached 8mln pps at max.
CONCLUSION
No matter which technological stack is chosen to develop a trading platform, servers are of key importance in all huge structure. Different market traders have different needs when it comes to specific requirements, but peculiarities like latency, multi-feed, public API and HFT remain among highly demanded ones. I was developing trading platform for various markets (including but not limited to Commodities, Forex and Stocks), working on blockchain and big data. So, according to my experience, there still remain common tendencies when it comes to their efficiency and functionality. Knowing these principles allows Xplicity team to provide flexible and top-quality solutions for our future customers, who need trading platform solutions.