Since the beginning of the year I have been working on a FAST based project. Bothered to implement and maintain a connectivity API it seems that markets are migrating to more standard protocols. Now, nobody can blame the API anymore. You have your clear specification and it’s up to you to implement it as fast as possible.

At a first glance the FAST protocol seems simple but the simplicity fades away quickly when talking about operators and dictionaries. It seems that after so many versions of FIX they have finally came up with a way to optimize the transfer.

So, you have this binary protocol to implement as good as you can as there are not many solutions:

  • openFast is a free (mozilla license) java based implementation which is the most complete implementation I have tested but suffers from the disadvantage of java. It is a very good starting point for prototyping.
  • the fastlib code is good at almost nothing because it does not even know basic types such as doubles
  • there are some commercial alternatives but none convinced me

Implementing the protocol seems the best choice but you soon realize that it’s only part of the deal. In the current case the market decided to mix FAST with UDP multicast. This results in one of the more complex synchronization algorithm at base protocol level I have ever seen for some obvious reasons:

  • FAST is designed for streaming thus the use of dictionaries and values operators but if you rely on UDP you are bound to loose some packets. Solution: reset everything in each packet thus loosing compression.
  • since multicast is one-way what is the solution for packet loss? Stream duplication :)) So you have to decode twice as much packets.

Mix this with the standard image + updates data propagation mechanism and you arrive to at least (1 image + 1 update)x2 streams. Multiply by instrument and you get a number of streams in the order of 100. Decode them and synchronize by per instrument sequence numbers.
As someone said at some point in the development it seemed that at each point when they noticed something was wrong they added a new set of sequence numbers.
Enjoy the new trends in market connectivity. Too bad that FAST is not so bad after all.

Ok, so now the project is complete, time to switch to FLEX, can’t wait to see the difference. Some things still remain the same however, still using Eclipse on Linux, at least that :)