FastNetMon Community on the way from PoC to MVP

I’m Pavel, I’m CTO and co-founder of FastNetMon LTD, London, 🇬🇧

FastNetMon Community is an open source DDoS detection tool licensed under GPLv2 terms.

In this article I would like to tell you about early stages of FastNetMon Community development.

I hope it will be a very valuable story for new open source projects and startups because it will explain our transition from PoC which barely worked to complete MVP which was ready to present to industry and potential customers.

Before starting FastNetMon I did extensive background research about similar tools and was well aware that open source tools for exactly this task did not exist and it was a nice opportunity to try all the power of open source by sharing it with the community.

Project itself was started around 2013 but very first code I uploaded to GitHub at 18th of October, 2013.

This version had no configuration file support, did not use Makefile for build and supported only ULOG2 and pcap as a traffic capture method. It was able to detect DDoS attacks using only a hard coded packet per second limit which was set to 10000.

FastNetMon was able to handle around 2 Gigabits of traffic from the port mirror interface and packet rate was around 250 000 packets per second. Hardly we can call it a proper product but it was perfectly fit PoC quality software.

From our own DDoS detection needs we had all things in place and further development of FastNetMon wasn’t required. It may have become yet-another DIY DDoS detection tool which was fit to use only for specific types of DDoS attacks and only in specific pretty exotic network configuration of a particular cloud compute company. Fortunately, we had another plan.

I was well aware how open source works and that it was not enough to put code to GitHub and then wait for magic to happen. The foundation of any open source project is a community and our challenge was to find one.

At that stage of my career I had decent software engineering skills and had excellent knowledge of Linux internals but my expertise in computer networks was close to non-existent but it was crucial for project’s success as we had to make it suitable for all kinds of networks.

Fortunately, I found a great community which was on the border between software engineering and computer networks.

On 21st of November, 2013 I made an announcement about the current status of FastNetMon and its capabilities in this community.

We got very positive feedback and it encouraged us to continue development.

Many members of community were very technical in their feedback and offered multiple very useful recommendations for us:

• C++ 11 was too new and was not available in many Linux distributions due to old compiler versions
• We have to use Makefile instead of bash script as it’s good industry practice for building projects
• I had to do comments in English
• Got recommendation to consider PF_RING instead of ULOG2 and pcap as it offers better performance
• Got recommendation to consider using Netmap for traffic capture in FreeBSD

On 28th of November, 2013 we got new Intel 82599EB 10-Gigabit card and it allowed us to do tests on higher packet rates.

On 12th of March, 2014 we added PF_RING support. Back in time it was the only reasonable option to process high volumes of traffic.

This change allowed us to handle way more traffic than previously. We were able to handle almost the whole 10G interface at a packet rate of around 5.5M packets per second. Additionally we had an option to read traffic from multiple interfaces.

On 9th of June, 2014 we added support for configuration file and significantly improved documentation.

Shortly after that at 21st of June, 2014 we did very first release of FastNetMon 1.0.0 It wasn't significant release by any means but it clearly was step forward.

On 30th of June, 2014 we added support for the Patricia lookup tree as previously we used linear search in all specific networks which significantly limited tool scalability for networks with more then 10 networks. Patricia tree allowed us to scale to any number of networks.

On 2nd of October, 2014 a customer reported that FastNetMon crashes when empty line happens to be in the networks list.

On 16th of October, 2014 we added support for blocking traffic using network card hardware filters. It’s not clear why we did it as nobody ever tried it and that capability was removed in future.

On 21st of October, 2014 we added an option to unban blocked hosts as our network engineer was tired with doing so manually.

On 16th of November, 2014 we added option for flow tracking as it proven to be helpful to detect attacks faster then they degrade network on our machines.

On 20th of November, 2014 we did very first announcement on NANOG maillist.

I would say that this announcement was clearly a pivotal moment in the project's history. Previously our main commnity was mostly in Eastern Europe but NANOG was a US based mail list and it provided us insight in a way how people operate networks in other places and it was exceptionally useful.

We got a well deserved grilling about absent support of Netflow and IPFIX. So it was good feedback and we started work.

On 27th of November, 2014 we added support for exponential moving average algorithm   which is foundation of FastNetMon even now

On 28th of November, 2014 we migrated build system to cmake as Makefile was too complicated to maintain with multiple external dependencies.

On 1st of December, 2014 we added sFlow support. We discovered this protocol on our Extreme X670 and tried to use it as alternative as port mirror was very complicated to maintain due to high CPU usage.

On 11th of December, 2014 we added ZC / DNA support for PF_RING to allow close to line rate monitoring of 10G interfaces.

On 22nd of December, 2014 we introduced fastnetmon_client as separate tool as FastNetMon’s main logic was very complicated and needed separation for different parts.

On 26th of January, 2015 we added support for Netflow v5 as it’s simplest protocol from Netflow family.

On 28th of January, 2015 we added support for Netflow v9.

On 5th of March, 2015 we got feature request to add support for IPFIX protocol which is very similar to Netflow v9.

On 10th of March, 2015 we added support for Netmap which allowed us to capture traffic on rates close to line rate without using any proprietary libraries.

On 16th of March, 2015 we added support for IPFIX.

On 16th of March, 2015 we did second stable release of FastNetMon 1.1.1 which can be clearly considered as proper MVP 🚀 It had lots of improvements and covered almost all possible network deployment scenarios.

After that in version 1.1.2 which was released on 2nd of June 2015 we added few more missing parts such as BGP integration and Graphite / Grafana support.

To highlight importance of community in projects's life I would like to mention that very first public presentation of FastNetMon was made by Job Snijders on 18th of September 2015 in Amsterdam at NLNOG 2015.

Job's presentation was followed by my own presentation at RIPE 71 in Buharest, Romania. It happened at 16th of November, 2015.

Another exceptionally important milestone which presented us to the audience of World's most important market was great presentation of my friend Vicente De Luca at NANOG 66, San Diego, US which happened on 10th of February, 2016. This presentation concluded our rough first appearance as early PoC back in 2014 with our show as properly finished product.

Now, in August of 2023, 8 years later FastNetMon Community features exactly the same core feature set which did not change for such long period of time. I think it’s very clear confirmation that in version 1.1.2 we managed to accumulate and implement all required features.

Majority of features were implemented because somebody from community reached us and explained importance of them to us. We just listened and followed their recommendations.

From code perspective every single plugin mentioned here was completely rewrote to modern day demands. We did it to make code more secure, to reduce maintenance complexity, to make it faster. We're very sure that we will do it not once again in future as technological progress brings more challenges to Telecom industry.