FastNetMon in search of perfect database for configuration and state storage

Hello! Iโ€™m Pavel and Iโ€™m CTO and co-founder of FastNetMon LTD, London, ๐Ÿ‡ฌ๐Ÿ‡ง. Weโ€™re cyber security software vendor and we develop DDoS ๐ŸŽฏ detection and mitigation platform for Telecoms.

Our ๐Ÿข very first product was completely free open source on premise DDoS ๐Ÿšจ detection platform called FastNetMon Community and you can find it on GitHub. It uses simple text based configuration file for configuration.

Example of FastNetMon Community configuration file

After great ๐ŸŒŸ feedback from Telecom and Cloud communities we started work on commercial edition called FastNetMon Advanced which introduced capabilities crucial for Terabit scale nationwide ๐ŸŒ networks such us scalability to millions of flows per second and lightning โšก fast DDoS detection.

To accommodate complex configurations of large networks we had to offer command line interface and API to store and alter product configuration.

In same time complexity of configuration was growing and we had such monstrosity ๐Ÿคฏ in text based configurations.

That was time when we started looking on JSON for configuration ๐Ÿ’พ storage format as it was the only way to accommodate complex configurations like this:

We agreed about data format but we had no persistent database which can store such data.

Why we actually need network ๐Ÿ’ฝ database? Why not use embedded database like SQLite or even LMDB?

Letโ€™s have a glance on FastNetMon Advanced design as it has multiple daemons.

FastNetMon design diagram

You may notice that three different independent applications use database which is one left side with large M on it:

  • fcli โ€” command line management tool (Go) which offers human friendly way to manage configuration
  • API โ€” which offers machine friendly way to manage configuration (Go)
  • FastNetMon โ€” daemon implemented in C++ just reads configuration from database but may occasionally export some state into database. Itโ€™s core of our product, it does traffic processing and detects DDoS attacks

We clearly need some way to implement read / write access to same data from multiple daemons and network database is a reasonable approach to implement it.

After doing initial research we ended up only with pretty small number of options to store ๐Ÿ“œ JSON documents:

Sadly RethinkDB was in process of bankruptcy and CouchDB was too heavy for our purpose and we ended up with using MongoDB as our configuration database.

We did not consider any relational databases like MySQL of PostgreSQL because their JSON management capabilities were on early stages and we had no schema โ›” policy in our product.

Sadly after MongoDB re-licensing under SSPL terms we started thinking about ways to replace MongoDB and itโ€™s not easy to do.

What is the special about configuration databases in compare with regular use of databases? Configuration database is hidden deep inside of product and customer should not know about itโ€™s existence until it fails.

We have 8 years of experience in operating MongoDB as configuration database with thousands of deployments worldwide ๐Ÿ—พ For all these years we faced multiple issues with MongoDB:

  • It fails immediately when disk space runs out
  • Recent versions of MongoDB started requiring CPU capabilities available only in latest CPU generations.
  • MongoDB is notoriously complicated to install with authorisation enabled as process of enabling authorisation is 3 step and involves
  • Upgrade process between multiple releases is a clear nightmare and full reinstall is the only option in many cases
  • Unusual licensing attracts too much attention as it requires involvement of lawyers for cases of cloud deployment
  • Many system administrators do not like MongoDB for variety of reasons
  • Lack of operational MongoDB experience in majority of companies
  • Modern MongoDB is not available in Debian, Ubuntu and Fedora due to licensing issues

Recently we finished very successful PoC with FerretDB which provides the way to use PostgreSQL using MongoDB protocol and it looks as very good candidate for potential replacement of MongoDB in future.

And in same time RethinkDB is slowly getting back to live again.

Iโ€™ll be very happy to see more JSON native lightweight databases preferably implemented in Go or Rust as both of languages offer great cross platform support capabilities.

Subscribe to Pavel's blog about underlying Internet technologies

Donโ€™t miss out on the latest issues. Sign up now to get access to the library of members-only issues.