BGP Flow Spec: Ultimate Weapon Against DDoS
Hello! I’m Pavel and I’m CTO and co-founder of FastNetMon LTD, London, 🇬🇧. We’re cyber security software vendor and we develop DDoS 🎯 detection and mitigation platform for Telecoms.
In this article we’re going to discuss how BGP Flow Spec protocol can allow you to mitigate volumetric DDoS attacks without using third party cloud based DDoS scrubbing centres.
As part of my role at FastNetMon I have worked closely with BGP Flow Spec since 2015 (ExaBGP, GoBGP) from both operational and implementation side.
What is a volumetric DDoS attack?
It's an attack type which generates a very significant amount of bandwidth or packet rate towards the victim. Bandwidth for such attacks may start from hundreds of megabits to many hundreds of gigabits. Usually such attacks are implemented using an approach called amplification.
What’s inside of DDoS packets?
There are hundreds of known volumetric attack types and basically all fields in IPv6 and IPv4 (legacy) headers can be used in attack:
In reality most likely we will have following traffic types involved in DDoS attacks:
- DNS amplification: UDP, source port 53, may be fragmented, large packets
- NTP amplification, UDP, source port 123, may be fragmented, large packets
- TCP SYN flood: TCP, SYN flag set, random source ports
- SSDP amplification: UDP, source port 1900, large packets, may be fragmented
As you may see these most common attacks do not use all varieties of L3 / L4 header fields and allow us to simplify the filtering process.
What do we need to filter malicious traffic?
To filter out packets involved in attack we need software firewall or hardware ASIC which is capable of filtering most common DDoS attack vectors. Almost all routers (software and hardware) and switches have ACL or firewall capability which allows them to filter traffic according to content of L3 / L4 fields.
In addition to equipment which is capable of filtering malicious traffic we need a DDoS sensor which can identify malicious traffic patterns and automatically generate filtering rules for particular devices.
What is the issue then?
The main issue is that each vendor available on market uses different approaches for controlling and programming their ACL and firewalls:
- NETCONF
- RESTCONF
- gNMI
- gRPC
- Vendor specific API
- Command line interface via ssh
It's extremely challenging to support and then maintain all these firewall management protocols from DDoS sensor side.
That’s how BGP Flow Spec helps us by being lingua franca which works for a very large number of vendors without minor differences.
BGP Flow Spec protocol filtering capabilities
To know how useful BGP Flow Spec for traffic filtering we need to learn BGP Flow Spec filtering alphabet and it’s following:
- Source prefix (IPv4 or IPv6)
- Destination prefix (IPv4 or IPv6)
- IP Protocol number
- List or range of source ports for TCP and UDP
- List or range of destination ports for TCP and UDP
- ICMP code
- TCP flags
- Packet length
- Fragmentation flags (do not fragment, is fragment, first or last fragment)
- DSCP
It's a pretty comprehensive list of filtering fields and it allows us to filter out many types of volumetric attack. Lack of TTL field in this list is pretty unfortunate as it may have allowed us to filter out spoofed traffic more efficiently.
BGP Flow Spec protocol traffic actions
In scope of DDoS attack mitigation the only actions we’re interested is discard and rate-limit but for more complicated scenarios we may need full arsenal for traffic engineering options provided by BGP Flow Spec:
- Drop
- Rate limit
- Accept
- Mark (DSCP)
- Redirect to VRF
- Redirect to nexthop
Which vendors support BGP Flow Spec protocol?
Majority of largest Telecom vendors have solid support for BGP Flow Spec protocol on their BGP daemons and can receive and announce BGP Flow Spec announces. To know which devices can apply rules to live traffic please proceed to the next section.
- Juniper routers
- Cisco
- Nokia
- Huawei
- Arista
- Extreme
- 6Wind
- Netscout Arbor
- Bird (only controller)
- GoBGP (only controller)
- ExaBGP (only controller)
- FRR (only client, can do filtering only via Linux nft)
Controller means that it can receive and announce BGP Flow Spec rules but cannot do filtering. Client means that device can receive BGP Flow Spec announces and do filtering according to received rules.
Which routers support BGP Flow Spec based filtering?
That's the most important part of the article as it clearly defines which models of network equipment can allow you to use their routers as BGP Flow Spec managed DDoS scrubbers:
- Juniper MX, PTX
- Cisco ASR 1000, ASR 9000, NCS 5500
- Nokia SR
- Huawei
- 6WIND VSR
- Arista
- Extreme
- FRR
- Netscout Arbor
Is it that easy? Sadly no and I have a whole presentation (slides) about caveats and intricacies of BGP Flow Spec implementations on each vendor.
Vendor specific BGP Flow Spec limitations
From perspective of most serious operational limitations in BGP Flow Spec implementation I would like to mention few vendors below.
All these limitations require a pretty serious amount of adjustments on DDoS sensors which makes interoperability with other vendors more complicated.
Arista
- For fragment flags, only the Is a fragment (IsF) bit is supported only for IPv4 packets. Combining source and destination ports and the Fragment flags in the same rule is not supported
- For TCP flags, the ECE, CWR, and NS flags are not supported.
Source: link
Extreme
- Only the IsF bit is supported for BGP flowspec NLRI sub-component type 12 (Fragment). DF, FF, and LF bit functionality is not supported.
- Two-byte TCP flags are not supported.
- When a rate-limiting action is set under a BGP flowspec rule, the operational rate value may differ from the rate value specified in the flowspec rule because operational values are selected in multiples of 22 kbits per second.
- IPv4 BGP flowspec rules are applied only to IPv4 data traffic. They are not applied to IPv6 data traffic.
- The following TCP flags are not supported: Explicit Congestion Notification Echo (ECE) and Congestion Window Reduced (CWR)
Source: link
6Wind VSR
We had reports from customers with previous version of VSR software on 5th August 2024 that BGP Flow Spec rules with is-fragment fragmentation flag do not apply to all fragmented traffic.
Cisco ASR 9000
- A maximum of five multi-value range can be specified in a flowspec rule.
- You cannot configure the IPv6 first-fragment match and last-fragment match simultaneously on the Cisco ASR 9000 series routers as they are mutually exclusive.
Source: link
Juniper
Issue with extremely convoluted dropped traffic monitoring on Juniper MX platform.
What about cross ASN BGP Flow Spec?
You definitely can use BGP Flow Spec between ASN boundaries and multiple Telecom companies allow it for their customers (usually for additional fee).
This approach is extremely powerful because it allows you to rely on very significant capacity available from your carrier to filter our volumetric DDoS attack without doing extremely expensive bandwidth expansion.
Companies which offer BGP Flow Spec support:
- Core Backbone
- Sparkle
- RETN
- Rascom
- SMARTNET (20 rules allowed for free)
- Inter.Link
- Hurricane Electric (via account manager)
Cross ASN BGP Flow spec is quite challenging as Carriers need to do extensive rule validation before accepting it to their network. Unfortunately, almost none of the Telecom equipment vendors provide complete BGP Flow Spec rule validation frameworks to implement it properly and companies use third party products (mostly based on GoBGP or ExaBGP) to implement validation and then pass Flow Spec rules to their networks.
Due to lack of proper standardised communication mechanisms about dropped traffic in this case we will have very limited visibility about efficiency of each rule (i.e. how much traffic was discarded by each rule) and it may lead to an increased number of rules then actually needed.
Yet another issue in this case is a hard limit about number of BGP Flow Spec rules and when customer reaches it Carrier has no other options but to break BGP session. You have to be very careful about number of rules and remove old rules as soon as they stop being useful.
Conclusions
BGP Flow Spec is a mature protocol supported by majority of Telecom vendors and it allows you to mitigate most dangerous volumetric DDoS attacks without relying on third party DDoS Scrubbing centres and propriety on premise scrubbing black boxes.
If you would like to add your vendor or company to list of BGP Flow Spec capable ones please reach me via email or LinkedIN.