Pages

4/08/2013

Testing how much routes a device could handle with Quagga

Hi,

I wanted to stress-test multilayer switch Juniper EX 4200 and its routing engine. Somehow. I wanted to simulate a number of BGP routes and how will switch react. Importing and filtering BGP routes to Adj-RIB-In table and then to routing table could be stressfull for device and this could lead to 100% cpu usage and for example OSPF adjacency flapping. Most of work in this blog is done by a route generator script. You could also use a spreadsheet editor (Excel) and then edit exported text to create routes in Quagga configuration file.

 What do you need?
  • router (EX4200 with a advanced license. But don't worry. BGP on EX4200 should work even without license. At least it used to work on older releases. It yelds message to console during commit  and regularry to syslog)
  • standard Linux or Ubuntu PC with NIC and Quagga
  • ethernet cable :-)

Juniper EX 4200 specifications [2]

IPv4 Unicast routes: 16,000

Install and configure Quagga on a Ubuntu PC
# apt-get install quagga

Change Ubuntu IP addresses
# ifconfig eth0 192.0.2.1/24

 Edit Quagga configuration file ...
# nano /etc/quagga/daemons

... and you should run at least these two Quagga daemons
zebra=yes
bgp=yes

How to generate many routes
I used script from this site V. Glinsky[1] blog.
Change these two lines in that script:

my $router_id="192.0.2.1"; #bgp router-id
my $remote_ip="192.0.2.2"; #BGP neighbor ip address


Leave other values unchanged. It is going to generate 300,000 routes. Much more than 16,000. Let's say we are all right with AS numbers. Ubuntu AS65099, Juniper AS65001. Actually I generated 599999 routes in my example.
 
Copy generated bgpd.conf file to a directory /etc/quagga/.

Here is how bgpd.conf should look
hostname quagga-host
password zebra
enable password zebra
line vty
router bgp 65099
  bgp router-id 192.0.2.1
  neighbor 192.0.2.2 remote-as 65001
  network 70.0.0.0/24
  network 70.0.1.0/24
  network 70.0.2.0/24
  network 70.0.3.0/24 


Run Quagga
# /etc/init.d/quagga restart

Configure Juniper BGP protocol and interface
root@ex4200> show configuration protocols bgp 
local-as 65001;
group bgp-test {
    type external;

    peer-as 65099;
    neighbor 192.0.2.1;

}


root@ex4200> show configuration interfaces ge-0/0/0   
unit 0 {
    family inet {
        address 192.0.2.2/24;
}


root@ex4200> show configuration routing-options
router-id 192.0.2.2;
autonomous-system 65001; 


There is a warning during commit. You should have a license. Don't worry, it's OK for now. It will work even without it.
{master:0}[edit]
root@ex4200# commit
[edit protocols]
  'bgp'
    warning: requires 'bgp' license
configuration check succeeds
commit complete 

Connect two devices with ethernet cable and observe results
Juniper EX 4200:

root@ex4200> show bgp summary  
Groups: 1 Peers: 1 Down peers: 0
Table          Tot Paths  Act Paths Suppressed    History Damp State    Pending
inet.0             
                   16372      16372          0          0          0          0
Peer                     AS      InPkt     OutPkt    OutQ   Flaps Last Up/Dwn State|#Active/Received/Accepted/Damped...
192.0.2.1             65099        299          4       0       0          28 16372/16372/16372/0  0/0/0/0

BGP neighbor adjacency status is Established. Highlighted is number of accepted routes in a Adj-Rib-In table. We don't use any import policy.

root@ex4200> show route

inet.0: 16384 destinations, 16384 routes (16384 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

2.0.0.0/24         *[BGP/170] 00:00:39, MED 0, localpref 100
                      AS path: 65099 I, validation-state: unverified
                    > to 192.0.2.1 via ge-0/0/0.0
2.0.1.0/24         *[BGP/170] 00:00:39, MED 0, localpref 100
                      AS path: 65099 I, validation-state: unverified
                    > to 192.0.2.1 via ge-0/0/0.0
2.0.2.0/24         *[BGP/170] 00:00:39, MED 0, localpref 100
                      AS path: 65099 I, validation-state: unverified
                    > to 192.0.2.1 via ge-0/0/0.0
2.0.3.0/24         *[BGP/170] 00:00:39, MED 0, localpref 100
                      AS path: 65099 I, validation-state: unverified
                    > to 192.0.2.1 via ge-0/0/0.0
2.0.4.0/24         *[BGP/170] 00:00:39, MED 0, localpref 100
                      AS path: 65099 I, validation-state: unverified
                    > to 192.0.2.1 via ge-0/0/0.0
2.0.5.0/24         *[BGP/170] 00:00:39, MED 0, localpref 100
                      AS path: 65099 I, validation-state: unverified
                    > to 192.0.2.1 via ge-0/0/0.0
2.0.6.0/24         *[BGP/170] 00:00:39, MED 0, localpref 100
---(more)---


Highlighted number is number of active routes in routing table.
 

root@ex4200> show chassis routing-engine
Routing Engine status:
  Slot 0:
    Current state                  Master
    Temperature                 34 degrees C / 93 degrees F
    CPU temperature             34 degrees C / 93 degrees F
    DRAM                      1024 MB
    Memory utilization          44 percent
    CPU utilization:
      User                      23 percent
      Background                 0 percent
      Kernel                    34 percent
      Interrupt                  0 percent
      Idle                      42 percent
    Model                          EX4200-24F
    Serial ID                      BR0210217636
    Start time                     2013-01-19 19:28:27 UTC
    Uptime                         31 minutes, 9 seconds
    Last reboot reason             0x2:watchdog
    Load averages:                 1 minute   5 minute  15 minute
                                       0.45       0.15       0.10

CPU utilization is a bit higher right after establishing adjacency and receiving routes.

You could also use command
> show system processes extended
to see how rpd process is using CPU [3].

Connect to Quagga and show bgp config
root@bt:~# telnet localhost 2605
Trying ::1...
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.

Hello, this is Quagga (version 0.99.15).
Copyright 1996-2005 Kunihiro Ishiguro, et al.


User Access Verification
 

Password:  (password is zebra or quagga)
quagga-host>
quagga-host> show ip bgp summary
BGP router identifier 192.0.2.1, local AS number 65099
RIB entries 599999, using 37 MiB of memory
Peers 1, using 2520 bytes of memory

Neighbor        V    AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
192.0.2.2       4 65001       5     304        0    0    0 00:02:00        0

Total number of neighbors 1
 

quagga-host> show ip bgp neighbors
BGP neighbor is 192.0.2.2, remote AS 65001, local AS 65099, external link
  BGP version 4, remote router ID 192.0.2.2
  BGP state = Established, up for 00:02:05
  Last read 00:00:19, hold time is 90, keepalive interval is 30 seconds
  Neighbor capabilities:
    4 Byte AS: advertised and received
    Route refresh: advertised and received(old & new)
    Address family IPv4 Unicast: advertised and received
    Graceful Restart Capabilty: received
      Remote Restart timer is 120 seconds
      Address families by peer:
        none
  Graceful restart informations:
    End-of-RIB send: IPv4 Unicast
    End-of-RIB received:
  Message statistics:
    Inq depth is 0
    Outq depth is 0
                         Sent       Rcvd
    Opens:                  1          0
    Notifications:          0          0
    Updates:              298          0
    Keepalives:             6          5
    Route Refresh:          0          0
    Capability:             0          0
    Total:                305          5
  Minimum time between advertisement runs is 30 seconds

 For address family: IPv4 Unicast
  Community attribute sent to this neighbor(both)
  0 accepted prefixes

  Connections established 1; dropped 0
  Last reset never
Local host: 192.0.2.1, Local port: 179
Foreign host: 192.0.2.2, Foreign port: 60656
Nexthop: 192.0.2.1
Nexthop global: ::
Nexthop local: ::
BGP connection: non shared network
Read thread: on  Write thread: off 


Session is established. 599999 routes loaded from config file.
 
Other ways to BGP routes generation
Nice guide on how to generate routes from real world bgp routes dump:
http://evilrouters.net/2009/08/21/getting-bgp-routes-into-dynamips-with-video/ 

How to mitigate this 'attack'?
There is a command to change behavior after certain number of routes are received.
http://www.juniper.net/techpubs/en_US/junos12.3/topics/reference/configuration-statement/accepted-prefix-limit-edit-protocols-bgp.html

References:
[1] http://blog.glinskiy.com/2009/10/how-to-generate-lots-of-bgp-routes.html
[2] http://www.juniper.net/us/en/products-services/switching/ex-series/ex4200/#specifications
[3] http://www.juniper.net/techpubs/en_US/junos9.6/information-products/topic-collections/nog-baseline/routing-engine--memory-introduction.html