Virtual Switching System

Posted on July 6, 2011 by Shafagh Zandi

 

VSS Overview

The Cisco VSS simplifies network configuration and operation by providing a loop-free Layer 2 topology using two Catalyst 6500 switches acting as one big Virtual switch. VSS reduces number of Layer 3 routing neighbors by providing a Layer 2 connectivity for access/distribution switches.

VSS Simple Drawing

A VSS is a pair of combined 6500 switches acting as a single network element with redundancy and load balancing over port-channels (etherchannels). One switch becomes the master or active chassis and the other one becomes the VSS standby.

 

VSS Switch Roles

  • VSS Active: The active chassis controls the VSS operation. It runs the control plane, L2 and L3 control protocols. It also runs the management plane functions like console interface, logs, file system and even power management.
  • VSS Standby: Listens to master, checks the status, forwards the ingress traffic but sends all control traffic to the VSS active chassis for processing.

Virtual Switch Link

To share control and data traffic between two chassis a VSL – Virtual Switch Link is required. VSL is implemented as a Port Channel. The control traffic gets higher priority over data on a VSL and never gets discarded.

 

before configuration make sure that you have right IOS image, proper VS-capable supervisor and line cards, you can check your line cards with the following command:

6500A#switch convert check vss-capable
This is a VSS capable switch.
VSL ports can be configured in slot: 4, 5, 6

 

VSS Configuration

By default, 6500 series switches are configured to operate in standalone mode. The following steps are required in order to setup virtual stack across two 6500 chassis.

Step1: Make sure SSO and NSF are configured and enabled

Step2: Assign Virtual Switch Doman and Switch numbers

Step3: Configure VSL Port Channel, PO# should be unique on each chassis.

Step4: Convert Standalone Chassis to Virtual Switch mode

VSS1

Step1:

6500A(config)#redundancy
6500A(config-red)#mode sso
!

Step2:

6500A(config)#switch virtual domain ?
  <1-255>  Virtual switch domain number

6500A(config)#switch virtual domain 100
Domain ID 100 config will take effect only
after the exec command ‘switch convert mode virtual’ is issued

6500A(config-vs-domain)#switch 1
6500A(config-vs-domain)#exit

…

6500B(config)#switch virtual domain 100
Domain ID 100 config will take effect only
after the exec command ‘switch convert mode virtual’ is issued

6500B(config-vs-domain)#switch 2
6500B(config-vs-domain)#exit

Step3:

6500A(config)#int port-channel 100
6500A(config-if)#switch virtual link 1
6500A(config-if)#no shutdown
6500A(config-if)#exit
6500A(config)#int te5/4
6500A(config-if)#channel-group 100 mode on
6500A(config-if)#no shutdown
6500A(config)#int te6/4
6500A(config-if)#channel-group 100 mode on
6500A(config-if)#no shutdown

…

6500B(config)#int port-channel 200
6500B(config-if)#switch virtual link 2
6500B(config-if)#no shutdown
6500B(config-if)#exit
6500B(config)#int te5/4
6500B(config-if)#channel-group 200 mode on
6500B(config-if)#no shutdown
6500B(config)#int te6/4
6500B(config-if)#channel-group 200 mode on
6500B(config-if)#no shutdown

Step4:

6500A#switch convert ?
  check  check if this switch and its modules are VSS capable or not
  mode   mode keyword virtual or standalone

6500A#switch convert mode ?
  stand-alone  stand-alone switch
  virtual      virtual switch

6500A#switch convert mode virtual

This command will convert all interface names
to naming convention “interface-type switch-number/slot/port”,
save the running config to startup-config and
reload the switch.

NOTE: Make sure to configure one or more dual-active detection methods
once the conversion is complete and the switches have come up in VSS mode.

Do you want to proceed? [yes/no]: yes
Converting interface names
Building configuration…
Saving converted configuration to bootflash: …
Destination filename [startup-config.converted_vs-20110705-214318]?

*** — SHUTDOWN NOW —

 

Ensure that same PFC operating mode is being used on both chassis in order to have a proper SSO redundancy mode:

6500A#show platform hardware pfc mode
PFC operating mode : PFC3C

Now, Let’s look at configuration of VSS switch after reboot:

6500A#sh run
…
!
hostname 6500A
!        
switch virtual domain 100
switch mode virtual
!        
mls netflow interface
mls cef error action reset
!
spanning-tree mode pvst
spanning-tree extend system-id
diagnostic bootup level minimal
!        
redundancy
main-cpu
  auto-sync running-config
mode sso
!
vlan internal allocation policy ascending
vlan access-log ratelimit 2000
!
!
interface Port-channel100
no switchport
no ip address
switch virtual link 1
mls qos trust cos
no mls qos channel-consistency
!
interface Port-channel200
no switchport
no ip address
switch virtual link 2
mls qos trust cos
no mls qos channel-consistency
!        
interface GigabitEthernet1/1/1
no switchport
no ip address
shutdown
!        
interface GigabitEthernet1/1/2
no switchport
no ip address
shutdown
…
…
interface TenGigabitEthernet1/5/4
no switchport
no ip address
mls qos trust cos
channel-group 100 mode on
…
…
interface TenGigabitEthernet1/6/4
no switchport
no ip address
mls qos trust cos
channel-group 100 mode on
…
…
interface TenGigabitEthernet2/5/4
no switchport
no ip address
mls qos trust cos
channel-group 200 mode on
…
…
interface TenGigabitEthernet2/6/4
no switchport
no ip address
mls qos trust cos
channel-group 200 mode on
…
…
interface Vlan1
no ip address
shutdown
!
ip classless
ip forward-protocol nd
!
control-plane
!
line con 0
line vty 0 4
login
!
mac-address-table aging-time 480
no event manager policy Mandatory.go_switchbus.tcl type system    
!
module provision switch 1
slot 1 slot-type 147 port-type 61 number 48  virtual-slot 17
slot 3 slot-type 152 port-type 31 number 48  virtual-slot 19
slot 4 slot-type 227 port-type 60 number 8  virtual-slot 20
slot 5 slot-type 254 port-type 31 number 2 port-type 61 number 1 port-type 60 number 2  virtual-slot 21
slot 6 slot-type 254 port-type 31 number 2 port-type 61 number 1 port-type 60 number 2  virtual-slot 22
!
module provision switch 2
slot 1 slot-type 147 port-type 61 number 48  virtual-slot 33
slot 3 slot-type 152 port-type 31 number 48  virtual-slot 35
slot 4 slot-type 227 port-type 60 number 8  virtual-slot 36
slot 5 slot-type 254 port-type 31 number 2 port-type 61 number 1 port-type 60 number 2  virtual-slot 37
slot 6 slot-type 254 port-type 31 number 2 port-type 61 number 1 port-type 60 number 2  virtual-slot 38
!
end

Verification:

Verification is the most imortant part of configuration, you need to make sure that commands that you have entered are working as expected!

6500A#show switch virtual
Switch mode                  : Virtual Switch
Virtual switch domain number : 100
Local switch number          : 1
Local switch operational role: Virtual Switch Active
Peer switch number           : 2
Peer switch operational role : Virtual Switch Standby

6500A#show switch virtual role

Switch  Switch Status  Preempt    Priority  Role     Session ID
        Number         Oper(Conf) Oper(Conf)         Local  Remote
——————————————————————
LOCAL    1     UP      FALSE(N )   100(100)  ACTIVE   0      0  
REMOTE   2     UP      FALSE(N )   100(100)  STANDBY  4004   1462
In dual-active recovery mode: No

6500A#show switch virtual link
VSL Status : UP
VSL Uptime : 43 minutes
VSL SCP Ping : Pass
VSL ICC Ping : Pass
VSL Control Link : Te1/5/4 

6500A#show switch virtual link port-channel
Flags:  D – down        P – bundled in port-channel
        I – stand-alone s – suspended
        H – Hot-standby (LACP only)
        R – Layer3      S – Layer2
        U – in use      N – not in use, no aggregation
        f – failed to allocate aggregator

        M – not in use, no aggregation due to minimum links not met
        m – not in use, port not aggregated due to minimum links not met
        u – unsuitable for bundling
        d – default port

        w – waiting to be aggregated
 
Group  Port-channel  Protocol    Ports
——+————-+———–+——————-
100    Po100(RU)        -        Te1/5/4(P)     Te1/6/4(P)    
200    Po200(RU)        -        Te2/5/4(P)     Te2/6/4(P)

If I connect my console to secondary chassis:

6500A-sdby>
Standby console disabled

 

Multi-Chassis EtherChannel

One VSS supports a maximum of 512 – 2 port channels (deducting 2 POs for VSL) Configuration of MEC is not different than any regular etherchannel! The Different is physical connectivity, one link connects to VSS Active and the other link connects to VSS Standby chassis creating a high bandwidth active/active aggregated link-bundle (Etherchannel)

VSS2

Access switch:

interface GigabitEthernet1/1/1
channel-group 10 mode on
!
interface GigabitEthernet1/1/2
channel-group 10 mode on

Core switch:

interface Port-channel10
switchport
!
interface GigabitEthernet1/3/17
switchport
channel-group 10 mode on
!
interface GigabitEthernet2/3/17
switchport
channel-group 10 mode on

Verify:

6500A#sh etherchannel summary
Number of channel-groups in use: 3
Number of aggregators:           3

Group  Port-channel  Protocol    Ports
——+————-+———–+————————————–
10     Po10(SU)         -        Gi1/3/17(P)    Gi2/3/17(P)   
100    Po100(RU)        -        Te1/5/4(P)     Te1/6/4(P)    
200    Po200(RU)        -        Te2/5/4(P)     Te2/6/4(P)    

 

Switch12#sh etherchannel summary
Number of channel-groups in use: 1
Number of aggregators:           1

Group  Port-channel  Protocol    Ports
——+————-+———–+————————————–
10     Po10(SU)         -        Gi1/1/1(P)  Gi1/1/2(P)

Posted in Datacenter, Ethernet Switching | Tagged , , , , | 5 Comments

Service Provider General Tasks

Posted on April 24, 2011 by Shafagh Zandi

As you may know, CCIE Service Provider has been updated to new version (V.3) as of April 18 2011. There are several changes in the exam which are very important, ATM seems to be removed and IOS XR has a major role along with more IPv6 plus extended troubleshooting section. I went through my preparation notes and found best practices note which is general rule of thumb for CCIE SP lab.

CiscoSP

General Best Practices for exam:

  • Configure ISIS or OSPF backbone in each AS (Loopback reach-ability)

configure loopback0 as router-id
configure metric parameters (e.g. auto-cost reference-bandwidth / metric-style wide)
configure network types / is-types
configure timers or protocol specific features, DR priority and so on.

  • Configure BGP in each AS (to advertise loopbacks to other AS)

configure “no bgp default ipv4-unicast”
configure neighbors remote-as
configure update-source as loopback0
configure address-family ipv4 and activate neighbors (and RR if necessary)
configure next-hop-self on border-routers (if necessary)
advertise loopback address with the network command

  • Configure eBGP neighbors

make sure you have advertised loopback addresses
make sure next-hop is propagated correctly
set “ip bgp-community new-format”
send bgp communities to required neighbors
configure routing policies/controls based on community lists

  • Configure MPLS Tagging

configure label protocol
configure loopback as ldp session source – router-id
configure label options (numbers, session protection, etc)
configure mpls interfaces

  • Configure MPLS TE

configure routers for MPLS TE
configure OSPF/ISIS for MPLS TE
configure routing area for MPLS TE (or Level1/2)
configure interfaces for TE and RSVP
configure tunnel interfaces
configure static or dynamic paths

  • Configure MPLS VPN

configure VRFs and try to import RTs
configure MP-IBGP
configure MP-eBGP
configure send-label in BGP (if required)
configure next-hop-unchanged (between MP-eBGP points if necessary)

  • Configure Route control in MPLS VPN

set BGP attributes to control routes. e.g:
send MED to MP-BGP CE neighbor (to make it less preferred)
set Weight for a neighbor in different AS to be preferred

 

For a detailed Lab exam blue print you can refer to:
https://learningnetwork.cisco.com/docs/DOC-10145

Posted in CCIE General-Info, SP | Leave a comment

CSM Basics and FT

Posted on April 12, 2011 by Shafagh Zandi

Cisco Content Switching Module adds layer 4 to layer 7 content switching capabilities to the Catalyst 6500/7600 Series providing high-performance load balancing for servers, firewalls or even NAC boxes! Cisco CSM is old and not a new product, it has been replaced by newer Cisco ACE load-balancers. Cisco ACE is being sold in two different fashions, as Blade modules just like CSM or as 4710 appliances just like CSS appliance-based load-balancers.

ACE supports virtualization (multiple-contexts and resource allocation) while CSM is monolithic. CSM uses IOS configuration file and stores its configuration in the running-config. It’s really easy and straight forward to configure if you know what you’re doing!

ImagejMObcW8Adv_full

It’s been awhile since the last time that I touched CSM as it’s end of sales and support. EOL/EOS reference link! But you might have some customers that are still using this product and they might need some levels of technical support. Last week, a call came through asking for CSM support, the customer has four CSM blades inside 6509s in redundant fashion in main and DR site…

This post quickly reviews the following topics:

  • CSM Basic setup
  • Fault tolerant
  • Multiple set of Client/Server VLAN pairs.
  • Dual / Multiple Gateways
  • Direct Server Access
  • CSM Verification with show commands
  • How to ping from CSM VIP address

Basic CSM Setup

The most basic and common method of setting up a load-balancer is to have a VLAN pair; one facing servers and the other VLAN facing clients (facing firewall/routers). Then the Load-balancer sits between client and server and creates a VIP (virtual ip address). That VIP is related to several real servers but all are seen as one entity or one VIP. That VIP is reflected by a NAT statement further on a firewall to a public address and is ready for being resolved by DNS.

Client request comes through client VLAN of load-balancer and hits the VIP address then it gets distributed among available real servers.

The first step is to create CLIENT and SERVER VLAN pair:

1. Create two VLANs on 6500 catalyst but do not assign an ip address.

2. Create same VLANs on the CSM and assign IP address inside the CSM:

module ContentSwitchingModule 3
vlan 302 server
  ip address 10.8.108.2 255.255.255.0
!
vlan 301 client
  ip address 10.8.8.2 255.255.255.0
  gateway 10.8.8.254

Gateway is required for CSM to talk to clients (it might be your DMZ firewall interface)

The next step is to create server-farm (real-servers sit there) and vserver (virtual server – VIP)

probe HTTP http
recover 3
request method head
expect status 200
interval 2
failed 6
port 80
!
serverfarm HTTP_2010
nat server
no nat client
real 10.8.108.17
inservice
real 10.8.108.18
inservice
probe HTTP
!
vserver HTTP_2010
virtual 10.8.8.201 tcp www
serverfarm HTTP_2010
replicate csrp connection
persistent rebalance
inservice

The first part of above configuration is probe config. It checks the real servers health and sees whether HTTP code 200 is returned by each HTTP server. If probe to a server fails, that server will not be used.

Serverfarm section configures two real servers (IP address is from SERVER VLAN) and uses HTTP probe for server health check. Vserver section, defines the VIP and correlates the server farm to vserver.

Easy! now let’s move to verification section:

dc1sw01#sh module csm 3 vlan

vlan   IP address       IP mask          type
—————————————————
301    10.8.8.2         255.255.255.0    CLIENT
302    10.8.108.2       255.255.255.0    SERVER
401    10.8.12.2        255.255.255.0    CLIENT
402    10.8.112.2       255.255.252.0    SERVER
997    0.0.0.0          0.0.0.0          FT

 

CSM Fault-tolerant configuration

CSM FT configuration is pretty straight-forward. FT is configured when you have two CSM blades located in two 6500 catalysts to work as a fail-over group. You can also create a fault-tolerant configuration with two CSMs in a single Catalyst 6500 series chassis.

First step is to create a VLAN on both catalysts specifically for FT and then configure priority on each CSM (master must have higher priority):

 vlan 997
name CSM_Failover
!
module ContentSwitchingModule 3
ft group 1 vlan 997
priority 110
preempt

Then you need to configure your VLAN-pair to include alternate CSM IP address tobe seen as single entity by servers to be used as the default gateway (the alias command):

vlan 302 server
ip address 10.8.108.2 255.255.255.0 alt 10.8.108.3 255.255.255.0
alias 10.8.108.1 255.255.255.0
!
vlan 301 client
ip address 10.8.8.2 255.255.255.0 alt 10.8.8.3 255.255.255.0
gateway 10.8.8.254

Secondary Set of VLAN-pair

What if we need to have more than one CLIENT/SERVER VLAN-pair?

Usually the client side is connected to a firewall (Cisco FWSM, ASA or something else). You can have different client/server VLAN pairs while firewall serves each client VLAN as a different DMZ zone.

You can also use this approach during server migration between DMZs!  or even during DMZ migration…

CSM

!
vlan 402 server
ip address 10.108.12.2 255.255.255.0 alt 10.108.12.3 255.255.255.0
alias 10.108.12.1 255.255.255.0
!
vlan 401 client
ip address 10.8.12.2 255.255.255.0 alt 10.8.12.2 255.255.255.0
gateway 10.8.12.254

real TEST_1
address 10.108.12.17
inservice
real TEST_2
address 10.108.12.18
inservice
!
serverfarm TEST_SERVERS
nat server
no nat client
real name TEST_1
inservice
real name TEST_2
inservice
probe HTTP
!
vserver TEST_HTTP
virtual 10.8.12.11 tcp www
serverfarm TEST_SERVERS
replicate csrp connection
persistent rebalance
inservice

Perhaps the secondary CLIENT VLAN requires its own default gateway. You need to consider that CSM always uses the lowest CLIENT VLAN ID to reach its configured gateway, so in case of having multiple gateways for different CLIENT VLANs you need to configure the following trick:

Dual Gateway

The first CSM VLAN pair has no problem with gateway reachability. But configuring a gateway for a secondary VLAN pair is not as easy as first one. Based on the above drawing to force CSM to use 10.8.12.254 as gateway for VLAN pair 401/402 you need to configure:

serverfarm VLAN402-out
no nat server
no nat client
real 10.8.12.254
inservice

vserver VLAN402-out
virtual 0.0.0.0 0.0.0.0 any
vlan 402
serverfarm VLAN402-out
inservice

Direct Server Access

If you need to access your servers from different VLANs without using the VIP address, this section is useful for you. You might find it impossible to connect to real servers. The following debug output demonstrate that something is wrong!

Failed LB : access-denied

For direct server access configure a separate serverfarm/vserver pair:

serverfarm ROUTE
no nat server
no nat client
predictor forward
!
vserver REAL_ACCESS
virtual 10.8.112.0 255.255.255.0 any
serverfarm ROUTE
persistent rebalance
inservice

Verification

dc1sw01#show module csm 3 reals

real        server farm    weight  state       conns/hits
———————————————————
TEST_1      TEST_SERVERS   8       OPERATIONAL 0
TEST_2      TEST_SERVERS   8       OPERATIONAL 0
10.8.12.1   VLAN402-OUT    8       OPERATIONAL 1  

dc1sw01#show module csm 3 vservers

vserver        type  prot virtual            vlan state        conns
——————————————————————–
DIRECT_ACCESS  SLB   any  10.8.108.0/24:0    ALL  OPERATIONAL  19
TEST_HTTP      SLB   TCP  10.8.12.11/32:80   ALL  OPERATIONAL  0
VLAN402-OUT    SLB   any  0.0.0.0/0:0        402  OPERATIONAL  3
REAL_ACCESS    SLB   any  10.8.112.0/24:0    ALL  OPERATIONAL  1

dc1sw01#show module csm 3 serverfarm

server farm     type  predictor   nat   reals  redirect bind id
—————————————————————
DIRECT_ACCESS   SLB   Forward     S     0      0        0
TEST_SERVERS    SLB   RoundRobin  S     2      0        0
VLAN402-OUT     SLB   RoundRobin  none  1      0        0
ROUTE           SLB   Forward     none  0      0        0   

dc1sw01#show module csm 3 vlan   

vlan   IP address       IP mask          type
—————————————————
208    10.8.8.2         255.255.255.0    CLIENT
318    10.8.108.2       255.255.255.0    SERVER
401    10.8.12.2        255.255.255.0    CLIENT
402    10.8.112.2       255.255.252.0    SERVER
997    0.0.0.0          0.0.0.0          FT

dc1sw01#show module csm 3 arp

Internet Address  Physical Interface  VLAN      Type       Status
——————————————————————–
10.8.108.1       00-01-64-F9-1A-01   318       -ALIAS-    local
10.8.108.2       00-0F-34-2C-52-06   318       –SLB–    local
10.8.108.3       00-02-FC-E1-CE-60   318       LEARNED    up(0 misses)
10.8.112.1       00-01-64-F9-1A-01   402       -ALIAS-    local
10.8.112.2       00-0F-34-2C-52-06   402       –SLB–    local
10.8.112.3       00-02-FC-E1-CE-60   402       LEARNED    up(0 misses)
10.8.112.17      00-50-56-AD-00-96   402       REAL       up(0 misses)
10.8.112.18      00-50-56-AD-00-96   402       REAL       up(0 misses)
10.8.8.254       00-C0-EA-EA-EA-EA   208       GATEWAY    up(0 misses)
10.8.8.2         00-0F-34-2C-52-06   208       –SLB–    local
10.8.8.3         00-02-FC-E1-CE-60   208       LEARNED    up(0 misses)
10.8.12.254      00-22-90-9D-D3-0F   401       GATEWAY    up(0 misses)
10.8.12.2        00-0F-34-2C-52-06   401       –SLB–    local
10.8.12.3        00-02-FC-E1-CE-60   401       LEARNED    up(0 misses)
10.8.12.11       00-01-64-F9-1A-01   0         VSERVER    local

CSM PING

If you want to PING from a VIP address:

dc1sw01#ping module csm 3 10.8.12.254
IP address       Reachable
————————–
10.8.12.254      Yes

dc1coresw01#ping module csm 3 gateways

IP address       Reachable
————————–
10.8.8.254       Yes
10.8.12.254      Yes

dc1sw01#ping module csm 3 reals

IP address       Reachable
————————–
10.8.108.10      Yes
10.8.108.11      Yes
10.8.112.17      Yes
10.8.112.18      Yes
10.8.12.254      Yes

Hope that helps!

Posted in Uncategorized | Leave a comment

MPLS Quality of Service – Part Two

Posted on September 13, 2010 by Shafagh Zandi

In the pervious post, we went through the Uniform model and its configuration. In this post we are going to see the differences between the Uniform model and the Pipe model. As mentioned earlier in the Part one of MPLS Quality of Service, the Pipe model does not change customer marking (IP TOS – DSCP). So in the following example, we will see that the provider is able to change the EXP marking but will not change the packet marking in the end. Provider also performs QoS tasks (even at the egress LSR) based on EXP and MPLS QoS marking, not based on IP header properties (Short-pipe model is different in this case).

As depicted in the above picture, egress PE is configured to send an Explicit Null to the P router. In this case the egress PE will be able to see transport label EXP field as well as VPN label EXP field.

 

R1(config)#mpls ldp explicit-null

R0#show mpls forwarding-table
Local  Outgoing    Prefix            Bytes tag  Outgoing   Next Hop   
tag    tag or VC   or Tunnel Id      switched   interface             
16     Pop tag     3.3.3.3/32        0          Et0/0      192.168.30.3
17     0           1.1.1.1/32        137        Et0/1      192.168.10.1

 

In this case, on R0 (the P router) we don’t need to copy EXP from topmost label (received from R3) to the topmost transport label sent to R1, because the Explicit Null (label 0) maintains the EXP bit. This is only one example for practicing purpose and learning the MPLS QoS feature. In this example, we like to perform our marking on the topmost label and not rely on the second label, to demonstrate explicit-null feature. The Uniform and the Pipe models difference is not in the ingress part but in the egress: to save or not to save the customer marking on the IP packet.

 

Configuration

Ingress PE:
!
access-list 110 permit icmp host 5.5.5.5 host 6.6.6.6
!
class-map match-any in1
  match access-group 110
class-map match-any out1
  match qos-group 7
!
policy-map in1
  class in1
   set mpls experimental imposition 3
   set qos-group 7
policy-map out1
  class out1
   set mpls experimental topmost 7
!

R3#sh policy-map interface e0/0
Ethernet0/0

  Service-policy output: out1

    Class-map: out1 (match-any)
      10 packets, 1220 bytes
      5 minute offered rate 0 bps, drop rate 0 bps
      Match: qos-group 7
        10 packets, 1220 bytes
        5 minute rate 0 bps
      QoS Set
        mpls experimental topmost 7
          Packets marked 10

    Class-map: class-default (match-any)
      28 packets, 11442 bytes
      5 minute offered rate 3000 bps, drop rate 0 bps
      Match: any

P:
no configuration!

 

Egress PE:
!
class-map match-all in1
  match mpls experimental topmost 7
class-map match-all out1
  match qos-group 7
!
!
policy-map in1
  class in1
   set qos-group 7
policy-map out1
  class out1
   priority 256
  class class-default
   shape average 500000
!

R1#sh policy-map interface e0/1
Ethernet0/1

  Service-policy output: out1

    Class-map: out1 (match-all)
      10 packets, 1140 bytes
      5 minute offered rate 0 bps, drop rate 0 bps
      Match: qos-group 7
      Queueing
        Strict Priority
        Output Queue: Conversation 264
        Bandwidth 256 (kbps) Burst 6400 (Bytes)
        (pkts matched/bytes matched) 0/0
        (total drops/bytes drops) 0/0

    Class-map: class-default (match-any)
      20 packets, 2080 bytes
      5 minute offered rate 0 bps, drop rate 0 bps
      Match: any
      Traffic Shaping

 

Why there is no configuration on P LSR? Because IOS by default maintains your EXP and as long as PE LSR requesting Explicit Null, the topmost label is kept and contains EXP value of incoming labeled packet. (there’s no label disposition on the P LSR – R0)

Posted in MPLS, Quality of Service, SP | Leave a comment

MPLS Quality of Service

Posted on September 12, 2010 by Shafagh Zandi

The Quality of Service (QoS) has become more popular than past few years, because of high-bandwidth and delay sensitive applications. QoS is prioritizing more important data over less important traffic. There are two ways to implement QoS in an IP network: IntServ and DiffServ. Internet is operating based on best-effort model.

As the DiffServ model does not require a signaling protocol such as RSVP, it’s more popular or better to say the only applicable model in today’s networks. (Although we use RSVP in MPLS TE for path signaling and reservation).

IETF in RFC 3270 has recommended three QoS models for DiffServ tunneled traffic in MPLS networks:

  • Pipe model
  • Short Pipe model
  • Uniform model

The Pipe and Short Pipe models are almost the same: They do not change IP ToS of customers data at all. They might change the EXP field in the path (EXP of MPLS Label) but the IP header QoS field remains unchanged. The Pipe model performs forwarding/discarding/scheduling based on EXP at the egress LSR while the Short Pipe model does that based on IP ToS. Because in the Short pipe model there might be no label at all. (remember PHP operation – Penultimate-Hop Popping)

In the Uniform model, there’s no guarantee that ToS of customers packet remains intact, but the EXP and IP ToS fields of a data packet will always show the same thing. It means if the provider changes the EXP header, that will be copied later to the IP ToS field at the egress point.

 

IOS Default Behavior:

In short IOS does not change your QoS in the path if you don’t change it. 
        Imposition – Copy TOS bits to EXP (TOS Reflection) when adding labels by ingress LSR.
        Swap – Copy EXP from old label to new label.
        Disposition – Do NOT Copy EXP from label to IP.

 

Configuration

 

Label to Label (on PHP) sets EXP 5 (if top is 5) after disposing the top label:

     class-map match-all in1
      match mpls experimental topmost 5
     class-map match-all out1
      match qos-group 5
    !
     policy-map in1
      class in1
       set qos-group mpls experimental topmost
     policy-map out1
      class out1
       set mpls experimental topmost 5
    !
    interface Ethernet0/0
     service-policy input in1
    !
    interface Ethernet0/1
     service-policy output out1
    !

Label to IP (on PE) sets precedence 5 (if top is 5) after popping the label:

         class-map match-all in1
          match mpls experimental topmost 5
         class-map match-all out1
          match qos-group 5
        !
         policy-map in1
          class in1
           set qos-group mpls experimental topmost
         policy-map out1
          class out1
           set ip precedence 5
        !
        interface Ethernet0/0
         service-policy input in1
        !
        interface Ethernet0/1
         service-policy output out1

 

Scenario

In this sceniaro based on the service provider’s policy, the EXP value is changed in the path and is copied to inner label by P router. At the egress LSR, the EXP value is copied to the DSCP. (Uniform model)

Ingress PE:
!
policy-map out1
  class class-default
   set mpls experimental topmost 7
policy-map in1
  class class-default
   set mpls experimental imposition 3
!

P:

!
class-map match-any in1
  match mpls experimental topmost 0  1  2  3  4  5  6  7
class-map match-any out1
  match qos-group 0
  match qos-group 1
  match qos-group 2
  match qos-group 3
  match qos-group 4
  match qos-group 5
  match qos-group 6
  match qos-group 7
!
policy-map in1
  class in1
   set qos-group mpls experimental topmost
policy-map out1
  class out1
   set mpls experimental topmost qos-group
!

Egress PE:

!
class-map match-any in1
  match mpls experimental topmost 0  1  2  3  4  5  6  7
class-map match-any out1
  match qos-group 0
  match qos-group 1
  match qos-group 2
  match qos-group 3
  match qos-group 4
  match qos-group 5
  match qos-group 6
  match qos-group 7
!
policy-map in1
  class in1
   set qos-group mpls experimental topmost
policy-map out1
  class out1
   set precedence qos-group
!

Posted in MPLS, Quality of Service, SP | Leave a comment

BGP Multipath – Part Three

Posted on September 7, 2010 by Shafagh Zandi

After publishing part 1 and part 2 of my BGP Multipath Trilogy! One of our good readers sent a very intelligent question here:

cc said:

Very nice article Shafagh,
Any difference in any of these examples if R0 were both a P and a Route Reflector?
http://www.shafagh.net/2010/09/bgp-multipath-part-two.html?showComment=1283780646233#c5196111639519265964

 

There’s a known limitation:

Route Reflector Limitation
When multiple iBGP paths installed in a routing table, a route reflector will advertise only one paths (next hop). If a router is behind a route reflector, all routers that are connected to multihomed sites will not be advertised unless a different route distinguisher is configured for each VRF.
http://www.cisco.com/en/US/docs/ios/12_2t/12_2t11/feature/guide/ft11bmpl.html#wp1027175

But like everything else in the computer world, there’s always a workaround, I found a way to make each path a “unique path” (playing with 64 bit prefix – RD) to send prefixes to ingress PE and let PE import different RDs inside the routing table. Here we go:

So in this example, R0 and R1 are using different RDs for the same CE (multi-homed CE – utlizing one or two routers at customer’s site) This makes two different vpnv4 prefixes pointing to the same ipv4 subnet.

The workaround is to advertise both or several parallel paths from different PEs with different RDs so that RRs advertise all paths. The RR will advertise all parallel paths with different RDs (as they are not same) and ingress PE can run BGP multipath and use all parallel paths.

The route reflector will not filter anything. It sends the best path to the R3. As long as these two prefixes are different they are both best path for their corresponding RDs:

 

R2#sh ip bgp vpn all
BGP table version is 4, local router ID is 2.2.2.2

   Network          Next Hop Metric LocPrf Weight Path
Route Distinguisher: 1:1
*>i69.69.69.69/32   1.1.1.1         0    100      0 ?
Route Distinguisher: 1:2
*>i69.69.69.69/32   10.10.10.10     0    100      0 ?

 

R3 needs this command:

R3(config-router)#address-family ipv4 vrf C1
R3(config-router-af)#maximum-paths ibgp 2 import 1
R3(config-router-af)#^Z
R3#sh ip route vrf C1

Routing Table: C1
Gateway of last resort is not set

     69.0.0.0/32 is subnetted, 2 subnets
B       69.69.69.69 [200/0] via 10.10.10.10, 00:01:30
                    [200/0] via 1.1.1.1, 00:01:22

Posted in IP Routing, MPLS, SP | 1 Comment

BGP Multipath – Part Two

Posted on September 5, 2010 by Shafagh Zandi

In the previous post, we reviewed basics of iBGP and eBGP multipath. In this post we get introduced with import feature of MP-BGP multipath. When there’s a difference in VRF’s RD, we have to use import keyword to import parallel paths from one VRF into another.

For example, we have the same diagram as BGP Multipath Part One, the only difference is the RD number that is used by R1. The Customer VRF RD at R2,3 and 4 is 1:1, while on the other side RD is 1:2.

BGP-Multipath-import

When the RD is different, we need additional “import” keyword to tell the router how many parallel routes import from one VRF into another. If RD number is same on both side (both PE devices) previous post and example is enough for you.

 

Let’s check the configuration without any “maximum-path” command, to see the BGP natural behavior when there are parallel paths. (We know that BGP only selects one)

 

R1(config-router-af)#do sh ip bgp vpnv4 all
BGP table version is 281, local router ID is 1.1.1.1
Status codes: * valid, > best, i – internal,
Origin codes: i – IGP, e – EGP, ? – incomplete

   Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 1:1
* i5.5.5.5/32       4.4.4.4                 10    100      0 1 ?
* i                 3.3.3.3                 10    100      0 1 ?
*>i                 2.2.2.2                 10    100      0 1 ?
* i55.55.55.0/24    4.4.4.4                 10    100      0 1 ?
* i                 3.3.3.3                 10    100      0 1 ?
*>i                 2.2.2.2                 10    100      0 1 ?
* i192.168.25.0     4.4.4.4                 10    100      0 1 ?
* i                 3.3.3.3                 10    100      0 1 ?
*>i                 2.2.2.2                 10    100      0 1 ?
* i192.168.35.0     4.4.4.4                 10    100      0 1 ?
* i                 3.3.3.3                 10    100      0 1 ?
*>i                 2.2.2.2                 10    100      0 1 ?
* i192.168.45.0     4.4.4.4                 10    100      0 1 ?
* i                 3.3.3.3                 10    100      0 1 ?
*>i                 2.2.2.2                 10    100      0 1 ?
Route Distinguisher: 1:2 (default for vrf C1)
   Network          Next Hop            Metric LocPrf Weight Path
* i5.5.5.5/32       3.3.3.3                 10    100      0 1 ?
*> 6.6.6.6/32       192.168.16.6             0             0 2 ?
*>i55.55.55.0/24    2.2.2.2                 10    100      0 1 ?
r> 192.168.16.0     192.168.16.6             0             0 2 ?
*>i192.168.25.0     2.2.2.2                 10    100      0 1 ?
*>i192.168.35.0     2.2.2.2                 10    100      0 1 ?
*>i192.168.45.0     2.2.2.2                 10    100      0 1 ?


R1(config-router-af)#do sh ip route vrf C1

Routing Table: C1

B    192.168.45.0/24 [200/10] via 2.2.2.2, 00:00:17
B    192.168.25.0/24 [200/10] via 2.2.2.2, 00:00:17
     55.0.0.0/24 is subnetted, 1 subnets
B       55.55.55.0 [200/10] via 2.2.2.2, 00:00:17
     5.0.0.0/32 is subnetted, 1 subnets
B       5.5.5.5 [200/10] via 2.2.2.2, 00:00:17
     6.0.0.0/32 is subnetted, 1 subnets
B       6.6.6.6 [20/0] via 192.168.16.6, 07:53:26
B    192.168.35.0/24 [200/10] via 2.2.2.2, 00:00:17
C    192.168.16.0/24 is directly connected, Ethernet0/1

 

As you can see, there’s only one path (per prefix) at customer VRF. So let’s add “maximum-path ibgp” command for Customer VRF:

 

R1(config-router)#address-family ipv4 vrf C1
R1(config-router-af)#maximum-paths ibgp 3
R1(config-router-af)#
R1(config-router-af)#do sh ip bgp vpnv4 all
BGP table version is 288, local router ID is 1.1.1.1

   Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 1:1
* i5.5.5.5/32       4.4.4.4                 10    100      0 1 ?
* i                 3.3.3.3                 10    100      0 1 ?
*>i                 2.2.2.2                 10    100      0 1 ?
* i55.55.55.0/24    4.4.4.4                 10    100      0 1 ?
* i                 3.3.3.3                 10    100      0 1 ?
*>i                 2.2.2.2                 10    100      0 1 ?
* i192.168.25.0     4.4.4.4                 10    100      0 1 ?
* i                 3.3.3.3                 10    100      0 1 ?
*>i                 2.2.2.2                 10    100      0 1 ?
* i192.168.35.0     4.4.4.4                 10    100      0 1 ?
* i                 3.3.3.3                 10    100      0 1 ?
*>i                 2.2.2.2                 10    100      0 1 ?
* i192.168.45.0     4.4.4.4                 10    100      0 1 ?
* i                 3.3.3.3                 10    100      0 1 ?
*>i                 2.2.2.2                 10    100      0 1 ?
Route Distinguisher: 1:2 (default for vrf C1)
   Network          Next Hop            Metric LocPrf Weight Path
*>i5.5.5.5/32       2.2.2.2                 10    100      0 1 ?
*> 6.6.6.6/32       192.168.16.6             0             0 2 ?
*>i55.55.55.0/24    2.2.2.2                 10    100      0 1 ?
r> 192.168.16.0     192.168.16.6             0             0 2 ?
*>i192.168.25.0     2.2.2.2                 10    100      0 1 ?
*>i192.168.35.0     2.2.2.2                 10    100      0 1 ?
*>i192.168.45.0     2.2.2.2                 10    100      0 1 ?

 

So the “maximum-path ibgp” only indicates how many routes from RIB (VRF’s BGP table) can be imported to the routing table. The problem is that the RIB – RD table for 1:2 (VRF C1) has only path, so maximum-path ibgp has no parallel path to import into the routing table.

 

The import keyword allows you to configure the VRF table to accept multiple redundant paths in addition to the best path. By default, a VRF will import only one path (best path) per prefix from the source VRF table. If the best path goes down, the destination will not be reachable until the next import event occurs, and then a new best path will be imported into the VRF table. The import event runs every 15 seconds by default.

 

Note: Configuring redundant paths with the import keyword can increase CPU and memory utilization significantly, especially in a network where there are many prefixes to learn and a large number of configured VRFs. It is recommended that this feature is only configured as necessary and that the minimum number of redundant paths are configured (Typically, not more than two).

 

maximum-paths ibgp 3 import 3

 

R1(config-router-af)#maximum-paths ibgp  3 import 3
R1(config-router-af)#do sh ip bgp vpnv4 rd 1:2


BGP table version is 269, local router ID is 1.1.1.1
   Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 1:2 (default for vrf C1)
* i5.5.5.5/32       3.3.3.3                 10    100      0 1 ?
* i                 4.4.4.4                 10    100      0 1 ?
*>i                 2.2.2.2                 10    100      0 1 ?
*> 6.6.6.6/32       192.168.16.6             0             0 2 ?
* i55.55.55.0/24    3.3.3.3                 10    100      0 1 ?
* i                 4.4.4.4                 10    100      0 1 ?
*>i                 2.2.2.2                 10    100      0 1 ?
r> 192.168.16.0     192.168.16.6             0             0 2 ?
* i192.168.25.0     3.3.3.3                 10    100      0 1 ?
* i                 4.4.4.4                 10    100      0 1 ?
*>i                 2.2.2.2                 10    100      0 1 ?
* i192.168.35.0     3.3.3.3                 10    100      0 1 ?
* i                 4.4.4.4                 10    100      0 1 ?
*>i                 2.2.2.2                 10    100      0 1 ?
* i192.168.45.0     3.3.3.3                 10    100      0 1 ?
* i                 4.4.4.4                 10    100      0 1 ?
*>i                 2.2.2.2                 10    100      0 1 ?

R1(config-router-af)#do sh ip route vrf C1

Routing Table: C1

B    192.168.45.0/24 [200/10] via 4.4.4.4, 00:04:05
                     [200/10] via 3.3.3.3, 00:00:35
                     [200/10] via 2.2.2.2, 00:04:05
B    192.168.25.0/24 [200/10] via 4.4.4.4, 00:04:05
                     [200/10] via 3.3.3.3, 00:00:35
                     [200/10] via 2.2.2.2, 00:04:05
     55.0.0.0/24 is subnetted, 1 subnets
B       55.55.55.0 [200/10] via 4.4.4.4, 00:04:05
                   [200/10] via 3.3.3.3, 00:00:35
                   [200/10] via 2.2.2.2, 00:04:05
     5.0.0.0/32 is subnetted, 1 subnets
B       5.5.5.5 [200/10] via 4.4.4.4, 00:04:06
                [200/10] via 3.3.3.3, 00:00:36
                [200/10] via 2.2.2.2, 00:04:06
     6.0.0.0/32 is subnetted, 1 subnets
B       6.6.6.6 [20/0] via 192.168.16.6, 07:49:57
B    192.168.35.0/24 [200/10] via 4.4.4.4, 00:04:06
                     [200/10] via 3.3.3.3, 00:00:36
                     [200/10] via 2.2.2.2, 00:04:06
C    192.168.16.0/24 is directly connected, Ethernet0/1

 

Now, let’s play with the variables and see the different results:

 

maximum-paths ibgp 2 import 3

R1(config-router-af)#maximum-paths ibgp 2 import 3
R1(config-router-af)#do sh ip bgp vpnv4 rd 1:1

BGP table version is 222, local router ID is 1.1.1.1

   Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 1:1
* i5.5.5.5/32       3.3.3.3                 10    100      0 1 ?
* i                 4.4.4.4                 10    100      0 1 ?
*>i                 2.2.2.2                 10    100      0 1 ?
* i55.55.55.0/24    3.3.3.3                 10    100      0 1 ?
* i                 4.4.4.4                 10    100      0 1 ?
*>i                 2.2.2.2                 10    100      0 1 ?
* i192.168.25.0     3.3.3.3                 10    100      0 1 ?
* i                 4.4.4.4                 10    100      0 1 ?
*>i                 2.2.2.2                 10    100      0 1 ?
* i192.168.35.0     3.3.3.3                 10    100      0 1 ?
* i                 4.4.4.4                 10    100      0 1 ?
*>i                 2.2.2.2                 10    100      0 1 ?
* i192.168.45.0     3.3.3.3                 10    100      0 1 ?
* i                 4.4.4.4                 10    100      0 1 ?
*>i                 2.2.2.2                 10    100      0 1 ?

R1(config-router-af)#do sh ip bgp vpnv4 rd 1:2
BGP table version is 222, local router ID is 1.1.1.1

   Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 1:2 (default for vrf C1)
* i5.5.5.5/32       4.4.4.4                 10    100      0 1 ?
* i                 3.3.3.3                 10    100      0 1 ?
*>i                 2.2.2.2                 10    100      0 1 ?
*> 6.6.6.6/32       192.168.16.6             0             0 2 ?
* i55.55.55.0/24    4.4.4.4                 10    100      0 1 ?
* i                 3.3.3.3                 10    100      0 1 ?
*>i                 2.2.2.2                 10    100      0 1 ?
r> 192.168.16.0     192.168.16.6             0             0 2 ?
* i192.168.25.0     4.4.4.4                 10    100      0 1 ?
* i                 3.3.3.3                 10    100      0 1 ?
*>i                 2.2.2.2                 10    100      0 1 ?
* i192.168.35.0     4.4.4.4                 10    100      0 1 ?
* i                 3.3.3.3                 10    100      0 1 ?
*>i                 2.2.2.2                 10    100      0 1 ?
* i192.168.45.0     4.4.4.4                 10    100      0 1 ?
* i                 3.3.3.3                 10    100      0 1 ?
*>i                 2.2.2.2                 10    100      0 1 ?

R1(config-router-af)#do sh ip route vrf C1

Routing Table: C1

B    192.168.45.0/24 [200/10] via 4.4.4.4, 00:01:09
                     [200/10] via 2.2.2.2, 00:01:09
B    192.168.25.0/24 [200/10] via 4.4.4.4, 00:01:10
                     [200/10] via 2.2.2.2, 00:01:10
     55.0.0.0/24 is subnetted, 1 subnets
B       55.55.55.0 [200/10] via 4.4.4.4, 00:01:10
                   [200/10] via 2.2.2.2, 00:01:10
     5.0.0.0/32 is subnetted, 1 subnets
B       5.5.5.5 [200/10] via 4.4.4.4, 00:01:10
                [200/10] via 2.2.2.2, 00:01:10
     6.0.0.0/32 is subnetted, 1 subnets
B       6.6.6.6 [20/0] via 192.168.16.6, 07:41:14
B    192.168.35.0/24 [200/10] via 4.4.4.4, 00:01:27
                     [200/10] via 2.2.2.2, 00:01:27
C    192.168.16.0/24 is directly connected, Ethernet0/1

 

maximum-paths ibgp 3 import 2

 

R1(config-router-af)#maximum-paths ibgp 3 import 2
R1(config-router-af)#do sh ip bgp vpnv4 rd 1:1
BGP table version is 264, local router ID is 1.1.1.1

   Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 1:1
* i5.5.5.5/32       4.4.4.4                 10    100      0 1 ?
* i                 3.3.3.3                 10    100      0 1 ?
*>i                 2.2.2.2                 10    100      0 1 ?
* i55.55.55.0/24    4.4.4.4                 10    100      0 1 ?
* i                 3.3.3.3                 10    100      0 1 ?
*>i                 2.2.2.2                 10    100      0 1 ?
* i192.168.25.0     4.4.4.4                 10    100      0 1 ?
* i                 3.3.3.3                 10    100      0 1 ?
*>i                 2.2.2.2                 10    100      0 1 ?
* i192.168.35.0     4.4.4.4                 10    100      0 1 ?
* i                 3.3.3.3                 10    100      0 1 ?
*>i                 2.2.2.2                 10    100      0 1 ?
* i192.168.45.0     4.4.4.4                 10    100      0 1 ?
* i                 3.3.3.3                 10    100      0 1 ?
*>i                 2.2.2.2                 10    100      0 1 ?

R1(config-router-af)#do sh ip bgp vpnv4 rd 1:2

   Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 1:2 (default for vrf C1)
* i5.5.5.5/32       4.4.4.4                 10    100      0 1 ?
*>i                 2.2.2.2                 10    100      0 1 ?
*> 6.6.6.6/32       192.168.16.6             0             0 2 ?
* i55.55.55.0/24    4.4.4.4                 10    100      0 1 ?
*>i                 2.2.2.2                 10    100      0 1 ?
r> 192.168.16.0     192.168.16.6             0             0 2 ?
* i192.168.25.0     4.4.4.4                 10    100      0 1 ?
*>i                 2.2.2.2                 10    100      0 1 ?
* i192.168.35.0     4.4.4.4                 10    100      0 1 ?
*>i                 2.2.2.2                 10    100      0 1 ?
* i192.168.45.0     4.4.4.4                 10    100      0 1 ?
*>i                 2.2.2.2                 10    100      0 1 ?


R1(config-router-af)#do sh ip route vrf C1

Routing Table: C1

B    192.168.45.0/24 [200/10] via 4.4.4.4, 00:00:22
                     [200/10] via 2.2.2.2, 00:00:22
B    192.168.25.0/24 [200/10] via 4.4.4.4, 00:00:22
                     [200/10] via 2.2.2.2, 00:00:22
     55.0.0.0/24 is subnetted, 1 subnets
B       55.55.55.0 [200/10] via 4.4.4.4, 00:00:22
                   [200/10] via 2.2.2.2, 00:00:22
     5.0.0.0/32 is subnetted, 1 subnets
B       5.5.5.5 [200/10] via 4.4.4.4, 00:00:22
                [200/10] via 2.2.2.2, 00:00:22
     6.0.0.0/32 is subnetted, 1 subnets
B       6.6.6.6 [20/0] via 192.168.16.6, 07:46:21
B    192.168.35.0/24 [200/10] via 4.4.4.4, 00:00:31
                     [200/10] via 2.2.2.2, 00:00:31
C    192.168.16.0/24 is directly connected, Ethernet0/1

 

Final R1 Configuration

 

!
router bgp 100
no synchronization
bgp log-neighbor-changes
neighbor 2.2.2.2 remote-as 100
neighbor 2.2.2.2 update-source Loopback0
neighbor 3.3.3.3 remote-as 100
neighbor 3.3.3.3 update-source Loopback0
neighbor 4.4.4.4 remote-as 100
neighbor 4.4.4.4 update-source Loopback0
no auto-summary
!
address-family vpnv4
neighbor 2.2.2.2 activate
neighbor 2.2.2.2 send-community extended
neighbor 3.3.3.3 activate
neighbor 3.3.3.3 send-community extended
neighbor 4.4.4.4 activate
neighbor 4.4.4.4 send-community extended
exit-address-family
!
address-family ipv4 vrf C1
neighbor 192.168.16.6 remote-as 2
neighbor 192.168.16.6 activate
 maximum-paths ibgp  3 import 3
no auto-summary
no synchronization
exit-address-family
!

 

Posted in IP Routing, MPLS, SP | Leave a comment

BGP Multipath

Posted on September 5, 2010 by Shafagh Zandi

AS you may know, BGP selects only one best path for each prefix it receives then installs in the IP routing table. So whenever we need load-balancing across different paths, we have to enable BGP multipath, by the “maximum-paths” command.

We can select iBGP and eBGP paths altogether as the best, that is called eiBGP multipath. It means, for example, CE1 is multihomed to PE1 and PE2 via BGP. PE1 has a best path to CE1, PE1 can still use PE2 to CE1 as a parallel pathto reach CE1, so one path is eBGP (to CE1) and another path is iBGP (to PE1 to CE1) this is called eiBGP multipath.

There’s a criteria and several conditions that BGP checks before selecting additional paths in parallel with the best one. The following attributes of parallel paths have to match with the best path:

  • Weight
  • Local Pref
  • Origin
  • AS-Path Length
  • MED
  • Neighbor AS or Sub-AS match for (eBGP multipath)
  • AS-PATH match (for eiBGP multipath)
  • IGP metric to BGP next hop

So let’s bring an example:

R5 & R6 = CE

R2, R3, R4 and R1 = PE

There’s iBGP between R2,R3,R4 and R1 (MP-iBGP)

There’s eBGP between R5 and PE R2,R3 & R4 and between R1 and R6

 

R5 Initial configuration

 

router bgp 1
redistribute connected metric 10
neighbor 192.168.25.2 remote-as 100
neighbor 192.168.35.3 remote-as 100
neighbor 192.168.45.4 remote-as 100
no auto-summary

R5# sh ip bgp
BGP table version is 8, local router ID is 55.55.55.55
Status codes: * valid, > best, i – internal
Origin codes: i – IGP, e – EGP, ? – incomplete

   Network       Next Hop     Metric LocPrf Weight Path
*> 5.5.5.5/32    0.0.0.0          10        32768 ?
*> 6.6.6.6/32    192.168.25.2                   0 100 2 ?
*                192.168.45.4                   0 100 2 ?
*                192.168.35.3                   0 100 2 ?

R5# sh ip route
Codes: C – connected, S – static, B – BGP
Gateway of last resort is not set

C    192.168.45.0/24 is directly connected, Ethernet0/2
C    192.168.25.0/24 is directly connected, Ethernet0/0
C    192.168.35.0/24 is directly connected, Ethernet0/1
     5.0.0.0/32 is subnetted, 1 subnets
C       5.5.5.5 is directly connected, Loopback0
     6.0.0.0/32 is subnetted, 1 subnets
B       6.6.6.6 [20/0] via 192.168.25.2, 00:00:13

 

As you can see, R5 has chosen R2, for 6.6.6.6 signed by “>” character.

 

eBGP Multipath

 

To activate the parallel paths, we need to enable eBGP multipath for R5:

R5(config)#router bgp 1
R5(config-router)#maximum-paths 4


R5# sh ip route
Codes: C – connected, S – static, B – BGP
Gateway of last resort is not set

C    192.168.45.0/24 is directly connected, Ethernet0/2
C    192.168.25.0/24 is directly connected, Ethernet0/0
     55.0.0.0/24 is subnetted, 1 subnets
C       55.55.55.0 is directly connected, Loopback1
     5.0.0.0/32 is subnetted, 1 subnets
C       5.5.5.5 is directly connected, Loopback0
     6.0.0.0/32 is subnetted, 1 subnets
B       6.6.6.6 [20/0] via 192.168.45.4, 00:00:21
                [20/0] via 192.168.35.3, 00:00:21
                [20/0] via 192.168.25.2, 00:04:54

 

Note: The best path won’t change, but parallel paths will be added to routing table.

Now, all three paths through R2, R3 and R4 are available for R5 to reach R6.

Are we done now? Nope! eBGP part is done, but R2, R3 and R4 have iBGP with R1, and R1 is only accepting one path back to R5.

 

R1# sh ip route vrf C1

Routing Table: C1
Codes: C – connected, S – static, B – BGP
Gateway of last resort is not set

B    192.168.45.0/24 [200/10] via 2.2.2.2, 00:00:14
B    192.168.25.0/24 [200/10] via 2.2.2.2, 00:00:14
B    192.168.35.0/24 [200/10] via 2.2.2.2, 00:00:14
     5.0.0.0/32 is subnetted, 1 subnets
B       5.5.5.5 [200/10] via 2.2.2.2, 00:00:14
     6.0.0.0/32 is subnetted, 1 subnets
B       6.6.6.6 [20/0] via 192.168.16.6, 00:49:14

R1# sh ip bgp vpnv4 all
BGP table version is 114, local router ID is 1.1.1.1
Status codes:  * valid, > best, i – internal
Origin codes: i – IGP, e – EGP, ? – incomplete

   Network          Next Hop       Metric LocPrf Weight Path
Route Distinguisher: 1:1 (default for vrf C1)
*>i5.5.5.5/32       2.2.2.2            10    100     0 1 ?
* i                 4.4.4.4            10    100     0 1 ?
* i                 3.3.3.3            10    100     0 1 ?
*> 6.6.6.6/32       192.168.16.6        0            0 2 ?
*>i192.168.25.0     2.2.2.2            10    100     0 1 ?
* i                 4.4.4.4            10    100     0 1 ?
* i                 3.3.3.3            10    100     0 1 ?
*>i192.168.35.0     2.2.2.2            10    100     0 1 ?
* i                 4.4.4.4            10    100     0 1 ?
* i                 3.3.3.3            10    100     0 1 ?
*>i192.168.45.0     2.2.2.2            10    100     0 1 ?
* i                 4.4.4.4            10    100     0 1 ?
* i                 3.3.3.3            10    100     0 1 ?

 

iBGP Multipath

 

R1 has to add parallel iBGP paths from R2, R3 & R4 into vrf ip routing table:

R1(config)#router bgp 100
R1(config-router)#address-family ipv4 vrf C1
R1(config-router-af)#maximum-paths ibgp 4

R1# sh ip route vrf C1

Routing Table: C1
Codes: C – connected, S – static, B – BGP
Gateway of last resort is not set

B    192.168.45.0/24 [200/10] via 4.4.4.4, 00:00:07
                     [200/10] via 3.3.3.3, 00:00:07
                     [200/10] via 2.2.2.2, 00:06:26
B    192.168.25.0/24 [200/10] via 4.4.4.4, 00:00:07
                     [200/10] via 3.3.3.3, 00:00:07
                     [200/10] via 2.2.2.2, 00:06:26
B    192.168.35.0/24 [200/10] via 4.4.4.4, 00:00:12
                     [200/10] via 3.3.3.3, 00:00:12
                     [200/10] via 2.2.2.2, 00:06:30
     5.0.0.0/32 is subnetted, 1 subnets
B       5.5.5.5 [200/10] via 4.4.4.4, 00:00:12
                [200/10] via 3.3.3.3, 00:00:12
                [200/10] via 2.2.2.2, 00:06:30
     6.0.0.0/32 is subnetted, 1 subnets
B       6.6.6.6 [20/0] via 192.168.16.6, 00:55:29

Now all three eBGP links PE-CE side (R5 to R2,3,4) are recognized by R1. But there’s a problem with return traffic! Three inside links between PE LSRs and P (R0) are not being used, let’s check the trace route from CE router R6 to R5":

R6#traceroute     
Target IP address: 5.5.5.5
Source address: 6.6.6.6

Type escape sequence to abort.
Tracing the route to 5.5.5.5

  1 192.168.16.1 
  2 192.168.10.10 [MPLS: Labels 17/23 Exp 0] 
  3 192.168.35.3 [AS 1] [MPLS: Label 23 Exp 0] 
  4 192.168.35.5 [AS 1]

 

All the traffic destined to 5.5.5.5 is passing through R3. Traffic path is R6->R1->R0->R3->R5

When I debugged MPLS tags on R0, I discovered that R1 is always sending traffic with label 17.

*00:38:04.903: MPLS: Et0/1: recvd: CoS=0, TTL=254, Label(s)=17/23
*00:38:04.903: MPLS: Et0/0: xmit: CoS=0, TTL=253, Label(s)=23

 

Label 17 refers to R3, thats why traffic is always passing that specific path, because R1 is labeling MPLS VPN tag inside a BGP next_hop label of R3 as top of the stack label:

 

R1#sh ip bgp vpnv4 all labels
   Network          Next Hop      In label/Out label
Route Distinguisher: 1:1 (C1)
   5.5.5.5/32       4.4.4.4         nolabel/23
                    3.3.3.3         nolabel/23
                    2.2.2.2         nolabel/23
   6.6.6.6/32       192.168.16.6    25/nolabel
   192.168.25.0     4.4.4.4         nolabel/25
                    3.3.3.3         nolabel/25
                    2.2.2.2         nolabel/25
   192.168.35.0     4.4.4.4         nolabel/26
                    3.3.3.3         nolabel/26
                    2.2.2.2         nolabel/26
   192.168.45.0     4.4.4.4         nolabel/27
                    3.3.3.3         nolabel/27
                    2.2.2.2         nolabel/27

R1#sh mpls forwarding-table

Local  Outgoing    Prefix            Outgoing   Next Hop   
tag    tag or VC   or Tunnel Id      interface             
16     Pop tag     192.168.30.0/24   Et0/0      192.168.10.10
17     Pop tag     192.168.40.0/24   Et0/0      192.168.10.10
18     Pop tag     192.168.20.0/24   Et0/0      192.168.10.10
19     Pop tag     10.10.10.10/32    Et0/0      192.168.10.10
20     18          4.4.4.4/32        Et0/0      192.168.10.10
21     16          2.2.2.2/32        Et0/0      192.168.10.10
22     17          3.3.3.3/32        Et0/0      192.168.10.10
25     Untagged    6.6.6.6/32[V]     Et0/1      192.168.16.6
26     Aggregate   192.168.16.0/24[V]

 

CEF and Load-Sharing

 

R1 to reach R2,3 and 4 has only one interface which is connected to the P-LSR – R0.

CEF by default compute the hash (per destination) and send the packet to the next hop, so for the same traffic from R6 to R5 the result of destination hash is always the same. To utilize all three P to PE links and labels, we need to change the default CEF behaviour to “per packet load-sharing”

R1(config)#int e0/0
R1(config-if)#ip load-sharing per-packet

 

With the above command, R1 uses different labels per packet (for forwarding). Now, Let’s check our trace route on R6:

R6#trace 5.5.5.5          

Type escape sequence to abort.
Tracing the route to 5.5.5.5

1 192.168.16.1
  2 192.168.10.10 [MPLS: Labels 18/23 Exp 0]
  3 192.168.45.4 [AS 1] [MPLS: Label 23 Exp 0]
    192.168.35.3 [AS 1] [MPLS: Label 23 Exp 0]
    192.168.25.2 [AS 1] [MPLS: Label 23 Exp 0]
  4 192.168.45.5 [AS 1]
    192.168.35.5 [AS 1]
    192.168.25.5 [AS 1]

 

Per-packet load sharing is not a recommended practice. Also note that when we have different source and destinations the shape and percentage of traffic share among the links gets better and better. As Cisco says: For per-packet load balancing the forwarding process determines the outgoing interface for each packet by looking up the route table and picking the least used interface. This ensures equal utilization of the links, but is a processor intensive task and impacts the overall forwarding performance. This form of per-packet load balancing is not well suited for higher speed interfaces.

 

R1 Final Configuration

ip vrf C1
rd 1:1
route-target export 1:1
route-target import 1:1
!
ip cef
mpls label protocol ldp
mpls ldp logging neighbor-changes
tag-switching tdp discovery hello interval 20
tag-switching tdp discovery hello holdtime 60
tag-switching tdp router-id Loopback0 force
!
interface Loopback0
ip address 1.1.1.1 255.255.255.255
ip router isis
!
interface Ethernet0/0
ip address 192.168.10.1 255.255.255.0
ip router isis
ip load-sharing per-packet
tag-switching ip
!
interface Ethernet0/1
ip vrf forwarding C1
ip address 192.168.16.1 255.255.255.0
!
interface Ethernet0/2
no ip address
shutdown
half-duplex
!
interface Ethernet0/3
no ip address
shutdown
half-duplex
!
router isis
net 01.abcd.abcd.0001.00
is-type level-2-only
!
router bgp 100
no synchronization
bgp log-neighbor-changes
neighbor 2.2.2.2 remote-as 100
neighbor 2.2.2.2 update-source Loopback0
neighbor 3.3.3.3 remote-as 100
neighbor 3.3.3.3 update-source Loopback0
neighbor 4.4.4.4 remote-as 100
neighbor 4.4.4.4 update-source Loopback0
no auto-summary
!
address-family vpnv4
neighbor 2.2.2.2 activate
neighbor 2.2.2.2 send-community extended
neighbor 3.3.3.3 activate
neighbor 3.3.3.3 send-community extended
neighbor 4.4.4.4 activate
neighbor 4.4.4.4 send-community extended
exit-address-family
!
address-family ipv4 vrf C1
neighbor 192.168.16.6 remote-as 2
neighbor 192.168.16.6 activate
maximum-paths ibgp 4
no auto-summary
no synchronization
exit-address-family
!

Posted in IP Routing, MPLS, SP | Leave a comment

Xirrus

Posted on July 19, 2010 by Shafagh Zandi

Perhaps you’ve heard about the Xirrus wireless vendor and its XN16 product: 16 Integrated Access Points in a single device which provides 4.8 Gbps total Wi-Fi bandwidth for up to thousand wireless clients. One array has 16 built-in AP with 48 integrated antennas, giving you 2 GigE ports as uplink to connect to your infrastructure.

 

What you might not be able to find easily, is how to configure them!!

I searched a lot to find what’s the console baud rate and found this:

 

Q: What is Xirrus console speed?

A: Use the following setting when establishing a serial connection:

Bits per second 115200

Databits 8

Parity None

Stopbits 1

Flow control None

 

Q: What is Xirrus default IP address?

A: If a DHCP server is not being used, you may connect using the Array’s default IP
addresses (10.0.2.1).

 

Q: What is Xirrus default username/password?

A: admin/admin

 

Sample Configuration?

administrator
  reset
  edit admin password admin read_write
exit
!
interface eth0
  ip dhcp
  up
exit
!
interface gig1
  ip addr      192.168.0.10
  ip mask      255.255.255.0
  ip gateway
  up
exit
!
interface gig2
  up
exit
!
date-time
  timezone      0 0
exit
!
ssid
  reset
  !
  edit "xirrus"
    band        both broadcast
    vlan        none
    qos         2
    encryption  none global_settings
    auth        open
    enable
  exit
exit
!

 

Further Reference:

To configure XS4

www.xirrus.com/pdfs/array_quick_install_guide_XS4.pdf

To configure XS8 or XS16

www.xirrus.com/pdfs/array_quick_install_guide_XS8-16.pdf

To configure Xirrus Management System (Linux) – XMS

www.xirrus.com/pdfs/XMS_QuickStart_4.0-002B.pdf

Posted in Wireless | Leave a comment

ASA Second Internet

Posted on April 25, 2010 by Shafagh Zandi

As you may or may not know, ASA does not support having two different default gateways through different interfaces, so you can not have two different internet links. As Internet is expensive in Dubai, our customer wants to use two internet ADSL links, One for browsing/emails and another link for VPN tunnels. VPN tunnels are IPsec – site to site tunnels, so we know where is the end-point. There’s a feature in ASA called tunneled route:

“Users will have the option to configure two default gateways, one with a “tunneled” option and one without. All traffic that arrives at the appliance and cannot be routed using learned routes or static routes will be routed through default gateways. If the traffic was encrypted when it initially arrived at the appliance, it will be routed through Default Tunnel Gateway (DTGW); otherwise, it will be routed through Default Gateway (DGW). A set of default gateways can be installed for each virtual context”
http://www.cisco.com/en/US/prod/collateral/iosswrel/ps6537/
ps6586/ps6635/ps6659/prod_white_paper0900aecd805f0bd6.html

But we have to keep in mind that it is not related to our issue, it’s for ingress traffic from tunnel terminating on our ASA… so this feature won’t work for us.

… Not a big deal… we don’t need to configure second default gateway, as we can use static route pointing to that specific site through second outside interface… something like:
route outside2 x.x.x.x 255.255.255.255 217.x.x.x (providers IP)

 After setting up a route to destination through the second link, we have to set our IPsec and ISAKMP packes to use the proper source address from second link using crypto identifiers, then check “show crypto isakmp/ipsec sa” to see if traffic is sourced/originated from second internet link’s IP address…

 But there’s a small problem, I saw traffic is coming through tunnel endpoint and they are able to send and recieve packets using encrypt/decrypt counter of “show crypto ipsec sa” but we were not able to ping or create a connection to the other side.

 Using “debug icmp trace” I figured out that ASA is sending traffic to outside interface (default gateway) instead of outside2, another static route was required for tunneled traffic.

route outside2 10.x.x.x 255.0.0.0 217.x.x.x (providers IP)

Posted in Security | Leave a comment