Juniper and DATACOM VPWS RSVP Affinity Interoperability Use Case

In this article, I’ll walk you through a classic MPLS RSVP use case between Juniper and DATACOM 2 with some manipulation in the affinity bits, also known as link coloring, in order to influence the potential paths that will be chosen by the CSPF (constrained shortest path first) . So, let’s assume a service provider has this design requirement:

A ISP needs to setup a VPWS (l2-circuit) between 2 PEs, Juniper vMX and DATACOM SW4_DM4001 (Figure 1), however, this LSP should never use a specific link, which for the sake of this example, let’s assume it is a high latency link. Plus, the service provider does not want to change OSPF costs because this traffic engineering should only influence this particular VPWS.

To address this design requirement, RSVP with affinity fits like a glove. Essentially, you can color MPLS RSVP links and include or exclude them from the CSPF algorithm. In this case, we’ll color this undesired high-latency link as yellow and when setting the RSVP LSPs between these two PEs we’ll exclude the yellow color. In this particular example, since we’re not using explicit-path to enforce a strict path and just excluding a certain link, ultimately the CSPF will end up following the IGP shortest path, however, it won’t use this latency link.

When interoperating RSVP with affinity in a multi-vendor network you have to keep in mind that, on wire, the affinity bits field is 32-bit long. However, when setting this attribute on the cli, some vendors have different approaches. For example, on Junos you set an integer, which represents the bit position ranging from 0 to 31. On DATACOM DM4000 Switches, on other hand, you also set an integer, but this represents a decimal absolute value that you’ll have on wire. So, let’s say this yellow link is supposed to be 0x10 (decimal 16) on wire:

  • On Junos 1, you should set this attribute as 4 (i.e., 2^4=16). By the way, Juniper uses the admin-group container to set the affinity bits:

    1
    set protocols mpls admin-groups yellow 4
  • On DATACOM DM4000, you have to set the affinity attribute as 16:

    1
    rsvp signalling link attributes 16

Topology

Figure 1 shows the topology used in this use case. The undesired high latency link is the one between SW2_DM4004 and SW4_DM4001 on VLAN 3004, which has the topology text label affinity 0x10 | yellow. In this topology, the RSVP LSPs will be setup between vMX and SW4_DM4001. As you can see in Figure 1, considering the perspective from SW4_DM4001 (working as both egress PE and ingress PE), there are three MPLS RSVP links available, VLAN 4004 (affinity 0), 4000 (affinity 0) and 3004 (affinity 0x10):

Figure 1

Configuration

For the sake of readability, I’ll show only the essential part of the configuration of both Junos vMX and SW4_DM4001. According to the configuration bellow, there are two RSVP LSPs, VMX_TO_SW4 and SW4_TO_VMX. They both exclude affinity 0x10. These two LSPs are used by this VPWS vc-id 1600, which xconnects customers A’s site#1 and site#2 (Figure 1).

Info
When configuring the affinity attribute, since this is meant to be a link-state attribute, usually the entire link is expected to have the same affinity attribute on each end of the link. Make sure you set the same affinity on both switches connecting this link.

Junos vMX

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
root@vMX# show
version 14.1R4.8;
system {
host-name vMX;
}
interfaces {
ge-0/0/0 {
vlan-tagging;
encapsulation vlan-ccc;
unit 1600 {
encapsulation vlan-ccc;
vlan-id 1600;
}
}
ge-0/0/1 {
vlan-tagging;
unit 2001 {
vlan-id 2001;
family inet {
address 2.3.10.10/24;
}
family mpls;
}
}
ge-0/0/2 {
vlan-tagging;
unit 2002 {
vlan-id 2002;
family inet {
address 2.1.10.10/24;
}
family mpls;
}
}
lo0 {
unit 1 {
family inet {
address 10.10.10.10/32;
}
}
}
}
protocols {
rsvp {
interface ge-0/0/1.2001;
interface ge-0/0/2.2002;
}
mpls {
admin-groups {
yellow 4;
}
label-switched-path VMX_TO_SW4 {
to 4.4.4.4;
admin-group exclude yellow;
}
interface ge-0/0/1.2001;
interface ge-0/0/2.2002;
}
ospf {
traffic-engineering;
area 0.0.0.0 {
interface lo0.1 {
passive;
}
interface ge-0/0/1.2001 {
interface-type p2p;
metric 10;
authentication {
md5 1 key "$9$oMaZjHkPznCs2aUHmF3IEc"; ## SECRET-DATA
}
}
interface ge-0/0/2.2002 {
interface-type p2p;
metric 10;
authentication {
md5 1 key "$9$Nl-VYg4Ziqf8X-wgJHk/Ct"; ## SECRET-DATA
}
}
}
}
ldp {
interface lo0.1;
}
l2circuit {
neighbor 4.4.4.4 {
interface ge-0/0/0.1600 {
virtual-circuit-id 1600;
mtu 9198;
pseudowire-status-tlv;
}
}
}
}

DATACOM SW4_DM4001

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
SW4_DM4001#show running-config
Building configuration...
!
! Board models in this configuration:
! Unit 1: ETH24GX+2x10GX H Series
!
hostname SW4_DM4001
!
ip routing
!
router ospf
router-id 4.4.4.4
network 4.3.4.0/24 area 0
network 4.4.4.4/32 area 0
network 4.2.4.0/24 area 0
network 3.2.4.0/24 area 0
mpls traffic-eng
!
interface vlan 3004
ip address 3.2.4.4/24
link-detect
ip ospf authentication message-digest
ip ospf message-digest-key 1 md5 ********
ip ospf cost 10
ip ospf network point-to-point
set-member tagged ethernet 1/25
rsvp signalling link attributes 16
rsvp enable
!
interface vlan 4000
ip address 4.3.4.4/24
link-detect
ip ospf authentication message-digest
ip ospf message-digest-key 1 md5 ********
ip ospf cost 10
ip ospf network point-to-point
set-member tagged ethernet 1/7
rsvp enable
!
interface vlan 4004
ip address 4.2.4.4/24
link-detect
ip ospf authentication message-digest
ip ospf message-digest-key 1 md5 ********
ip ospf cost 10
ip ospf network point-to-point
set-member tagged port-channel 24
rsvp enable
!
interface port-channel 24
set-member ethernet range 1/5 1/6
!
interface loopback 0
ip address 4.4.4.4/32
mpls enable
!
mpls te
interface te-tunnel 10
tunnel name SW4_TO_VMX
tunnel mpls destination 10.10.10.10
tunnel mpls traffic-eng affinity 0 0 16
no shutdown
!
mpls ldp neighbor 10.10.10.10
!
mpls vpws
vpn 1600
xconnect vlan 1600 vc-type vlan
neighbor 10.10.10.10 pwid 1600 mplstype te-tunnel 10
no shutdown
!

Verification and Network Failures Reconvergence

Now we’ll verify how these LSPs will reconverge as we shut network links down. Specifically, we’ll focus on all links connected to SW4_DM4001, since both LSPs will have to go over them, they’re strategic to analyze the reconvergence of this topology:

VLAN affinity SW_DM4001 interface
4004 0 (no color) port-channel 24
4000 0 (no color) ethernet 1/7
3004 0x10 (yellow) ethernet 1/25

Initially, all links are UP. As shown in Figure 2, both RSVP LSPs are following the shortest path, going through SW3_DM4001 (which is the PHP LSR switch for both LSPs). As expected, they both avoided the yellow link - the shortest path will change though, as I start to shut some links down:

Figure 2

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
root@vMX# run show mpls lsp detail
Ingress LSP: 1 sessions
4.4.4.4
From: 10.10.10.10, State: Up, ActiveRoute: 0, LSPname: VMX_TO_SW4
ActivePath: (primary)
LSPtype: Static Configured, Penultimate hop popping
LoadBalance: Random
Encoding type: Packet, Switching type: Packet, GPID: IPv4
*Primary State: Up
Priorities: 7 0
SmartOptimizeTimer: 180
Exclude: yellow
Computed ERO (S [L] denotes strict [loose] hops): (CSPF metric: 15)
2.3.10.3 S 4.3.4.4 S
Received RRO (ProtectionFlag 1=Available 2=InUse 4=B/W 8=Node 10=SoftPreempt 20=Node-ID):
2.3.10.3 4.3.4.4
Total 1 displayed, Up 1, Down 0
Egress LSP: 1 sessions
10.10.10.10
From: 4.4.4.4, LSPstate: Up, ActiveRoute: 0
LSPname: SW4_TO_VMX
Suggested label received: -, Suggested label sent: -
Recovery label received: -, Recovery label sent: -
Resv style: 1 FF, Label in: 3, Label out: -
Time left: 140, Since: Tue Mar 21 16:20:03 2017
Tspec: rate 0bps size 0bps peak 0bps m 20 M 1500
Port number: sender 0 receiver 10 protocol 0
PATH rcvfrom: 2.3.10.3 (ge-0/0/1.2001) 5 pkts
Adspec: received MTU 1500
PATH sentto: localclient
RESV rcvfrom: localclient , Entropy label: Yes
Record route: 4.3.4.4 2.3.10.3 <self>
Total 1 displayed, Up 1, Down 0
Transit LSP: 0 sessions
Total 0 displayed, Up 0, Down 0
[edit]
root@vMX#
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
SW3_DM4001#show mpls forwarding-table
Number of entries: 2 (2 ILMs)
Action Codes: FWD - Forward, PHP - Penultimate Hop Popping,
POP - Pop and L3 Lookup, PSH - Push Label, SWP - Label Swap,
DIS - Discard
Role Codes: D - Detour Tunnel, LR - Local Repair
Status Codes: A - Active, I - Inactive, P - Pending, S - Stale
--------------------+-----+---------------+---------------+-----------+---------
Prefix, Tunnel ID | Act | Incoming | Outgoing | Outgoing | Status
or Lookup Table | ion | Label/ | Label/ | Interface | & Role
| | Protocol | Protocol | |
--------------------+-----+---------------+---------------+-----------+---------
VMX_TO_SW4 PHP 28/RSVP ImpNull/RSVP VLAN 4000 A
SW4_TO_VMX PHP 27/RSVP ImpNull/RSVP VLAN 2001 A
--------------------+-----+---------------+---------------+-----------+---------
SW3_DM4001#
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
SW4_DM4001#show mpls te traffic-eng tunnels
Tunnel-Name Destination Protect. Up-If Down-If Adm/Oper
-------------------- ---------------- -------- --------- --------- ----------
SW4_TO_VMX 10.10.10.10 none 4000,3 up/up
VMX_TO_SW4 4.4.4.4 4000,0 up/up
SW4_DM4001#show mpls te traffic-eng tunnels role head detail
Name: SW4_TO_VMX [Instance 0]
Src: 4.4.4.4 Dst: 10.10.10.10
Status:
Admin: up Oper: up Role: head Dir: out
Setup Prio: 0 Holding Prio: 0 LSP_ID: 0
Affinity : 0x 0 [Incl.Any] 0x 0 [Incl.All] 0x 10 [Excl.Any]
Frr Affinity : 0x 0 [Incl.Any] 0x 0 [Incl.All] 0x 0 [Excl.Any]
(No Fast Reroute)
Area: 0 (OSPF)
Resources:
Max rate: (best effort)
History:
Tunnel:
Time since created : 0 days, 1 hours, 29 minutes, 54 seconds
Total up time : 0 days, 1 hours, 29 minutes, 28 seconds
InLabel: -
OutLabel: VLAN 4000, 3
SW4_DM4001#

Shutting VLAN 4000 - Ethernet 1/7 Down

1
2
3
4
5
6
SW4_DM4001(config-if-eth-1/7)#shut
SW4_DM4001(config-if-eth-1/7)#
Mar 21 14:31:33.569332 : [Master] <5> Interface Ethernet 1/7 changed state to down (shutdown)
Mar 21 14:31:33.905389 : [Master] <5> RSVP-ADJCHG - Tunnel (10) to destination 10.10.10.10 is DOWN
Mar 21 14:31:38.035333 : [Master] <5> RSVP-ADJCHG - Tunnel (10) to destination 10.10.10.10 is UP
SW4_DM4001(config-if-eth-1/7)#

As you can see in Figure 3, both LSPs don’t go through SW3_DM4001 anymore. Now, they have shifted to the top paths in this topology going through SW1_DM4004 and SW2_DM4004. As expected, again, they both avoided the yellow link, even though the yellow link has the same metric as the current shortest path:

Figure 3

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
root@vMX# run show mpls lsp detail
Ingress LSP: 1 sessions
4.4.4.4
From: 10.10.10.10, State: Up, ActiveRoute: 0, LSPname: VMX_TO_SW4
ActivePath: (primary)
LSPtype: Static Configured, Penultimate hop popping
LoadBalance: Random
Encoding type: Packet, Switching type: Packet, GPID: IPv4
*Primary State: Up
Priorities: 7 0
SmartOptimizeTimer: 180
Exclude: yellow
Computed ERO (S [L] denotes strict [loose] hops): (CSPF metric: 20)
2.1.10.1 S 3.1.2.2 S 4.2.4.4 S
Received RRO (ProtectionFlag 1=Available 2=InUse 4=B/W 8=Node 10=SoftPreempt 20=Node-ID):
2.1.10.1 3.1.2.2 4.2.4.4
Total 1 displayed, Up 1, Down 0
Egress LSP: 1 sessions
10.10.10.10
From: 4.4.4.4, LSPstate: Up, ActiveRoute: 0
LSPname: SW4_TO_VMX
Suggested label received: -, Suggested label sent: -
Recovery label received: -, Recovery label sent: -
Resv style: 1 FF, Label in: 3, Label out: -
Time left: 154, Since: Tue Mar 21 16:20:03 2017
Tspec: rate 0bps size 0bps peak 0bps m 20 M 1500
Port number: sender 0 receiver 10 protocol 0
PATH rcvfrom: 2.1.10.1 (ge-0/0/2.2002) 35 pkts
Adspec: received MTU 1500
PATH sentto: localclient
RESV rcvfrom: localclient , Entropy label: Yes
Record route: 4.2.4.4 3.1.2.2 2.1.10.1 <self>
Total 1 displayed, Up 1, Down 0
Transit LSP: 0 sessions
Total 0 displayed, Up 0, Down 0
[edit]
root@vMX#
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
SW2_DM4004#show mpls forwarding-table
Number of entries: 2 (2 ILMs)
Action Codes: FWD - Forward, PHP - Penultimate Hop Popping,
POP - Pop and L3 Lookup, PSH - Push Label, SWP - Label Swap,
DIS - Discard
Role Codes: D - Detour Tunnel, LR - Local Repair
Status Codes: A - Active, I - Inactive, P - Pending, S - Stale
--------------------+-----+---------------+---------------+-----------+---------
Prefix, Tunnel ID | Act | Incoming | Outgoing | Outgoing | Status
or Lookup Table | ion | Label/ | Label/ | Interface | & Role
| | Protocol | Protocol | |
--------------------+-----+---------------+---------------+-----------+---------
SW4_TO_VMX SWP 17/RSVP 17/RSVP VLAN 3005 A
VMX_TO_SW4 PHP 16/RSVP ImpNull/RSVP VLAN 4004 A
--------------------+-----+---------------+---------------+-----------+---------
SW2_DM4004#

Shutting VLAN 4004 - Port-channel 24 Down

After shutting VLAN 4004 - Port-channel 24 down, the only remaining MPLS/RSVP-enabled link is the VLAN 3004, which has the yellow affinity color - Figure 4. Even though we have back-to-back L3 reachability (see the ping test down bellow) between vMX and SW4_DM4001, both LSPs won’t consider this link as an eligible path. As a result, they will be operational down and the VPWS circuit that uses these underlying LSPs won’t be set-up, as we initially intended:

Figure 4

1
2
3
4
5
6
7
8
9
10
11
12
13
SW4_DM4001(config)#interface port-channel 24
SW4_DM4001(config-if-port-ch-24)#show this
interface port-channel 24
set-member ethernet range 1/5 1/6
!
SW4_DM4001(config-if-port-ch-24)#shut
SW4_DM4001(config-if-port-ch-24)#
Mar 21 14:42:00.768217 : [Master] <5> Interface Ethernet 1/5 changed state to down (shutdown) - member of Port-channel 24
Mar 21 14:42:01.015131 : [Master] <5> Interface Ethernet 1/6 changed state to down - member of Port-channel 24
Mar 21 14:42:01.041489 : [Master] <5> Interface Port-channel 24 changed state to down
Mar 21 14:42:01.527625 : [Master] <5> RSVP-ADJCHG - Tunnel (10) to destination 10.10.10.10 is DOWN
SW4_DM4001(config-if-port-ch-24)#
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
root@vMX# run show mpls lsp detail
Ingress LSP: 1 sessions
4.4.4.4
From: 0.0.0.0, State: Dn, ActiveRoute: 0, LSPname: VMX_TO_SW4
ActivePath: (none)
LSPtype: Static Configured, Penultimate hop popping
LoadBalance: Random
Encoding type: Packet, Switching type: Packet, GPID: IPv4
Primary State: Dn
Priorities: 7 0
SmartOptimizeTimer: 180
Exclude: yellow
Will be enqueued for recomputation in 2 second(s).
1 Mar 21 17:12:53.888 CSPF failed: no route toward 4.4.4.4[61 times]
Total 1 displayed, Up 0, Down 1
Egress LSP: 0 sessions
Total 0 displayed, Up 0, Down 0
Transit LSP: 0 sessions
Total 0 displayed, Up 0, Down 0
[edit]
root@vMX# run show route 4.4.4.4
inet.0: 15 destinations, 15 routes (15 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
4.4.4.4/32 *[OSPF/10] 00:41:30, metric 30
> to 2.1.10.1 via ge-0/0/2.2002
[edit]
root@vMX# run traceroute 4.4.4.4 source 10.10.10.10
traceroute to 4.4.4.4 (4.4.4.4) from 10.10.10.10, 30 hops max, 40 byte packets
1 2.1.10.1 (2.1.10.1) 3.824 ms 3.475 ms 3.179 ms
2 3.1.2.2 (3.1.2.2) 3.682 ms 3.172 ms 3.136 ms
3 4.4.4.4 (4.4.4.4) 1.767 ms 1.644 ms 1.491 ms
[edit]
root@vMX#
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
SW4_DM4001(config-if-port-ch-24)#
SW4_DM4001(config-if-port-ch-24)#show mpls forwarding-table
No entries to display.
SW4_DM4001(config-if-port-ch-24)#show mpls te traffic-eng tunnels
Tunnel-Name Destination Protect. Up-If Down-If Adm/Oper
-------------------- ---------------- -------- --------- --------- ----------
SW4_TO_VMX 10.10.10.10 none up/down
SW4_DM4001(config-if-port-ch-24)#show mpls te traffic-eng tunnels detail
Name: SW4_TO_VMX [Instance 0]
Src: 4.4.4.4 Dst: 10.10.10.10
Status:
Admin: up Oper: down Role: head Dir: out
Setup Prio: 0 Holding Prio: 0 LSP_ID: 0
Affinity : 0x 0 [Incl.Any] 0x 0 [Incl.All] 0x 10 [Excl.Any]
Frr Affinity : 0x 0 [Incl.Any] 0x 0 [Incl.All] 0x 0 [Excl.Any]
(No Fast Reroute)
Area: 0 (OSPF)
Resources:
Max rate: (best effort)
History:
Tunnel:
Time since created : 0 days, 2 hours, 21 minutes, 4 seconds
Total up time : 0 days, 1 hours, 48 minutes, 46 seconds
InLabel: -
OutLabel: -
SW4_DM4001(config-if-port-ch-24)#

Final Thoughts

RSVP is extremely flexible when it comes to LSP traffic engineering, meaning you can include or exclude certain criteria, in this case we leveraged the affinity bits to avoid an undesired high latency link when the CSPF is being computed. Plus, an advantage of RSVP is the fact that you can traffic engineer paths without messing up with the IGP forwarding table. If you are using affinity in a multi-vendor network watch out how your vendor set these bits, the math should be straight forward, make sure you get it right, otherwise the LSP won’t be computed correctly by the CSPF.

References