Background
For unicast traffic, the fabric supports
routing between racks i.e from leaf to leaf via spines, or within leaf ports configured in different subnets
and bridging within a rack i.e. within the ports on the same ToR/leaf configured in the same subnet.
The support so far had been limited to untagged packets (at ingress and egress) to which vlan tags were internally assigned - essentially all ports were similar to 802.1Q access ports. The user had no control over which (internal) vlans were assigned and there was no support for trunk-port like functionality (ports with multiple vlans, where packets ingress and egress the switch with vlan tags).
This feature adds the capability to assign vlan tags to ports and use the ports as access or trunk ports (albiet with slightly different config terminology).
Configuration
This feature significantly changes the way ports are configured in ONOS network-config compared to previous release.
Configuring Access Ports
The necessary but minimum configuration for an access port is simply a vlan.
"ports" : {
"of:0000000000000204/12" : {
"interfaces" : [{
"vlan-untagged": 10
}]
},
"of:0000000000000204/16" : {
"interfaces" : [{
"vlan-untagged": 10
}]
}
}
The example above shows two ports (12 and 16) on switch of:204 that have been assigned to vlan 10 using the “vlan-untagged” keyword. It simply means that packets come in and leave out of these switches untagged, but internally they are assigned vlan 10 and they belong to the bridging domain defined for vlan 10.
With the configuration shown above, the packets will always be bridged, but they cannot be routed out of the vlan (eg. to other subnets). To add the capability to route out of vlan 10, we need to add a subnet/gateway IP (similar to interface-vlans or SVIs in traditional networks).
"ports" : {
"of:0000000000000204/12" : {
"interfaces" : [{
"ips" : [ "10.0.1.254/24"],
"vlan-untagged": 10
}]
},
"of:0000000000000204/16" : {
"interfaces" : [{
"ips" : [ "10.0.1.254/24"],
"vlan-untagged": 10
}]
}
}
Note that vlan 10 is in this case assigned the subnet 10.0.1.0/24, and the gateway IP for hosts in this subnet is 10.0.1.254/32. When the desire is to route out of a vlan, this assignment is currently necessary on all ports configured in the same vlan.
Note: Typically we only expect a single subnet for a vlan. Similar to traditional networks, for us, a subnet == vlan. Different vlans should be configured in different subnets. In certain use-cases, it may be necessary to configure multiple subnets in the same vlan. This is possible by adding more subnet/gateway IPs in the "ips" array.
Configuring Tagged Ports
Tagged port configuration is similar
"ports" : {
"of:0000000000000204/24" : {
"interfaces" : [ {
"name" : "serverA-intf",
"ips" : [ "10.0.2.254/24", "10.0.4.254/24" ],
"vlan-tagged" : [ 20, 40 ]
} ]
}
The configuration above for port 24 on switch of:204 shows two vlans 20 and 40 configured on that port, with corresponding subnets and gateway IPs. Note that there is no specific ordering required in the “ips” or “vlan-tagged” arrays to correlate the vlans to their corresponding subnets. In a future release, we will correlate vlan and subnets config in a more readable way.
Configuring Native Vlans on Tagged ports
An additional configuration possible on tagged ports includes the ability to specify a vlan (and thus a bridging domain) for incoming untagged packets. Typically, such configuration in trunk ports in traditional networks is referred to a native-vlan.
"ports" : {
"of:0000000000000204/24" : {
"interfaces" : [ {
"name" : "serverA-intf",
"ips" : [ "10.0.2.254/24", "10.0.4.254/24", "10.0.1.254/24" ],
"vlan-tagged" : [ 20, 40 ],
"vlan-native" : 10
} ]
}
Note that it is also necessary to configure the subnet/gateway IP corresponding to the native vlan if you wish to route out of that vlan.
Verification
In the fabric, using tagged ports means that hosts connected to the leaf switches should be able to send and receive vlan tagged packets. Configure vlan interfaces in your host, and connect to tagged ports on the ToR. In ONOS, you should be able to “see” your hosts as tagged or untagged
onos> hosts -s id=B2:A4:E2:72:D1:91/None, mac=B2:A4:E2:72:D1:91, location=of:0000000000000204/16, vlan=None, ip(s)=[10.0.1.20, fe80::b0a4:e2ff:fe72:d191] id=FE:60:03:40:3D:07/20, mac=FE:60:03:40:3D:07, location=of:0000000000000204/24, vlan=20, ip(s)=[10.0.2.253]
The results above show that host 10.0.2.253 was discovered on port 24 on switch of:204 with incoming packets from this host bearing vlan 20 tag.
Usage
Switch to switch links - As before there is no need to configure ports on switches that are meant to connect to other switches. The vlan (untagged or tagged) configuration is only meant for ports that are connected to hosts (edge ports).
Furthermore, note that the same vlan can be configured on multiple ToRs - eg. vlan 20 in the figure above. However this does not mean that the ports are in the same bridging domain, because in the fabric, the communication between ToRs is through a routed network. In other words, a host on vlan 20 (untagged or tagged) connected to one ToR can communicate with another host on vlan 20 (untagged or tagged) connected to a different ToR, but the MAC addresses will change as the traffic goes through a routed network.
Please do not use this feature to connect switches in unsupported topologies -- in particular do not use it to connect two leafs (ToRs) directly, as shown below. The fabric is not designed to be one big ethernet fabric. The bridging domain is restricted to within one ToR. If the bridging domain is extended across two ToRs directly linked to each other, there is a chance of loops. In other words, the ToRs/Leafs are not standalone 802.1Q bridges, and should not be used as such.