VTN Components

ServiceNetworkManger

ServiceNetworkManager maintains virtual service network elements and states in a local store. It also exposes REST APIs for creating, updating, getting, and removing service network elements, which can be found in VTN API. Virtual networks and ports consumed by OpenStack Nova are created via Neutron, which is set to use ML2 plugin and networking-onos mechanism driver. When VTN receives Neuton network state update, VTN coverts it to VTN service network to store them. XOS VTN synchronizer, on the other hand, directly calls VTN APIs to make changes on service network states. ServiceNetworkManager is responsible for notifying network state updates as well so that InstanceHandler and DependencyHandler can take actions to data plane.

NodeManager

NodeManager manages compute nodes and bootstraps them to be ready to program. It leverages OVSDB southbound and SSH exec to configure the OVS and the interfaces on compute nodes. The node bootstrap procedure includes the following steps.  

  • Connect to OVSDB server on the compute node
  • Create an integration bridge, typically “br-int”, and set its controller
  • Add VXLAN port and interface to br-int bridge
  • Add data network interface to br-int
  • Set local management network IP and data network IP address to “br-int” bridge

Addition or configuration of the compute node is done via network configuration.

DependencyHandler

DependencyHandler creates a network connectivity between two service networks, subscriber and provider. Basically, subscriber network instances have an indirect access to provider network. That is, a subscriber network instance is not allowed to access individual provider network instances directly with their IP but automatically selected instance with provider network’s service IP (subnet gateway IP). To enable direct access to a particular provider network instance, “bidirectional” flag should be set true for the provider network. DependencyHandler leverages OpenFlow select group to achieve load-balanced indirect access. Refer to this page(https://wiki.opencord.org/display/CORD/Service+Composition+and+the+Role+of+VTN) for more about interconnecting networks. Creating or removing a dependency is done by updating “providers” on a subscriber service network via service network API

InstanceHandler

InstanceHandler creates a network connectivity for an instance. What kind of network connectivity is decided by what type of service network is attached to the instance. Here are the list of supported service network types.

  • PRIVATE: Isolated tenant network

  • PUBLIC: Provider network that offers connectivity to external network via L3. This network relies on the underlay network infrastructure or vRouter for gateway and first hop routing service.

  • MANAGEMENT_LOCAL: Virtual machine management network that offers limited connectivity from the virtual instance's host machine. This network does not span compute nodes, and cannot be part of service chain.

  • MANAGEMENT_HOST: Virtual instance management network that offers connectivity to head node. This network runs over the physical management network, and cannot be part of service chain.

  • ACCESS_AGENT: special network for R-CORD access agent instance
  • (deprecated) VSG: special network for a vSG instances

When InstanceHandler detects a new port on the OVS switch, it searches a service port mapped to the newly added port with the port name on OVS, and then installs a set of flow rules based on the service network and port information. It also adds a host to ONOS for the port with the MAC and IP address of the service port to easily handle ARP and DHCP requests from the instance.

CordVtnArpProxy

VTN proxies ARP requests from instances. Basically, it responses or forwards the request to appropriate port only if the request is for a known virtual instances or a service network gateway IP. Otherwise, it drops the request. Note that as PUBLIC network relies on vRouter or underlay network infrastructure for the gateway and first hop routing, VTN needs to reply with real MAC for the gateway ARP request so that VM sets correct destination MAC for outbound packets, or underlay infrastructure discards the packets. We put this information via publicGateways field of network configuration.

CordVtnDhcpProxy

VTN proxies DHCP requests as well. It responses only if the request comes from a host in ONOS system and replies with the host's IP address.

OVS Pipeline

Ingress Table (id = 0)

Matches if the packet is tagged or not. Tagged packets go to the VLAN table and all others proceed to the next table, input port table, except for ARP and DHCP requests which are forwarded to the controller. There are also some special rules for handling VXLAN and MANAGEMENT_LOCAL traffic as below.

Additional rules for MANAGEMENT_LOCAL network

priority=55000,arp,arp_tpa=172.27.0.1 actions=LOCAL
priority=55000,arp,in_port=LOCAL,arp_spa=172.27.0.1 actions=CONTROLLER:65535
priority=55000,ip,nw_dst=172.27.0.1 actions=LOCAL
priority=55000,ip,in_port=LOCAL,nw_src=172.27.0.1 actions=goto_table:4 (dstIp)

 

Additional rules for VXLAN

priority=50000,udp,in_port=dataPlaneIntf,tp_dst=4789 actions=LOCAL
priority=50000,ip,in_port=dataPlaneIntf,nw_dst=dataPlaneIp actions=LOCAL
priority=50000,arp,in_port=dataPlaneIntf,arp_tpa=dataPlaneIp actions=LOCAL
priority=50000,in_port=LOCAL actions=output:data

 


Input Port Table (id = 1)

Checks if the packet is received from local ports or the tunnel port. 
  1. if in_port is tunnel port, take the packet through VNI table

  2. if in_port is data port, take the packet though dstIP table

  3. if in_port is local port and src_ip is a service instance, take the packet through access table

  4. if in_port is local port and src_ip is not a service instance, take the packet through service chaining table

Access Table (id = 2)

Matches source and destination IP range and see if the packet is allowed or not. Allowed packets are sent to dstIP table or group table according to the destination. By default, instances in the same network are allowed to talk to each other. For instances in different service networks, service dependency is required for one service instance to send a packet to another service. 
  1. (higher priority) if dst_ip is a service IP (normally gateway IP of the service network) and src and dst networks are in a dependency relationship, take the packet to the group of the dst network service

  2. (medium priority) if src and dst networks are in a dependency relationship, take the packet through dstIP table

  3. (lower priority) if dst_ip is not in any of service network IP range (not external access), drop

  4. (table miss) take the packet through data interface connecting to the underlay fabric

Service Chaining Table (id = 3)

Matches input port and forwards the packet to the next service network group in a dependency. 

DstIP Table(id = 4)

Matches destination IP and forward the packet to local instance port or vxlan port based on the destination instance location.
  1. if dst_ip is the local instance IP, set dst_mac to the destination instance MAC and send to the port where the destination instance connected

  2. if dst_ip is the remote instance IP, set dst_mac to the destination instance MAC, tun_id to the destination service network VNI, tun_dst to the remove compute node data network IP, and send the packet to the vxlan port

VNI Table (id = 5)

Matches VNI and destination MAC, and take the packet though the correct local instance port.

VLAN Table (id = 6)

Matches input port and VLAN ID, and take the packet through the local instance port or the underlay fabric based on the input port.
  1. if in_port is the data interface, take the packet to the correct local instance port by matching vlan_id with in_port

  2. if in_port is the local instance port and the vlan_id is something allowed to the instance, take the packet through the data plane interface

Note that VLAN ID 500 is reserved for vSG WAN.

Group Table

A service network has its own select group consisted of all instances in the service. Each service instance is represented as a bucket in the group, which is an OF rule to reach to the instance. As only one of the bucket is assigned to a packet taken through the service group, the packet is forwarded to the one of the service instance.

Example flow rules

In the example, 10.0.5.0/24 is a subscriber service of the service network 10.0.6.0/24, and bidirectional access is enabled. Service instance of the 10.0.5.0/24 network with IP address 10.0.5.2 is in the local port 11, and the other service network instance with IP address 10.0.6.2/24 is in the remote node. 

example table=0 flows
table=0, priority=55000,vlan_tci=0x1000/0x1000 actions=goto_table:6
table=0, priority=40000,arp actions=CONTROLLER:65535,clear_actions
table=0, priority=40000,udp,tp_src=68,tp_dst=67 actions=CONTROLLER:65535,clear_actions
table=0, priority=0 actions=goto_table:1
example table=1 (inPort) flows
table=1, priority=5000,in_port=1(vxlan) actions=goto_table:5
table=1, priority=5000,in_port=2(data) actions=goto_table:4
table=1, priority=5000,ip,in_port=11(instance port),nw_src=10.0.5.2(instance IP) actions=goto_table:2
table=1, priority=4000,in_port=11(instance port) actions=goto_table:3
example table=2 (access) flows
table=2, priority=50000,ip,nw_src=10.0.5.0/24,nw_dst=10.0.6.1 actions=group:3888259854
table=2, priority=5000,ip,nw_src=10.0.6.0/24,nw_dst=10.0.5.0/24 actions=goto_table:4
table=2, priority=5000,ip,nw_src=10.0.5.0/24,nw_dst=10.0.6.0/24 actions=goto_table:4
table=2, priority=5000,ip,nw_src=10.0.5.0/24,nw_dst=10.0.5.0/24 actions=goto_table:4
table=2, priority=5000,ip,nw_src=10.0.6.0/24,nw_dst=10.0.6.0/24 actions=goto_table:4
table=2, priority=4000,ip,nw_dst=10.0.5.0/24 actions=drop
table=2, priority=4000,ip,nw_dst=10.0.6.0/24 actions=drop
table=2, priority=0 actions=output:2(data)
example table=3 (service chaining) flows
table=3, priority=5000,in_port=11(instance port) actions=group:3888259854
example table=4 (dstIp) flows
table=4, priority=5000,ip,nw_dst=10.0.5.2 actions=set_field:fa:16:3e:13:e0:e4->eth_dst,output:11
table=4, priority=5000,ip,nw_dst=10.0.6.2 actions=set_field:fa:16:3e:44:dc:0a->eth_dst,set_field:0x435->tun_id,set_field:10.2.2.29->tun_dst,output:1(vxlan)
example table=5 (vni) flows
table=5, priority=5000,tun_id=0x447,dl_dst=fa:16:3e:13:e0:e4 actions=output:11(instance port)
example table=6 (vlan) flows
table=6, priority=50000,arp,dl_vlan=500 actions=CONTROLLER:65535
table=6, priority=5000,dl_vlan=500 actions=pop_vlan,output:2(data port)
table=6, priority=5000,in_port=2,dl_vlan=222 actions=output:7(instance port allowed vlan 222)
table=6, priority=5000,in_port=7,dl_vlan=222 actions=output:2
example group table
OFPST_GROUP_DESC reply (OF1.3) (xid=0x2):
 group_id=3888259854,type=select,bucket=actions=set_field:fa:16:3e:44:dc:0a->eth_dst(MAC addr of the instance 10.0.6.2),set_field:10.2.2.29->tun_dst,set_field:0x435->tun_id,output:1(vxlan)

Attachments: