VTN Components

ServiceNetworkManger

ServiceNetworkManager maintains virtual service network elements and states in a local store. It also exposes REST APIs for creating, updating, getting, and removing service network elements, which can be found in VTN API. Virtual networks and ports consumed by OpenStack Nova are created via Neutron, which is set to use ML2 plugin and networking-onos mechanism driver. When VTN receives Neuton network state update, VTN coverts it to VTN service network to store them. XOS VTN synchronizer, on the other hand, directly calls VTN APIs to make changes on service network states. ServiceNetworkManager is responsible for notifying network state updates as well so that InstanceHandler and DependencyHandler can take actions to data plane.

NodeManager

NodeManager manages compute nodes and bootstraps them to be ready to program. It leverages OVSDB southbound and SSH exec to configure the OVS and the interfaces on compute nodes. The node bootstrap procedure includes the following steps.  

• Connect to OVSDB server on the compute node
• Create an integration bridge, typically “br-int”, and set its controller
• Add VXLAN port and interface to br-int bridge
• Add data network interface to br-int
• Set local management network IP and data network IP address to “br-int” bridge

Addition or configuration of the compute node is done via network configuration.

DependencyHandler

DependencyHandler creates a network connectivity between two service networks, subscriber and provider. Basically, subscriber network instances have an indirect access to provider network. That is, a subscriber network instance is not allowed to access individual provider network instances directly with their IP but automatically selected instance with provider network’s service IP (subnet gateway IP). To enable direct access to a particular provider network instance, “bidirectional” flag should be set true for the provider network. DependencyHandler leverages OpenFlow select group to achieve load-balanced indirect access. Refer to this page(https://wiki.opencord.org/display/CORD/Service+Composition+and+the+Role+of+VTN) for more about interconnecting networks. Creating or removing a dependency is done by updating “providers” on a subscriber service network via service network API. 

InstanceHandler

InstanceHandler creates a network connectivity for an instance. What kind of network connectivity is decided by what type of service network is attached to the instance. Here are the list of supported service network types.

• PRIVATE: Isolated tenant network

• PUBLIC: Provider network that offers connectivity to external network via L3. This network relies on the underlay network infrastructure or vRouter for gateway and first hop routing service.

• MANAGEMENT_LOCAL: Virtual machine management network that offers limited connectivity from the virtual instance's host machine. This network does not span compute nodes, and cannot be part of service chain.

• MANAGEMENT_HOST: Virtual instance management network that offers connectivity to head node. This network runs over the physical management network, and cannot be part of service chain.

• ACCESS_AGENT: special network for R-CORD access agent instance

• (deprecated) VSG: special network for a vSG instances

When InstanceHandler detects a new port on the OVS switch, it searches a service port mapped to the newly added port with the port name on OVS, and then installs a set of flow rules based on the service network and port information. It also adds a host to ONOS for the port with the MAC and IP address of the service port to easily handle ARP and DHCP requests from the instance.

CordVtnArpProxy

VTN proxies ARP requests from instances. Basically, it responses or forwards the request to appropriate port only if the request is for a known virtual instances or a service network gateway IP. Otherwise, it drops the request. Note that as PUBLIC network relies on vRouter or underlay network infrastructure for the gateway and first hop routing, VTN needs to reply with real MAC for the gateway ARP request so that VM sets correct destination MAC for outbound packets, or underlay infrastructure discards the packets. We put this information via publicGateways field of network configuration.

CordVtnDhcpProxy

VTN proxies DHCP requests as well. It responses only if the request comes from a host in ONOS system and replies with the host's IP address.

OVS Pipeline

Ingress Table (id = 0)

priority=55000,arp,arp_tpa=172.27.0.1 actions=LOCAL
priority=55000,arp,in_port=LOCAL,arp_spa=172.27.0.1 actions=CONTROLLER:65535
priority=55000,ip,nw_dst=172.27.0.1 actions=LOCAL
priority=55000,ip,in_port=LOCAL,nw_src=172.27.0.1 actions=goto_table:4 (dstIp)

priority=50000,udp,in_port=dataPlaneIntf,tp_dst=4789 actions=LOCAL
priority=50000,ip,in_port=dataPlaneIntf,nw_dst=dataPlaneIp actions=LOCAL
priority=50000,arp,in_port=dataPlaneIntf,arp_tpa=dataPlaneIp actions=LOCAL
priority=50000,in_port=LOCAL actions=output:data

Example flow rules

In the example, 10.0.5.0/24 is a subscriber service of the service network 10.0.6.0/24, and bidirectional access is enabled. Service instance of the 10.0.5.0/24 network with IP address 10.0.5.2 is in the local port 11, and the other service network instance with IP address 10.0.6.2/24 is in the remote node. 

table=0, priority=55000,vlan_tci=0x1000/0x1000 actions=goto_table:6
table=0, priority=40000,arp actions=CONTROLLER:65535,clear_actions
table=0, priority=40000,udp,tp_src=68,tp_dst=67 actions=CONTROLLER:65535,clear_actions
table=0, priority=0 actions=goto_table:1

table=1, priority=5000,in_port=1(vxlan) actions=goto_table:5
table=1, priority=5000,in_port=2(data) actions=goto_table:4
table=1, priority=5000,ip,in_port=11(instance port),nw_src=10.0.5.2(instance IP) actions=goto_table:2
table=1, priority=4000,in_port=11(instance port) actions=goto_table:3

table=2, priority=50000,ip,nw_src=10.0.5.0/24,nw_dst=10.0.6.1 actions=group:3888259854
table=2, priority=5000,ip,nw_src=10.0.6.0/24,nw_dst=10.0.5.0/24 actions=goto_table:4
table=2, priority=5000,ip,nw_src=10.0.5.0/24,nw_dst=10.0.6.0/24 actions=goto_table:4
table=2, priority=5000,ip,nw_src=10.0.5.0/24,nw_dst=10.0.5.0/24 actions=goto_table:4
table=2, priority=5000,ip,nw_src=10.0.6.0/24,nw_dst=10.0.6.0/24 actions=goto_table:4
table=2, priority=4000,ip,nw_dst=10.0.5.0/24 actions=drop
table=2, priority=4000,ip,nw_dst=10.0.6.0/24 actions=drop
table=2, priority=0 actions=output:2(data)

table=3, priority=5000,in_port=11(instance port) actions=group:3888259854

table=4, priority=5000,ip,nw_dst=10.0.5.2 actions=set_field:fa:16:3e:13:e0:e4->eth_dst,output:11
table=4, priority=5000,ip,nw_dst=10.0.6.2 actions=set_field:fa:16:3e:44:dc:0a->eth_dst,set_field:0x435->tun_id,set_field:10.2.2.29->tun_dst,output:1(vxlan)

table=5, priority=5000,tun_id=0x447,dl_dst=fa:16:3e:13:e0:e4 actions=output:11(instance port)

table=6, priority=50000,arp,dl_vlan=500 actions=CONTROLLER:65535
table=6, priority=5000,dl_vlan=500 actions=pop_vlan,output:2(data port)
table=6, priority=5000,in_port=2,dl_vlan=222 actions=output:7(instance port allowed vlan 222)
table=6, priority=5000,in_port=7,dl_vlan=222 actions=output:2

OFPST_GROUP_DESC reply (OF1.3) (xid=0x2):
 group_id=3888259854,type=select,bucket=actions=set_field:fa:16:3e:44:dc:0a->eth_dst(MAC addr of the instance 10.0.6.2),set_field:10.2.2.29->tun_dst,set_field:0x435->tun_id,output:1(vxlan)