Table of Contents

Introduction

In this page, we'll guide you through the network configurations consumed by the underlay fabric control apps.

The network configuration varies from case to case: it depends on how switches are connected to servers and other equipment. When using CORD, the network config could be automatically generated. But when used outside of CORD, users need to customize their network configuration according to their environment manually.

As an example, we are going to take the following 2-by-2 leaf-spine topology as example for the rest of this page.

Fig.1 2-by-2 leaf-spine topology

Basic (Mandatory) Configuration

There are three types of mandatory configurations for basic L2/L3 connectivity - devices, ports and hosts.

NOTE: This config assumes the following ONOS apps are being used: drivers, openflow-base, netcfghostprovider, lldpprovider, segmentrouting

Devices

EACH router of the underlay fabric require a device config. (Both leaf and spine routers require it) 

"devices" : {
  "of:0000000000000001" : {
    "segmentrouting" : {
      "name" : "Leaf-R1",
      "nodeSid" : 101,
      "routerIp" : "10.6.1.254",
      "routerMac" : "00:00:00:00:01:80",
      "isEdgeRouter" : true,
      "adjacencySids" : []
    },
    "basic" : {
      "driver" : "ofdpa-cpqd"
    }
  }
}

line 2: DPID of the device. 
line 4: Name of the device. It is an arbitrary name to identify the device easily.
line 5: Segment ID assigned to this device. Arbitrary globally unique SID.
line 6: Router IP address. Arbitrary unique router IP.
line 7: Router MAC address. Arbitrary globally unique MAC. This MAC address will be used to reply the ARP request for the Router IP or the Interface IP that will be introduced later. (We recommend using the MAC address of the device's management interface as the router MAC.)
line 8: True for leaf routers. False for spine routers.
line 9: Reserved.  Just put an empty array for now.
line 11-13: This tells ONOS which OpenFlow driver should be loaded for this device.
When using CpqD switches, the driver need to be set to ofdpa-cpqd.
When using hardware switches with OFDPA < 2.0GA (CORD 1.0), the driver need to be set to ofdpa.
When using hardware switches with OFDPA >= 2.0GA (CORD >= 2.0), the driver need to be set to ofdpa3 or this entire block can be eliminated and the driver configuration will be learned automatically.

Warning: The device configuration is different when your ONOS version is newer than 1.9.0. See Configuration guide for ONOS 1.10 for more details

Ports

Every host-facing port of the leaf switch requires a port config. Ports that connect leaf and spine MUST NOT be configured.

"ports" : {
  "of:0000000000000001/1" : {
    "interfaces" : [{	
      "ips" : [ "10.6.1.254/24" ]
    }]
  }
} 

line 2: Connect point, or port, which is identified by switch-dpid/port-number tuple.
line 4: The ips configuration denotes the IP address of the interface as well as subnet. In this example, the interface IP address is 10.0.1.254 and the subnet configured on this interface is 10.0.1.0/24. While it is possible to configure multiple subnets on different ports of the same switch, we usually configure the same subnet for all ports on the same switch. Note that one subnet cannot appear on multiple switches since this violates the basic assumption of prefix routing. In addition, there can be at most one subnet configured on a port.

Hosts

Every host attached to the fabric requires a host config, including physical compute nodes and virtual vSG VM or containers.

"hosts" : {
  "00:00:00:00:00:01/-1" : {
    "basic": {
      "ips": ["10.6.1.1"],
      "location": "of:0000000000000001/1"
    }
  }
}

line 2: Host is identified by mac-address/vlan tuple. In this case vlan -1 means there's no vlan tag.
line 4: IP address of the host. There can be multiple IP addresses.
line 5: The port to which this host attaches identified by device-id/port-number tuple. 

Summary

Putting things all together - the entire network config file for the 2-by-2 leaf-spine should look like:

{
  "ports" : {
  "of:0000000000000001/1" : {
    "interfaces" : [
      {
        "ips" : [ "10.6.1.254/24" ]
      }
    ]
  },
  "of:0000000000000001/2" : {
    "interfaces" : [
      {
        "ips" : [ "10.6.1.254/24" ]
      }
    ]
  },
  "of:0000000000000002/3" : {
    "interfaces" : [
      {
        "ips" : [ "10.6.2.254/24" ]
      }
    ]
  },
  "of:0000000000000002/4" : {
    "interfaces" : [
      {
        "ips" : [ "10.6.2.254/24" ]
      }
    ]
  }
  },
  "devices" : {
    "of:0000000000000001" : {
      "segmentrouting" : {
        "name" : "Leaf-R1",
        "nodeSid" : 101,
        "routerIp" : "10.6.1.254",
        "routerMac" : "00:00:00:00:01:80",
        "isEdgeRouter" : true,
        "adjacencySids" : []
      },
      "basic" : {
        "driver" : "ofdpa-cpqd"
      }
    },
    "of:0000000000000002" : {
      "segmentrouting" : {
        "name" : "Leaf-R2",
        "nodeSid" : 102,
        "routerIp" : "10.6.2.254",
        "routerMac" : "00:00:00:00:02:80",
        "isEdgeRouter" : true,
        "adjacencySids" : []
      },
      "basic" : {
        "driver" : "ofdpa-cpqd"
      }
    },
    "of:0000000000000191" : {
      "segmentrouting" : {
        "name" : "Spine-R1",
        "nodeSid" : 103,
        "routerIp" : "192.168.0.11",
        "routerMac" : "00:00:01:00:11:80",
        "isEdgeRouter" : false,
        "adjacencySids" : []
      },
      "basic" : {
        "driver" : "ofdpa-cpqd"
      }
    },
    "of:0000000000000192" : {
      "segmentrouting" : {
        "name" : "Spine-R2",
        "nodeSid" : 104,
        "routerIp" : "192.168.0.22",
        "routerMac" : "00:00:01:00:22:80",
        "isEdgeRouter" : false,
        "adjacencySids" : []
      },
      "basic" : {
        "driver" : "ofdpa-cpqd"
      }
    }
  },
  "hosts" : {
    "00:00:00:00:00:01/-1" : {
      "basic": {
        "ips": ["10.6.1.1"],
        "location": "of:0000000000000001/1"
      }
    },
    "00:00:00:00:00:02/-1" : {
      "basic": {
        "ips": ["10.6.1.2"],
        "location": "of:0000000000000001/2"
      }
    },
    "00:00:00:00:00:03/-1" : {
      "basic": {
        "ips": ["10.6.2.1"],
        "location": "of:0000000000000002/3"
      }
    },
    "00:00:00:00:00:04/-1" : {
      "basic": {
        "ips": ["10.6.2.2"],
        "location": "of:0000000000000002/4"
      }
    }
  }
}

Advanced (Optional) Configuration

VLAN Cross Connect

"apps": {
  "org.onosproject.segmentrouting": {
    "xconnect": {
      "of:0000000000000001": [{
        "vlan": 10,
        "ports": [5, 73],
        "name": "OLT1"
      }]
    }
  }
}

line 4: DPID of the device we want to configure VLAN cross connect. There can be zero or more VLAN cross connects configured on one device. However, the (DPID, VLAN ID) pair needs to be globally unique.
line 5: VLAN ID we want to cross connect
line 6: Ports we want to cross connect. Note that it has to be exactly two ports.
line 7: Optional Arbitrary name for this cross connect.

vRouter Integration

Please refer to Connecting the POD to Upstream Networks.

Multicast

This configuration is shared between Segment Routing and vRouter. If not configured, both ingressVlan and egressVlan are None (untagged).

"apps": {
  "org.onosproject.core": {
    "multicast": {
      "ingressVlan": "None",
      "egressVlan": "None"
    }
  }
}

line 4: Expected VLAN of the multicast traffic coming from to upstream metro router.
line 5: VLAN of the multicast traffic going to downstream OLTs. 

Dynamic Configurability

Hosts, Cross Connects, and Multicast can be dynamically configured, which means we are able to add/remove them during runtime by posting or deleting the configs through REST API.
However, Devices, Ports and vRouter Integration cannot be dynamically configured at this moment and thus need to be uploaded to ONOS at the beginning.