Test automation for a multi-instance ONOS deployment in a cluster of one or more ONOS instances with CORD apps like vrouter, AAA , IGMP etc configured.

configured .

IDTitleFunction NameTest StepsExpected Result
cluster_1Verify if cluster exists with provided no.of ONOS instancestest_onos_cluster_formation_verifyExecute ‘summary’ command on ONOS cli and grep no.of nodes value to verify cluster formationIf cluster already exists, test case should pass else Fail
cluster_2Verify adding additional ONOS instances to already existing clustertest_onos_cluster_adding_members1. Verify if cluster already exists 2. Add few more ONOS instances to the cluster.A new cluster with added ONOS instances should come up without restarting existing cluster
cluster_3Verify cluster status if cluster master instance removedtest_onos_cluster_remove_masterverify if cluster already exists. Grep cluster current master instance Remove master instance ONOS containerCluster with one instance less and new mater elected
cluster_4Verify cluster status if one member goes downtest_onos_cluster_remove_one_memberVerify if cluster exists. Grep one of the member ONOS instance Kill the member containerCluster with one member less and with same master should come up
cluster_5Verify cluster status if two member goes downtest_onos_cluster_remove_two_members1. Verify if cluster exists. 2. Grep two of the member ONOS instances Kill the member containersCluster with two member less and with same master should come up
cluster_6Verify cluster status if N member instances goes downtest_onos_cluster_remove_N_members1. Verify if cluster exists . 2. Grep and kill N no.of member ONOS instancesCluster with N instances less and same master should come up.
cluster_7Verify cluster if few ONOS instances added and removedtest_onos_cluster_add_remove_membersverify if cluster exists Add few instances to cluster Remove the added instancesCluster should be stable before and after addition and deletion of member instances
cluster_8Verify cluster status if few instances removed and addedtest_onos_cluster_remove_add_memberverify if cluster exists Removed few member instances Add back the same instancesCluster should be stable in each stage
cluster_9Verify cluster status after entire cluster restarttest_onos_cluster_restartverify if cluster exists Restart entire clusterCluster should come up with as it is ( master may change )
cluster_10Verify cluster status if current master restarts.test_onos_cluster_master_restartVerify cluster exists Restart current master instance containerCluster master restart should success Cluster with same no.of instances should come up
cluster_11Verify master IP if current master restartstest_onos_cluster_master_ip_after_master_restartVerify if cluster exists Restart current master ONOS instancecluster with new master should come up
cluster_12Verify cluster status if one member restartstest_onos_cluster_one_member_restartverify if cluster exists Grep one of member ONOS instance Restart the instanceCluster should come up with same master
cluster_13Verify cluster status if two members restartstest_onos_cluster_two_members_restartVerify if cluster exists Grep two member instances restart the ONOS containersCluster should come up without changing master
cluster_14Verify cluster status if N members restartstest_onos_cluster_N_members_restartVerify if cluster exists Grep N no.od ONOS instances and restart containersCluster should come with same master
cluster_15Verify cluster master can be changedtest_onos_cluster_master_change1.verify if cluster exists with a master 2. using ONOS cli change cluster master to other than existedCluster master should be changed to new master
cluster_16Verify if cluster current master withdraw it mastershiptest_onos_cluster_withdraw_cluster_current_mastership1. verify if cluster exists with a master 2. using ONOS cli change cluster master by making current master an none to deviceCluster master should changed to new master
cluster_17Verify routes pushed from Quagga to cluster master distributed to all cluster memberstest_onos_cluster_vrouter_routes_in_cluster_members1.verify if cluster exists 2. push few routes from quagga to cluster master 3. verify master has routesAll cluster member should receive the route information
cluster_18Verify vrouter functionality works fine even if cluster master goes downtest_onos_cluster_vrouter_master_down1.verify if cluster exists 2.push few routes from quagga to onos cluster 3.send traffic to above routes 4. kill cluster master instance containerVerify traffic forwards to routes even after cluster master goes down
cluster_19Verify vrouter functionality works fine even if cluster master restartstest_onos_cluster_vrouter_master_restarts1.verify if cluster exists 2.push few routes from quagga to onos cluster 3.send traffic to above routes 4. restarts cluster master instance containerVerify traffic forwards to routes even after cluster master restarts
cluster_20Verify vrouter functionality when vrouter app deactivated on clustertest_onos_cluster_vrouter_app_deactivate1.verify cluster exists 2.verify vrouter functionality 3.deactivate vrouter app in onos cluster master instanceTraffic should not received to routes after the app deactivates
cluster_21Verify vrouter functionality works fine when vrouter app deactivated and cluster master goes downtest_onos_cluster_vrouter_app_deactivate_master_down1.verify if cluster exists 2.verify vrouter works fine 3.deactivate vrouter app and kill master onos instance containerVrouter functionality should not work after app deactivate
cluster_22Verify vrouter functionality works fine even if cluster member goes downtest_onos_cluster_vrouter_member_down1.verify if cluster exists 2.push few routes from quagga to onos cluster 3.send traffic to above routes 4. kill cluster member instance containerVerify traffic forwards to routes even after cluster member goes down
cluster_23Verify vrouter functionality works fine even if cluster member restartstest_onos_cluster_vrouter_member_restart1.verify if cluster exists 2.push few routes from quagga to onos cluster 3.send traffic to above routes 4. restart cluster member instance containerVerify traffic forwards to routes even after cluster member restarts
cluster_24Verify vrouter functionality works fine even if cluster restartstest_onos_cluster_vrouter_cluster_restart1.verify if cluster exists 2.push few routes from quagga to onos cluster 3.send traffic to above routes 4. restart clustertraffic should forwards to routes even after cluster restarts
cluster_25Verify flows works fine on cluster even if cluster master goes downtest_onos_cluster_flow_master_down_flow_udp_port1.push a flow to onos cluster master 2.verify traffic forwards to as per flow 3. now kill cluster master onos instance containerFlow traffic should forward properly as per flow added even after master goes down
cluster_26Verify flows works fine on cluster even if cluster master changetest_cluster_flow_master_change_flow_ecn1.push a flow to onos cluster master 2.verify traffic forwards to as per flow 3. now change cluster masterFlow traffic should forward properly as per flow added even after master changes
cluster_27Verify flows works fine on cluster even if cluster master restartstest_cluster_flow_master_restart_ipv6_extension_header1.push a flow to onos cluster master 2.verify traffic forwards to as per flow 3. now restart cluster masterFlow traffic should forward properly as per flow added even after master restarts
cluster_28Verify igmp include and exclude modes with cluster master restartstest_onos_cluster_igmp_include_exclude_modes_master_restart1.verify if cluster exists 2.verify cluster include and excludes works fine 3. restart cluster masterIgmp include and exclude modes should work properly before and after cluster master restarts
cluster_29Verify igmp include and exclude modes with cluster master goes downtest_onos_cluster_igmp_include_exclude_modes_master_down1.verify if cluster exists 2.verify cluster include and excludes works fine 3. Kill onos cluster master instance containerIgmp include and exclude modes should work properly before and after cluster master goes down
cluster_30Verify igmp data traffic recovery time when master goes downtest_onos_cluster_igmp_include_calculate_traffic_recovery_time_after_master_downVerify if cluster exists Keep sending igmp include and verify traffic Now kill cluster master onos instance containerCalculate time to recover igmp data traffic after master goes down
     
cluster_31Verify Igmp leave after master changetest_onos_cluster_igmp_leave_group_after_master_changeVerify if cluster exists Send igmp include mode and verify traffic Change cluster master Send igmp leave nowNew master should process igmp leave and traffic should not receive after sending leave
cluster_32Verify igmp join and traffic with cluster master changingtest_onos_cluster_igmp_join_after_master_change_traffic_after_master_change_againVerify if cluster exists Send igmp include mode Change cluster master now Send data traffic above registered igmp groupIgmp data traffic should receive to client
cluster_33Verify eap tls authentication on cluster setuptest_onos_cluster_eap_tlsverify if cluster exists Configure radius server ip in onos cluster master Initiate eap tls authentication process from client sideClient should get authenticated for valid certificate
cluster_34Verify eap tls authentication before and after cluster master changetest_onos_cluster_eap_tls_before_and_after_master_changeVerify if cluster exists Verify eap tls authentication process Change cluster master Initiate eap tls authentication process againAuthentication should get success before and after cluster master change
cluster_35Verify eap tls authentication before and after cluster master goes downtest_onos_cluster_eap_tls_before_and_after_master_down1. Verify if cluster exists 2. Verify eap tls authentication process 3. Kill cluster master onos instance container 4. Initiate eap tls authentication process againAuthentication should get success before and after cluster master goes down
cluster_36Verify eap tls authentication with no certificate and master restartstest_onos_cluster_eap_tls_with_no_cert_before_and_after_member_restartverify if cluster exists Verify eap tls authentication fail with no certificates Restart master and repeat step 2Authentication should get fail before and after cluster master restart
cluster_37Verify proxy arp functionality on cluster setup before and after the app deactivatetest_onos_cluster_proxyarp_master_change_and_app_deactivateverify if cluster exists Verify proxy arp functionality Deactivate the app on cluster master Verify proxy apr functionality againProxy arp functionality should work before app deactivate
cluster_38Verify proxyarp functionality on cluster before and after on member goes downtest_onos_cluster_proxyarp_one_member_downVerify if cluster exists Verify if proxyarp works fine on cluster setup Kill one of cluster member onos instance container Verify proxyarp functionality nowProxy arp functionality should before and after cluster member goes down
cluster_39Verify proxyarp functionality with concurrent requests on cluster setuptest_onos_cluster_proxyarp_concurrent_requests_with_multiple_host_and_different_interfacesverify if cluster exists Create multiple interfaces and hosts on OvS Initiate multiple proxy arp requests in parallelCluster should be stable for multiple arp requests in parallel and arp replies should receive for all requests
cluster_40Verify acl rule addition and remove before and after cluster master changetest_onos_cluster_add_acl_rule_before_master_change_remove_acl_rule_after_master_changeVerify if cluster exists Add an acl rule in onos cluster master Change cluster master Remove the acl rule in new cluster masterShould be able to remove acl in new cluster master
cluster_41Verify if acl traffic works fine before and after cluster members goes downtest_onos_cluster_acl_traffic_before_and_after_two_members_downAdd an acl rule Send traffic to match above rule Kill two onos cluster instance containers Send acl traffic againAcl traffic should receive on interface before and after cluster members goes down
cluster_42Verify dhcp relay release on cluster new mastertest_onos_cluster_dhcpRelay_release_dhcp_ip_after_master_changeVerify if cluster exists Initiate dhcp discover and get an ip address from server Change cluster master Send dhcp release to release the leased ipNew master should be able to process dhcp release packet and send to server
cluster_43Verify client gets same dhcp ip after cluster master goes downtest_onos_cluster_dhcpRelay_verify_dhcp_ip_after_master_downVerify if cluster exists Initiate dhcp process and get ip from server Kill cluster master onos instance container Send dhcp request from client to verify if same ip getsClient should receive same ip after cluster master goes down
cluster_44Verify simulating dhcp clients by changing cluster mastertest_onos_cluster_dhcpRelay_simulate_client_by_changing_masterverify if cluster exists Simulate dhcp client-1 Change cluster master Simulate client-2 Change cluster master again Simulate one more client-3All the clients should get valid ip from cluster irrespective cluster change
Cluster_45Verify cord_subscriber functionality works before and after cluster restartstest_onos_cluster_cord_subscriber_join_next_before_and_after_cluster_restartVerify if cluster exists Verify cord_subscriber functionality works Restart cluster Repeat step 2Cord_subscriber should work properly before and after cluster restarts
cluster_46Verify cord_subscriber on 10 channels when cluster member goes downtest_onos_cluster_cord_subscriber_join_recv_10channels_one_cluster_member_downverify if cluster exists Verify cord_subscriber on 10 channels Kill one of the cluster member onos instance container Repeat step 2Cord_subscriber functionality should work properly even after cluster member goes down
cluster_47Verify cord_subscriber on 10 channels when cluster members goes downtest_onos_cluster_cord_subscriber_join_next_10channels_two_cluster_members_down1. verify if cluster exists 2.Verify cord_subscriber on 10 channels 3.Kill two of the cluster member onos instance containers 4. Repeat step 2Cord_subscriber functionality should work properly even after cluster member s goes down
cluster_48Verify multiple devices connected to cluster setuptest_onos_cluster_multiple_ovs_switchesverify if cluster exists Connect multiple devices to cluster setupVerify if all the devices connected to onos cluster setup and each device has master elected
cluster_49Verify multiple devices connected to cluster setuptest_onos_cluster_verify_multiple_ovs_switches_in_cluster_instances1. verify if cluster exists 2. Connect multiple devices to cluster setupEach every cluster member should has information all the devices connected to cluster setup
cluster_50Verify multiple switches connected to cluster setuptest_onos_cluster_verify_multiple_ovs_switches_master_restartverify if cluster exists Connect multiple devices to cluster setup Verify devices information in cluster members Restart master of a deviceWhen master of a device restarts, new master should elected for that device
cluster_51Verify multiple switches connected to cluster setuptest_onos_cluster_verify_multiple_ovs_switches_one_master_down1.verify if cluster exists 2. Connect multiple devices to cluster setup 3. Verify devices information in cluster members 4. Kill cluster onos master of a deviceWhen master of a device goes down, new master should elected for that device
cluster_52Verify multiple switches connected to cluster setuptest_onos_cluster_verify_multiple_ovs_switches_current_master_withdraw_mastership1.verify if cluster exists 2. Connect multiple devices to cluster setup 3. Verify devices information in cluster members 4. Withdraw cluster onos mastership of a deviceWhen master of a device withdraws mastership, new master should elected for that device
cluster_53Verify multiple switches connected to cluster setuptest_onos_cluster_verify_multiple_ovs_switches_cluster_restart1. verify if cluster exists 2. Connect multiple devices to cluster setup 3. Verify devices information in cluster members 4. Restart entire clusterAll the device information should appear in onos instances after cluster restart.masters may change
cluster_54Verify cord_subscriber functionality when cluster master withdraw its mastershiptest_onos_cluster_cord_subscriber_join_next_before_and_after_cluster_mastership_withdrawVerify if cluster exists Verify cord-subscriber functionality Withdraw cluster master Repeat step 2Cord subscriber functionality should work properly before and after cluster master change