CORD : Appendix C: Experimental - Installing CORD behind a Proxy

A number of community members have expressed interest in installing a CORD POD behind a proxy.  We have made some progress implementing support for this, as described in these notes. 

Assumptions

The installation makes the following assumptions about the network environment:

• All interaction with the Internet is through an HTTP proxy.
• Public DNS servers are not available.  All DNS resolution happens through a specified name server. 

For development we used a two-machine setup: proxy and head.

• The proxy machine is running squid and dnsmasq. From head it has IP address 10.10.1.1.
• The head machine is being installed with CORD-in-a-Box. It is isolated from the Internet.

Overview of Changes

The changes below were made to support installation of the CORD-in-a-Box (a.k.a. single-node install).  Installation behind a proxy is still not supported for the full CORD installation.

Adding proxy support required numerous changes to CORD 1.0's platform-install module.

Diff of changes: https://github.com/andybavier/platform-install/compare/cord-1.0...andybavier:feature/behind-proxy

• Add proxy vars into files in vars/
• Add proxy environment to each step of the cord-single-playbook.yml
• Add proxy to apt-cacher-ng configuration
• Add proxy environment to existing generated playbooks:
  • docker-install-playbook.yml
  • onos-setup-playbook.yml
  • xos-setup-playbook.yml
  • simulate-fabric-playbook.yml
  • test-client-playbook.yml
• Create new generated playbooks to encapsulate other tasks run inside the VMs from the head node:
  • onos-load-apps-playbook.yml
  • xos-start-cord-pod-playbook.yml
• Configure Docker in VMs to use HTTP proxy by changing /etc/default/docker
• Add dns_forwarders to "unbound" configuration
• Add proxy vars to Juju environment
• Add proxy configuration for Maven to ONOS Docker containers
  • containers that build/push config and vtn apps via XOS
  • containers running ONOS
• Add -E option to sudo in single-node-pod.sh to preserve environment during node bootstrap
• Change hardcoded DNS nameservers in XOS synchronizer code
• Add proxy to Docker configuration in vSG VM by changing /etc/default/docker

In addition, a few minor changes are required to the xos module.

Diff of changes: https://github.com/andybavier/xos/compare/cord-1.0...andybavier:feature/docker-build-proxy

• Pass proxy vars to Docker builds via --build-args in container Makefiles:
  • containers/cord-apps/Makefile
  • containers/onboarding_synchronizer/Makefile
  • containers/onos/Makefile
  • containers/xos/Makefile
• Change git:// to https://
  • containers/xos/Dockerfile
  • containers/xos/Dockerfile.base
  • containers/xos/Makefile

Steps to bring up CORD-in-a-Box

Set proxy variables in the target machine's environment.  For development I've got:

# source ~/environment.sh
# cat ~/environment.sh
export http_proxy=http://10.10.1.1:3128
export https_proxy=https://10.10.1.1:3128
export no_proxy=.cord.lab,localhost
export HTTP_PROXY=http://10.10.1.1:3128
export HTTPS_PROXY=https://10.10.1.1:3128
export NO_PROXY=.cord.lab,localhost

Checkout the GitHub repo hosting the experimental proxy support and switch to the correct branch:

# git clone https://github.com/andybavier/platform-install.git
# cd platform-install
# git checkout feature/behind-proxy

Edit the proxy variables toward the end of vars/cord.yaml, then build the POD:

# cd scripts
# bash single-node-pod.sh -t

The build takes quite a while to complete – at least an hour.  If you don't see an error then assume that everything is OK. 

Status

In my development environment the build completes and passes the tests.

Known Issues

• /etc/resolv.conf inside the service VMs gets reset to nameserver 8.8.8.8 every few minutes