Resolve "Create a virtual machine for running the labs"

See merge request !462

Co-authored-by: Katharina Renk <katharina.renk@stud.h-da.de>

.gitignore

@@ -60,7 +60,7 @@ applications/venv-manager/*.yaml
 applications/venv-manager/*.json
 applications/venv-manager/clab-*
 
-#debug
+# debug
 __debug_bin
 
 #misc
@@ -68,3 +68,7 @@ __debug_bin
 
 # plugins
 plugin-registry/plugins/
+
+# lab-vm
+lab-vm/vm-with-packer/VM/
+lab-vm/vm-with-vagrant/.vagrant/
\ No newline at end of file

lab-vm/README.md

+# Lab-VM
+
+Currently, the Lab-VM has two virtual machine providers Vagrant and Packer
+
+---
+
+## Vagrant
+
+### What is Vagrant?
+
+- Vagrant is a tool to build a virtual machine based on a single file called Vagrantfile (similar to a Dockerfile). You can manage the virtual machine over Vagrant itself or over VirtualBox.
+
+### Why do we use Vagrant?
+
+- We use Vagrant to simulate the **goSDN-Controller** and the **gNMI-Target** in an virtual environment for education purposes. In addition, it can also be used to quickly set up a virtual test environment. Vagrant will build a virtual machine on the target system. Every target system has to build the virtual machine for itself, but you can avoid transferring a whole virtual machine image over networks or storage systems.
+
+### How do we use Vagrant?
+
+1. Firstly, Vagrant and VirtualBox have to be installed. The installation steps can be found at the [wiki](https://code.fbi.h-da.de/danet/gosdn/-/wikis/Deployment/Virtual-Machine-with-Vagrant) page.
+2. After that, you can simply modify the existing Vagrantfile in the directory or create a new one. You can find a detailed documentation for the Vagrantfile at the [Vagrant Documentation Website](https://developer.hashicorp.com/vagrant/docs/vagrantfile).
+3. Build the Vagrantfile. With the commands that are listed one the [wiki](https://code.fbi.h-da.de/danet/gosdn/-/wikis/Deployment/Virtual-Machine-with-Vagrant) page you can build and manage the virtual machine. 
+4. Connect to the virtual machine over VirtualBox or following command: 
+   ```bash
+   vagrant ssh
+   ```
+
+### How is the vm-with-vagrant folder structured?
+
+```
+vm-with-vagrant
+└── Vagrantfile (file to build the virtual machine)
+```
+
+### How can I use Vagrant?
+
+Take a look at the [wiki](https://code.fbi.h-da.de/danet/gosdn/-/wikis/Deployment/Virtual-Machine-with-Vagrant).
+
+---
+
+## Packer
+
+### What is Packer?
+
+- [Packer](https://developer.hashicorp.com/packer) is an automatic virtual machine image creator.
+
+### Why do we use Packer?
+
+- We use **Packer** to create a virtual machine for use with the **goSDN** controller in education. Using packer has the advantage of building a single virtual machine image once and deploying it for all other users that want to use this image. Therefore, no installation of another tool on the user's system is required, except the software to run a virtual machine.
+
+### How do we use Packer?
+
+- We use a shell script which consists of three steps.
+  - The first step is to install the current **Ubuntu Server LTS** release and install the necessary software for building and running **goSDN** and the **gNMI-Target**.
+  - After each of the first two steps a VirtualBox Image will be created.
+  - In the second step **goSDN** and **gNMI-Target** will be cloned and built for use.
+  - In the last step the image of the first step will be deleted, if the second step was successful, to save space and clean up.
+
+### How is the vm-with-packer folder structured?
+
+```
+vm-with-packer
+├── gosdn_vm.pkr.hcl (Packer builder file with the virtualbox plugin [1])
+├── http
+│   ├── meta-data (Ubuntu Server Autoinstall config file [2])
+│   └── user-data (Ubuntu Server Autoinstall config file [2], currently empty)
+├── pipeline.sh (Shell script which containes the commands for building the virtual machine image)
+├── scripts
+│   ├── gosdn_clone.sh (Shell script, which contains the commands for cloning and building goSDN and the gNMI-Target)
+│   └── setup.sh (Shell script, which contains the commands for installing the necessary software for building and running like Docker [3] and containerlab [4])
+└── VM
+    └── goSDN VM.ova (Virtual machine image, if the script run successfully)
+```
+
+### How can I use Packer?
+
+- Take a look at the [wiki](https://code.fbi.h-da.de/danet/gosdn/-/wikis/Deployment/Virtual-Machine-with-Packer).
+
+### ⚠️ Note
+
+If you want to change the password of the user **gosdn** for Ubuntu Server a new one has to be created with the command **mkpasswd -m sha512crypt newpassword** (**newpassword** is a placeholder for the password you want to use). **mkpasswd** is in the **whois** package for Debian/Ubuntu.
+Alternatively you can use the command **openssl passwd -6 newpassword** if you can not use **mkpasswd**.
+
+You **can not** set the password in **plaintext** in meta-data!
+
+---
+
+## References
+
+[1] <https://developer.hashicorp.com/packer/plugins/builders/virtualbox/iso>  
+[2] <https://ubuntu.com/server/docs/install/autoinstall-reference>  
+[3] <https://www.docker.com/>  
+[4] <https://containerlab.dev/>

lab-vm/vm-with-packer/gosdn_vm.pkr.hcl

+packer {
+  required_plugins {
+    virtualbox = {
+      version = ">= 0.0.1"
+      source  = "github.com/hashicorp/virtualbox"
+    }
+  }
+}
+
+source "virtualbox-iso" "baseimage" {
+  guest_os_type  = "Ubuntu_64"
+  headless       = false
+  http_directory = "http"
+
+  boot_command = ["<wait>c<wait>set gfxpayload=keep<enter><wait>linux /casper/vmlinuz quiet autoinstall ds=nocloud-net\\;s=http://{{.HTTPIP}}:{{.HTTPPort}}/ ---<enter><wait>initrd /casper/initrd<wait><enter><wait>boot<enter><wait>"]
+
+  vm_name = "goSDN VM"
+
+  disk_size = 40000
+
+  gfx_controller = "vmsvga"
+  gfx_vram_size  = 128
+
+  iso_url                = "https://ftp.halifax.rwth-aachen.de/ubuntu-releases/jammy/ubuntu-22.04.2-live-server-amd64.iso"
+  iso_checksum           = "file:https://ftp.halifax.rwth-aachen.de/ubuntu-releases/jammy/SHA256SUMS"
+  ssh_username           = "gosdn"
+  ssh_password           = "gosdn"
+  ssh_timeout            = "10m"
+  ssh_handshake_attempts = 1000
+  shutdown_command       = "echo 'packer' | sudo -S shutdown -P now"
+
+  hard_drive_interface = "sata"
+
+  rtc_time_base        = "UTC"
+  guest_additions_mode = "disable"
+  vboxmanage = [
+    ["modifyvm", "{{.Name}}", "--memory", "4096"],
+    ["modifyvm", "{{.Name}}", "--cpus", "4"],
+    #["modifyvm", "{{.Name}}", "--nat-localhostreachable1", "on"], # Uncomment if you are using VirtualBox 7.0.0 or newer
+  ]
+}
+
+source "virtualbox-ovf" "goSDN" {
+
+  source_path            = "output-baseimage/goSDN VM.ovf"
+  ssh_username           = "gosdn"
+  ssh_password           = "gosdn"
+  ssh_timeout            = "10m"
+  ssh_handshake_attempts = 1000
+  shutdown_command       = "echo 'packer' | sudo -S shutdown -P now"
+
+  vm_name = "goSDN VM"
+
+  guest_additions_mode = "disable"
+
+  export_opts = [
+    "--manifest",
+    "--vsys", "0",
+    "--description", "goSDN VM for education",
+    "--version", "0.1"
+  ]
+  format           = "ova"
+  output_directory = "VM"
+}
+
+build {
+  sources = ["sources.virtualbox-iso.baseimage"]
+  provisioner "shell" {
+    script = "scripts/setup.sh"
+  }
+}
+
+build {
+  sources = ["sources.virtualbox-ovf.goSDN"]
+  provisioner "shell" {
+    script = "scripts/gosdn_clone.sh"
+  }
+}
+
+

lab-vm/vm-with-packer/http/user-data

+#cloud-config
+autoinstall:
+  version: 1
+  locale: en_US.UTF-8
+  identity:
+    hostname: gosdn
+    username: gosdn
+    password: $6$onwrUGYtAm.NjFol$5eKUGV5QeprotZLE9RYhEl5OfrgP2.6XvQc9Q0Qlie4zze4RVymeHyMBdMvd.6/zULwm5uuaKwDxmZE1X9u.q.
+    
+  keyboard:
+    layout: de
+  storage:
+    layout:
+      name: direct
+  ssh:
+    install-server: true
+    allow-pw: yes
+  drivers:
+    install: true
+  packages: 
+    - server^
+    - standard^
+    - make
+    - curl
+    - ca-certificates
+    - curl
+    - gnupg
+    - zip
+  timezone: Europe/Berlin
+  updates: security
+
+  late-commands:
+    - echo "gosdn ALL=(ALL:ALL) NOPASSWD:ALL" > /target/etc/sudoers.d/gosdn && chmod 0440 /target/etc/sudoers.d/gosdn

lab-vm/vm-with-packer/pipeline.sh

+#!/bin/bash
+set -e # abort if there is an issue with any build
+packer init .
+packer build -only='virtualbox-iso.baseimage' .
+packer build -only='virtualbox-ovf.goSDN' .
+rm -r output-baseimage

lab-vm/vm-with-packer/scripts/gosdn_clone.sh

+#!/bin/bash
+
+source /home/gosdn/.profile
+# Clone goSDN
+git clone https://code.fbi.h-da.de/danet/gosdn.git
+cd ~/gosdn
+git submodule update --init --recursive
+
+# Build goSDN
+make build
+
+# Clone gNMI-Target
+cd ~/
+git clone --branch develop https://code.fbi.h-da.de/danet/gnmi-target.git
+cd gnmi-target
+
+# Build gNMI-Target
+docker build -f target.Dockerfile .
+make container

lab-vm/vm-with-packer/scripts/setup.sh

+#!/bin/bash
+
+# Update packages
+sudo apt update
+sudo apt upgrade -y
+
+# Install go
+wget https://go.dev/dl/go1.20.3.linux-amd64.tar.gz
+sudo tar -C /usr/local/ -xzf go1.20.3.linux-amd64.tar.gz
+rm go1.20.3.linux-amd64.tar.gz
+echo "export PATH=$PATH:/usr/local/go/bin" >>/home/gosdn/.profile
+echo "export GOPATH=$HOME/go " >>/home/gosdn/.profile
+source /home/gosdn/.profile
+
+# Add Docker keyring
+sudo install -m 0755 -d /etc/apt/keyrings
+curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
+sudo chmod a+r /etc/apt/keyrings/docker.gpg
+
+# Add Docker Repository
+echo "deb [arch="$(dpkg --print-architecture)" signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu "$(. /etc/os-release && echo "$VERSION_CODENAME")" stable" | sudo tee /etc/apt/sources.list.d/docker.list >/dev/null
+
+# Install Docker
+sudo apt update
+sudo apt install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
+
+# Add gosdn User to Docker Group
+sudo usermod -aG docker gosdn
+
+# Install Containerlab
+bash -c "$(curl -sL https://get.containerlab.dev)"

lab-vm/vm-with-vagrant/Vagrantfile

+# encoding: utf-8
+# -*- mode: ruby -*-
+# vi: set ft=ruby :
+# Box / OS
+VERSION_CODENAME = 'jammy'
+ARCHITECTURE = '64'
+VAGRANT_BOX = 'ubuntu/' + VERSION_CODENAME + ARCHITECTURE
+# Memorable name for your
+VM_NAME = 'gosdn-vm'
+# VM User — 'vagrant' by default
+VM_USER = 'vagrant'
+
+Vagrant.configure(2) do |config|
+  # Vagrant box from Hashicorp
+  config.vm.box = VAGRANT_BOX
+  # Actual machine name
+  config.vm.hostname = VM_NAME
+  # Set VM name in Virtualbox
+  config.vm.provider "virtualbox" do |v|
+    v.name = VM_NAME
+    v.memory = 4096
+  end
+
+  #DHCP — comment this out if planning on using NAT instead
+  config.vm.network "private_network", type: "dhcp"
+
+  # Install git, go, docker, containerlab and download repositories
+  config.vm.provision "shell", inline: <<-SHELL
+    apt-get update
+    apt-get upgrade -y
+    apt-get install -y \
+    ca-certificates \
+    curl \
+    git \
+    gnupg \
+    make \
+    zip
+    apt-get autoremove -y
+    ####### installing go #######
+    wget https://go.dev/dl/go1.20.3.linux-amd64.tar.gz
+    rm -rf /usr/local/go && sudo tar -C /usr/local -xzf go1.20.3.linux-amd64.tar.gz
+    rm go1.20.3.linux-amd64.tar.gz
+    echo 'export PATH=$PATH:/usr/local/go/bin' >> /home/vagrant/.profile
+    echo 'export GOPATH=$HOME/go' >> /home/vagrant/.profile
+    source /home/vagrant/.profile
+    ####### installing docker #######
+    install -m 0755 -d /etc/apt/keyrings
+    curl -fsSL https://download.docker.com/linux/ubuntu/gpg | gpg --dearmor -o /etc/apt/keyrings/docker.gpg
+    chmod a+r /etc/apt/keyrings/docker.gpg
+    echo \
+    "deb [arch="$(dpkg --print-architecture)" signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \
+    "$(. /etc/os-release && echo "$VERSION_CODENAME")" stable" | \
+    tee /etc/apt/sources.list.d/docker.list > /dev/null
+    apt-get update
+    apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin -y
+    usermod -aG docker vagrant
+    ####### installing containerlab #######
+    bash -c "$(curl -sL https://get.containerlab.dev)"
+    ####### get gosdn repository #######
+    su - vagrant -c "git clone https://code.fbi.h-da.de/danet/gosdn.git"
+    cd gosdn
+    su - vagrant -c "git submodule update --init --recursive"
+    su - vagrant -c "cd gosdn && make build"
+    ####### get gnmi target repository #######
+    cd ..
+    su - vagrant -c "git clone --branch develop https://code.fbi.h-da.de/danet/gnmi-target.git"
+    su - vagrant -c "cd gnmi-target && docker build -f target.Dockerfile ."
+    su - vagrant -c "cd gnmi-target && make container"
+  SHELL
+end
\ No newline at end of file