05-implementation.md

Implementation Aspects of the goSDN Controller

Why we do this in go

Because it rocks, but let's see afterwards what can be written here.

Storing Information

Section XXX (Conceptual Design of a SDN Controller as Network Supervisor) discusses the need to store information about for element inventories and topology inventories.

Element Inventories

Storing information about network elements and their properties is a relative static process, at least when one considers potential changes over time. Typically such network elements are added to a network and they will remain in the network for a longer time, i.e., multiple minutes or even longer.

Topology Inventory

Every network has one given physical topology (G_physical ) and on top of this at least one logical topology (G_logical1). There may be multiple logical topologies (G_n+1) on top logical topologies (G_n), i.e., a recursion. Such logical topologies (G_n+1) can again have other logical topologies as recursion or other logical topologies in parallel.

A topology consists out of interfaces, which are attached to their respective network elements, and links between these interfaces.

Mathematically, such a topology can be described as a directed graph, whereas the interfaces of the network elements are the nodes and the links are the edges.

G_physical ist a superset of G_logical1.

The topology inventory has to store the particular graph for any topology and also the connections between the different levels of topologies. For instance, the G_logical1 is linked to G_physical. (needs to be clear if changes in n-1 graph has impact on n graph).

For further study at this point: Which type of database and implementation of databases should be used to store the different topology graphs and their pontential dependencies? How should the interface between gosdn and this database look like?

Here is an attempt to describe the above text in a graphical reprensetation (kinda of...not perfect yet):

graph TB

  SubGraph1 --> SubGraph1Flow
  subgraph "G_logical1"
  SubGraph1Flow(Logical Net)
  Node1_l1[Node1_l1] <--> Node2_l1[Node2_l1] <--> Node3_l1[Node3_l1] <--> Node4_l1[Node4_l1] <--> Node5_l1[Node5_l1] <--> Node1_l1[Node1_l1]
  end

  subgraph "G_physical"
  Node1[Node 1] <--> Node2[Node 2] <--> Node3[Node 3]
  Node4[Node 4] <--> Node2[Node 2] <--> Node5[Node 5]

  Net_physical[Net_physical] --> SubGraph1[Reference to G_logical1]

end

Potential other Inventories

There may be the potential need to store information beyond pure topologies, actually about network flows, i.e., information about a group of packets belonging together.

neo4j

Due to the fact that network topologies, with all their elements and connections, can be represented well by a graph, the choice of a graph database for persistence was obvious. After some initial experiments with RedisGraph, neo4j was chosen, because neo4j allows the use of multiple labels (for nodes as well as edges) and offers a wider range of plugins.

The current implementation offers the possibility to persist different network elements and their physical topology. It became clear that within the graph database one has to move away from the basic idea of different independent graphs (topologies) and rather see the whole construct as a single huge graph with a multitude of relations.

The following figure shows our first idea of a persistence of network topologies with neo4j.

graph TD
subgraph "representation in Database"
PND[PND 1]
A --> |belongs to| PND
B --> |belongs to| PND
C --> |belongs to| PND
D --> |belongs to| PND
E --> |belongs to| PND

A[Node 1] --> |physical| B[Node 2]
D[Node 4] --> |physical| B
B --> |physical| C[Node 3]
B --> |physical| E[Node 5]

A --> |logical1| B
B --> |logical1| C
C --> |logical1| D
D --> |logical1| E
E --> |logical1| A
end

The basic idea is to assign the different network elements to a specific Principal Network Domain (PND). The different topologies are represented by a neo4j relationship between the network elements that are stored as neo4j nodes. However, with this current variant it is not possible, as required in Topology Inventory, to represent topologies that are hierarchically interdependent, since neo4j does not allow relations to be stored as properties (as described here

For the reason mentioned above, a more complex idea for persistence is available for the further development, which hopefully allows us to persist and map network elements, PNDs and topologies with all their hirarchical dependencies.

The following figure tries to visualize this idea. The main difference is, that for the different topologies separate nodes are created, to which so-called links belong. The links themselves form a connection between the respective network elements. A link can have several layer protocols, like OTUCN, ODUCN etc.

graph TD
subgraph "dependencies of topologies"
    logical1 -->|related_to| physical
    logical5 -->|related_to| physical
    logical3 -->|related_to| logical1
end

subgraph "every node belongs to a specific PND"
  Node1 -->|belongs_to| PND
  Node2 -->|belongs_to| PND
  Node3 -->|belongs_to| PND
  Node4 -->|belongs_to| PND
  Node5 -->|belongs_to| PND
end

subgraph "relationship between nodes (nodes can be linked by 0...n links)"
  lp2[link_physical]
  lp3[link_physical]
  lp4[link_physical]
  lp5[link_logical1]
  lp2 --> |connects| Node4
  lp2 --> |connects| Node2
  lp3 --> |connects| Node2
  lp3 --> |connects| Node3
  lp4 --> |connects| Node2
  lp4 --> |connects| Node5
  lp5 --> |connects| Node1
  lp5 --> |connects| Node2
end

subgraph "links are part of a topology"
lp1[link_physical]
lp1 --> |connects| Node1
lp1 --> |connects| Node2
lp1 --> |part_of| physical
end

subgraph "links can contain 1...n layers"
lp2 --> |contains| ODUH
lp2 --> |contains| OTUCN
lp2 --> |contains| ODUCN
end

The above idea is not yet approved and there are still open questions.

• Is there a better solution for the assumption that there are several different physical connections between the same nodes than separate link nodes between them?
• Can topologies run over different PNDs -> membership to different PNDs?
• Where can we benefit from using different layers? (e.g. possible saving of unnecessary relations between nodes)
• ...

YANG to code

The base of the development of goSDN are YANG modules. The RESTful API used for RESTCONF is defined in an OpenAPI 2.0 file. This API documentation is generated from the YANG module. The YANG module description is also used to generate code stubs for the goSDN RESTCONF client.

\includegraphics{gfx/yang-schematics.pdf}

YANG

YANG defines an abstract network interface. It is the foundation of the RESTCONF protocol. Several code generators exist to generate code stubs from a given definition.

OpenAPI

OpenAPI - formerly known as Swagger - is a framework that defines RESTful APIs. We use OenAPI documentations to define the RESTCONF server implementation of the cocsn YANG modules.

Toolchain

We use 3 different tools for the code generation workflow. For the RESTCONF server yanger is used to generate the OpenAPI documentation from the YANG file. go-swagger is used to generate a RESTCONF server with stubs for the REST calls.

The RESTCONF client stubs used by goSDN are generated from YANG files using YGOT.

Dependencies

For now we can only use the OpenAPI 2.0 standard. This is because go-swagger does not support OpenAPI 3.0 specifications yet.

Storing Information

This section keeps by now some loose thoughts about what information has to be stored how and where.

There seem to be two classes of information to be stored in the controller:

• short-living information, such as, current configured network flows or obtained network configuration out of use case #1 (closed) (CoCSN)
• long-time information, such as, information about principle network domains, elements in such a domain if directly learned from SBI, etc

Long-time information should be persistenly stored in the database and survive reboots of goSDN etc. Short-Living information doesn't have to survive reboots of goSDN

Some more details for implementation for the database(s)

We define the principle network domain (PND) and each piece of information of any PND has to be stored in relation the particular PND.

Specification of a PND:

• Human readable name of PND
• Textual description for further information
• Set of supported Southbound-Interfaces, e.g., RESTCONF, TAPI, OpenFlow etc
• Physical Inventory Network Elements, hosts and links, pontentially only the SBI SDN controller

A PND entry must be explicitly generated, though some information can be automatically be generated, e.g., the physical inventory for use-case #1 (closed) (CoCSN) would mean that the information about the SBI domain specific SDN controller is entered.