diff --git a/documentation/design/05-implementation.md b/documentation/design/05-implementation.md index 15f7f7617518d734fcc16bf664d4c85d904c4f1c..6a1c3aaee2b0d9703e7e6cf76358609180f522e6 100644 --- a/documentation/design/05-implementation.md +++ b/documentation/design/05-implementation.md @@ -71,101 +71,150 @@ There may be the potential need to store information beyond pure topologies, actually about network flows, i.e., information about a group of packets belonging together. -### neo4j -Due to the fact that network topologies, with all their elements and connections, -can be represented well by a graph, the choice of a graph database for persistence was obvious. -After some initial experiments with RedisGraph, neo4j was chosen, -because neo4j allows the use of multiple labels (for nodes as well as edges) -and offers a wider range of plugins. +## Database +A database will be used for the management and persistence of network +topologies and their associated elements within goSDN. + +Since network topologies are often depicted as graphs, it was obvious to stick +to this concept and, also due to their increasing popularity, to use a graph +database. After a more intensive examination of graph databases it was found +that they (with their labels, nodes, relations and properties) are well suited +for a representation of network topologies. + +The first basic idea was to create different single graphs representing the +different network topologies and label each node and edge to ensure a clear +assignment to a topology. +This would mean that nodes and edges of a graph have 1...n labels. +Therefore if you want to display a simple network topology in a graph, you can +display the different network elements as individual nodes and the edges between +network elements as their respective connections, such as Ethernet. +This works with both physical and logical topologies, which are described in +more detail [here](#topology-inventory). +So a simple topology in a graph database could look like shown below. -The current implementation offers the possibility to persist different network elements -and their physical topology. It became clear that within the graph database one has to -move away from the basic idea of different independent graphs (topologies) and rather see -the whole construct as a single huge graph with a multitude of relations. +```mermaid +graph TD +A[Node 1 - Label: 'Host,physical'] -->|Ethernet - Label: 'physical'| B[Node 2 - Label: 'Hub,physical'] +C[Node 3 - Label: 'Host,physical'] -->|Ethernet - Label: 'physical'| B +B -->|Ethernet - Label: 'physical'| D[Node 4 - Label: 'Host,physical'] +B -->|Ethernet - Label: 'physical'| E[Node 5 - Label: 'Host,physical'] +``` -The following figure shows our first idea of a persistence of network topologies with neo4j. +For this purpose some experiments with the [Redis](https://redis.io/)-Database +module [`RedisGraph`](https://oss.redislabs.com/redisgraph/) were carried out +first. The basic implementation was possible, but the function of assigning +several labels to one node/edge is missing (originally we considered this to be +indispensable especially to map different topologies). +For this reason we looked around for an alternative and with +[neo4j](https://neo4j.com/) we found a graph database, which gives us the +possibility to label nodes and edges with a multitude of labels and offers a +wide range of additional plugins such as [apoc](https://neo4j.com/labs/apoc/). +### neo4j +TODO: add a little description for neo4j in general + +#### Implementation With neo4j +The current implementation offers the possibility to persist different network +elements (e.g. devices, interfaces...) and their physical topology and mainly +serves to represent the prototypical dataflow of goSDN to the database. +The following figure shows our first idea of a persistence of network +topologies with neo4j (to save space, only the labels were included). ```mermaid graph TD -subgraph "representation in Database" -PND[PND 1] -A --> |belongs to| PND -B --> |belongs to| PND -C --> |belongs to| PND -D --> |belongs to| PND -E --> |belongs to| PND - -A[Node 1] --> |physical| B[Node 2] -D[Node 4] --> |physical| B -B --> |physical| C[Node 3] -B --> |physical| E[Node 5] - -A --> |logical1| B -B --> |logical1| C -C --> |logical1| D -D --> |logical1| E -E --> |logical1| A -end + PND[PND 1] + A --> |belongs to| PND + B --> |belongs to| PND + C --> |belongs to| PND + D --> |belongs to| PND + E --> |belongs to| PND + + A[Label: 'Host,physical,logical1'] --> |Label: 'physical'| B[Label: 'Hub,physical,logical1'] + D[Label: 'Host,physical,logical1'] --> |Label: 'physical'| B + B --> |Label: 'physical'| C[Label: 'Host,physical,logical1'] + B --> |Label: 'physical'| E[Label: 'Host,physical,logical1'] + + A --> |Label: 'logical1'| B + B --> |Label: 'logical1'| C + C --> |Label: 'logical1'| D + D --> |Label: 'logical1'| E + E --> |Label: 'logical1'| A ``` -The basic idea is to assign the different network elements to a specific Principal Network Domain (PND). -The different topologies are represented by a neo4j relationship between the network elements that are -stored as neo4j nodes. However, with this current variant it is not possible, as required in +The basic idea is to assign the different network elements to a specific +Principal Network Domain (PND). The different topologies are represented by a +neo4j relationship between the network elements that are stored as neo4j nodes. +However, with this current variant it is not possible, as required in [Topology Inventory](#topology-inventory), to represent topologies that are hierarchically -interdependent, since neo4j does not allow relations to be stored as properties (as described [here](https://neo4j.com/docs/cypher-manual/current/syntax/values/#structural-types) - -For the reason mentioned above, a more complex idea for persistence is available for the further development, which hopefully allows us to persist and map network elements, PNDs and topologies with all their hirarchical dependencies. +interdependent, since neo4j does not allow relations to be stored as properties +(as described [here](https://neo4j.com/docs/cypher-manual/current/syntax/values/#structural-types)). +Furthermore, multiple links between the same nodes which belong to the same +topology are difficult to represent, since this model only provides a single +link between nodes of a certain topology. -The following figure tries to visualize this idea. The main difference is, that for the different topologies separate nodes are created, to which so-called links belong. The links themselves form a connection between the respective network elements. A link can have several layer protocols, like OTUCN, ODUCN etc. +For the reason mentioned above, a more complex idea for persistence is available +for the further development, which hopefully allows us to persist and map +network elements, PNDs and topologies with all their hirarchical dependencies. +The following figure tries to visualize this idea. ```mermaid graph TD -subgraph "dependencies of topologies" - logical1 -->|related_to| physical - logical5 -->|related_to| physical - logical3 -->|related_to| logical1 -end + subgraph "dependencies of topologies" + logical1 -->|related_to| physical + logical5 -->|related_to| physical + logical3 -->|related_to| logical1 + end -subgraph "every node belongs to a specific PND" - Node1 -->|belongs_to| PND - Node2 -->|belongs_to| PND - Node3 -->|belongs_to| PND - Node4 -->|belongs_to| PND - Node5 -->|belongs_to| PND -end + subgraph "every node belongs to a specific PND" + Node1 -->|belongs_to| PND + Node2 -->|belongs_to| PND + Node3 -->|belongs_to| PND + Node4 -->|belongs_to| PND + Node5 -->|belongs_to| PND + end -subgraph "relationship between nodes (nodes can be linked by 0...n links)" - lp2[link_physical] - lp3[link_physical] - lp4[link_physical] - lp5[link_logical1] - lp2 --> |connects| Node4 - lp2 --> |connects| Node2 - lp3 --> |connects| Node2 - lp3 --> |connects| Node3 - lp4 --> |connects| Node2 - lp4 --> |connects| Node5 - lp5 --> |connects| Node1 - lp5 --> |connects| Node2 -end + subgraph "relationship between nodes (nodes can be linked by 0...n links)" + lp2[link_physical] + lp3[link_physical] + lp4[link_physical] + lp5[link_logical1] + lp2 --> |connects| Node4 + lp2 --> |connects| Node2 + lp3 --> |connects| Node2 + lp3 --> |connects| Node3 + lp4 --> |connects| Node2 + lp4 --> |connects| Node5 + lp5 --> |connects| Node1 + lp5 --> |connects| Node2 + end -subgraph "links are part of a topology" -lp1[link_physical] -lp1 --> |connects| Node1 -lp1 --> |connects| Node2 -lp1 --> |part_of| physical -end + subgraph "links are part of a topology" + lp1[link_physical] + lp1 --> |connects| Node1 + lp1 --> |connects| Node2 + lp1 --> |part_of| physical + end -subgraph "links can contain 1...n layers" -lp2 --> |contains| ODUH -lp2 --> |contains| OTUCN -lp2 --> |contains| ODUCN -end + subgraph "links can contain 1...n layers" + lp2 --> |contains| ODUH + lp2 --> |contains| OTUCN + lp2 --> |contains| ODUCN + end ``` +The basic structure explained in the upper part remains the same. +However, the relations, which previously served as links between the respective +nodes, now become **separate nodes**. These nodes now act as links between the +respective network elements and are part of a network topology (which itself +is represented as a separate node in the graph). By this change, network +topologies can now be interdependent. Furthermore, as can be seen in the figure +above, you can add additional nodes to the link nodes by using this scheme. +So a physical link between two nodes could e.g. **contain** several cables. +All other information can be stored in the properties of the respective nodes/edges. + The above idea is not yet approved and there are still open questions. - Is there a better solution for the assumption that there are several different physical connections between the same nodes than separate link nodes between them? - Can topologies run over different PNDs -> membership to different PNDs? - Where can we benefit from using different layers? (e.g. possible saving of unnecessary relations between nodes) +- Do the sdn controllers provide us with the necessary information to map the topologies in this way? - ... ## YANG to code