Skip to content
Snippets Groups Projects
Commit c97e36ef authored by Malte Bauch's avatar Malte Bauch
Browse files

updated readme for database

parent e970c0de
No related branches found
No related tags found
3 merge requests!90Develop,!62Resolve "Update database in order to achieve the long-term goals of the project",!53V.0.1.0 Codename Threadbare
Pipeline #53006 passed
Stage: test
Stage: documentation
This commit is part of merge request !53. Comments created here will be created in the context of that merge request.
Hide whitespace changes
Inline Side-by-side
documentation/design/05-implementation.md
+ 123
− 74
View file @ c97e36ef
...@@ -71,101 +71,150 @@ There may be the potential need to store information beyond pure topologies, ...@@ -71,101 +71,150 @@ There may be the potential need to store information beyond pure topologies,
actually about network flows, i.e., information about a group of packets actually about network flows, i.e., information about a group of packets
belonging together. belonging together.
### neo4j ## Database
Due to the fact that network topologies, with all their elements and connections, A database will be used for the management and persistence of network
can be represented well by a graph, the choice of a graph database for persistence was obvious. topologies and their associated elements within goSDN.
After some initial experiments with RedisGraph, neo4j was chosen,
because neo4j allows the use of multiple labels (for nodes as well as edges) Since network topologies are often depicted as graphs, it was obvious to stick
and offers a wider range of plugins. to this concept and, also due to their increasing popularity, to use a graph
database. After a more intensive examination of graph databases it was found
that they (with their labels, nodes, relations and properties) are well suited
for a representation of network topologies.
The first basic idea was to create different single graphs representing the
different network topologies and label each node and edge to ensure a clear
assignment to a topology.
This would mean that nodes and edges of a graph have 1...n labels.
Therefore if you want to display a simple network topology in a graph, you can
display the different network elements as individual nodes and the edges between
network elements as their respective connections, such as Ethernet.
This works with both physical and logical topologies, which are described in
more detail [here](#topology-inventory).
So a simple topology in a graph database could look like shown below.
The current implementation offers the possibility to persist different network elements ```mermaid
and their physical topology. It became clear that within the graph database one has to graph TD
move away from the basic idea of different independent graphs (topologies) and rather see A[Node 1 - Label: 'Host,physical'] -->|Ethernet - Label: 'physical'| B[Node 2 - Label: 'Hub,physical']
the whole construct as a single huge graph with a multitude of relations. C[Node 3 - Label: 'Host,physical'] -->|Ethernet - Label: 'physical'| B
B -->|Ethernet - Label: 'physical'| D[Node 4 - Label: 'Host,physical']
B -->|Ethernet - Label: 'physical'| E[Node 5 - Label: 'Host,physical']
```
The following figure shows our first idea of a persistence of network topologies with neo4j. For this purpose some experiments with the [Redis](https://redis.io/)-Database
module [`RedisGraph`](https://oss.redislabs.com/redisgraph/) were carried out
first. The basic implementation was possible, but the function of assigning
several labels to one node/edge is missing (originally we considered this to be
indispensable especially to map different topologies).
For this reason we looked around for an alternative and with
[neo4j](https://neo4j.com/) we found a graph database, which gives us the
possibility to label nodes and edges with a multitude of labels and offers a
wide range of additional plugins such as [apoc](https://neo4j.com/labs/apoc/).
### neo4j
TODO: add a little description for neo4j in general
#### Implementation With neo4j
The current implementation offers the possibility to persist different network
elements (e.g. devices, interfaces...) and their physical topology and mainly
serves to represent the prototypical dataflow of goSDN to the database.
The following figure shows our first idea of a persistence of network
topologies with neo4j (to save space, only the labels were included).
```mermaid ```mermaid
graph TD graph TD
subgraph "representation in Database" PND[PND 1]
PND[PND 1] A --> |belongs to| PND
A --> |belongs to| PND B --> |belongs to| PND
B --> |belongs to| PND C --> |belongs to| PND
C --> |belongs to| PND D --> |belongs to| PND
D --> |belongs to| PND E --> |belongs to| PND
E --> |belongs to| PND
A[Label: 'Host,physical,logical1'] --> |Label: 'physical'| B[Label: 'Hub,physical,logical1']
A[Node 1] --> |physical| B[Node 2] D[Label: 'Host,physical,logical1'] --> |Label: 'physical'| B
D[Node 4] --> |physical| B B --> |Label: 'physical'| C[Label: 'Host,physical,logical1']
B --> |physical| C[Node 3] B --> |Label: 'physical'| E[Label: 'Host,physical,logical1']
B --> |physical| E[Node 5]
A --> |Label: 'logical1'| B
A --> |logical1| B B --> |Label: 'logical1'| C
B --> |logical1| C C --> |Label: 'logical1'| D
C --> |logical1| D D --> |Label: 'logical1'| E
D --> |logical1| E E --> |Label: 'logical1'| A
E --> |logical1| A
end
``` ```
The basic idea is to assign the different network elements to a specific Principal Network Domain (PND). The basic idea is to assign the different network elements to a specific
The different topologies are represented by a neo4j relationship between the network elements that are Principal Network Domain (PND). The different topologies are represented by a
stored as neo4j nodes. However, with this current variant it is not possible, as required in neo4j relationship between the network elements that are stored as neo4j nodes.
However, with this current variant it is not possible, as required in
[Topology Inventory](#topology-inventory), to represent topologies that are hierarchically [Topology Inventory](#topology-inventory), to represent topologies that are hierarchically
interdependent, since neo4j does not allow relations to be stored as properties (as described [here](https://neo4j.com/docs/cypher-manual/current/syntax/values/#structural-types) interdependent, since neo4j does not allow relations to be stored as properties
(as described [here](https://neo4j.com/docs/cypher-manual/current/syntax/values/#structural-types)).
For the reason mentioned above, a more complex idea for persistence is available for the further development, which hopefully allows us to persist and map network elements, PNDs and topologies with all their hirarchical dependencies. Furthermore, multiple links between the same nodes which belong to the same
topology are difficult to represent, since this model only provides a single
link between nodes of a certain topology.
The following figure tries to visualize this idea. The main difference is, that for the different topologies separate nodes are created, to which so-called links belong. The links themselves form a connection between the respective network elements. A link can have several layer protocols, like OTUCN, ODUCN etc. For the reason mentioned above, a more complex idea for persistence is available
for the further development, which hopefully allows us to persist and map
network elements, PNDs and topologies with all their hirarchical dependencies.
The following figure tries to visualize this idea.
```mermaid ```mermaid
graph TD graph TD
subgraph "dependencies of topologies" subgraph "dependencies of topologies"
logical1 -->|related_to| physical logical1 -->|related_to| physical
logical5 -->|related_to| physical logical5 -->|related_to| physical
logical3 -->|related_to| logical1 logical3 -->|related_to| logical1
end end
subgraph "every node belongs to a specific PND" subgraph "every node belongs to a specific PND"
Node1 -->|belongs_to| PND Node1 -->|belongs_to| PND
Node2 -->|belongs_to| PND Node2 -->|belongs_to| PND
Node3 -->|belongs_to| PND Node3 -->|belongs_to| PND
Node4 -->|belongs_to| PND Node4 -->|belongs_to| PND
Node5 -->|belongs_to| PND Node5 -->|belongs_to| PND
end end
subgraph "relationship between nodes (nodes can be linked by 0...n links)" subgraph "relationship between nodes (nodes can be linked by 0...n links)"
lp2[link_physical] lp2[link_physical]
lp3[link_physical] lp3[link_physical]
lp4[link_physical] lp4[link_physical]
lp5[link_logical1] lp5[link_logical1]
lp2 --> |connects| Node4 lp2 --> |connects| Node4
lp2 --> |connects| Node2 lp2 --> |connects| Node2
lp3 --> |connects| Node2 lp3 --> |connects| Node2
lp3 --> |connects| Node3 lp3 --> |connects| Node3
lp4 --> |connects| Node2 lp4 --> |connects| Node2
lp4 --> |connects| Node5 lp4 --> |connects| Node5
lp5 --> |connects| Node1 lp5 --> |connects| Node1
lp5 --> |connects| Node2 lp5 --> |connects| Node2
end end
subgraph "links are part of a topology" subgraph "links are part of a topology"
lp1[link_physical] lp1[link_physical]
lp1 --> |connects| Node1 lp1 --> |connects| Node1
lp1 --> |connects| Node2 lp1 --> |connects| Node2
lp1 --> |part_of| physical lp1 --> |part_of| physical
end end
subgraph "links can contain 1...n layers" subgraph "links can contain 1...n layers"
lp2 --> |contains| ODUH lp2 --> |contains| ODUH
lp2 --> |contains| OTUCN lp2 --> |contains| OTUCN
lp2 --> |contains| ODUCN lp2 --> |contains| ODUCN
end end
``` ```
The basic structure explained in the upper part remains the same.
However, the relations, which previously served as links between the respective
nodes, now become **separate nodes**. These nodes now act as links between the
respective network elements and are part of a network topology (which itself
is represented as a separate node in the graph). By this change, network
topologies can now be interdependent. Furthermore, as can be seen in the figure
above, you can add additional nodes to the link nodes by using this scheme.
So a physical link between two nodes could e.g. **contain** several cables.
All other information can be stored in the properties of the respective nodes/edges.
The above idea is not yet approved and there are still open questions. The above idea is not yet approved and there are still open questions.
- Is there a better solution for the assumption that there are several different physical connections between the same nodes than separate link nodes between them? - Is there a better solution for the assumption that there are several different physical connections between the same nodes than separate link nodes between them?
- Can topologies run over different PNDs -> membership to different PNDs? - Can topologies run over different PNDs -> membership to different PNDs?
- Where can we benefit from using different layers? (e.g. possible saving of unnecessary relations between nodes) - Where can we benefit from using different layers? (e.g. possible saving of unnecessary relations between nodes)
- Do the sdn controllers provide us with the necessary information to map the topologies in this way?
- ... - ...
## YANG to code ## YANG to code
......
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment