Skip to content
Snippets Groups Projects
Commit c290dfd8 authored by S.H.'s avatar S.H.
Browse files

Merge branch 'master' into heiss_bachelor_thesis

merge it so I am up to date with master an my commits can more easily be
merged back
parents 2a6203af 0590b112
Branches
No related tags found
No related merge requests found
Pipeline #261792 failed
code-quality:
image: ${CI_DEPENDENCY_PROXY_GROUP_IMAGE_PREFIX}/golangci/golangci-lint:v1.62.2-alpine
image: ${CI_DEPENDENCY_PROXY_GROUP_IMAGE_PREFIX}/golangci/golangci-lint:v1.63.4-alpine
stage: analyze
script:
# writes golangci-lint output to gl-code-quality-report.json
......
build-mkdocs:
image: ${CI_DEPENDENCY_PROXY_GROUP_IMAGE_PREFIX}/python:3.13.1-slim-bookworm
image: ${CI_DEPENDENCY_PROXY_GROUP_IMAGE_PREFIX}/python:3.13.2-slim-bookworm
stage: build
before_script:
- pip install mkdocs-material
......@@ -13,7 +13,7 @@ build-mkdocs:
- if: $CI_COMMIT_REF_PROTECTED == "true"
.pages-options: &pages-options
image: ${CI_DEPENDENCY_PROXY_GROUP_IMAGE_PREFIX}/python:3.13.1-slim-bookworm
image: ${CI_DEPENDENCY_PROXY_GROUP_IMAGE_PREFIX}/python:3.13.2-slim-bookworm
stage: deploy
script:
- mv mkdocs-built public
......
renovate:
stage: tools
image: renovate/renovate:39.48.1
image: renovate/renovate:39.165.1
variables:
LOG_LEVEL: debug
......
......@@ -14,13 +14,13 @@ output:
- format: colored-line-number
print-issued-lines: true
print-linter-name: true
uniq-by-line: true
path-prefix: ""
issues:
exclude-use-default: false
max-issues-per-linter: 0
max-same-issues: 0
uniq-by-line: true
exclude-files:
- http.go
# directories to be ignored by linters
......
......@@ -18,7 +18,7 @@ PLUGIN_NAME= bundled_plugin.zip
# Tool Versions
GOTESTSUM_VERSION=v1.8.1
GOLANGCI_LINT_VERSION=v1.62.0
GOLANGCI_LINT_VERSION=v1.63.4
MOCKERY_VERSION=v2.20.0
YGOT_GENERATOR_VERSION=v0.27.0
YGOT_GENERATOR_GENERATOR_VERSION=v0.0.4
......
......@@ -20,7 +20,7 @@ You can find a detailed manual to install all necessary tools at our [Wiki](http
There you can also find some tutorials to get to know the SDN controller:
- [Lab00 education](https://code.fbi.h-da.de/danet/gosdn/-/wikis/Labs/Lab00-education), which is used in education
- [Lab00](https://code.fbi.h-da.de/danet/gosdn/-/wikis/Labs/Lab00), which uses servers with gNMI targets instead of Arista switch images (needs less storage than Lab01)
- [Lab01](https://code.fbi.h-da.de/danet/gosdn/-/wikis/Labs/Lab01), which uses servers with Arista switch images (needs less storage than Lab01)
- [Lab01](https://code.fbi.h-da.de/danet/gosdn/-/wikis/Labs/Lab01), which uses servers with Arista switch images
---
## Table of Contents
......@@ -72,6 +72,10 @@ A simple showcase how the controller can be addressed after
on them. They are currently unsupported.
- `controller` represents the `goSDN-controller`.
---
## Concepts
<div align="center" style="margin-bottom:3em; margin-top:4em">
<img src="documentation/figures/overview/updated_controller_architecture_overview.drawio.png" />
<p> Overview of the controller architecture</p>
......@@ -87,15 +91,27 @@ A simple showcase how the controller can be addressed after
<p> Detailed view of plugin mechanism</p>
</div>
---
## Concepts
### Nucleus
The `goSDN` controllers core - also called `nucleus` - is a lightweight library
that manages principal network domains and provides southbound interface
operations for managed network elements.
In addition, we provide a simple Northbound-API (gRPC) for the controller [right here](https://code.fbi.h-da.de/danet/gosdn/-/tree/master/controller/api).
### Principal Networking Domain (PND)
The PND is the single source of truth within a network. Its state is held and
maintained by the controller. Any configuration of an MNE has to be applied by
the PND.
### Managed Network Element (MNE)
Any network element directly configured by `goSDN`
### Northbound Interface (NBI)
The NBI is implemented using gRPC which means one can interact with the controller with any language that supports gRPC when using our own [proto definitions](https://code.fbi.h-da.de/danet/gosdn/-/tree/master/api/proto/gosdn).
In addition, we provide a simple abstraction layer of the Northbound-API to use with the controller in Go [right here](https://code.fbi.h-da.de/danet/gosdn/-/tree/master/controller/api).
To use the API, you can build a login method as is done in the `inventory-manager` in `utils.go`.
You log in, create a session context with the returned token, then you can simply make API calls with this context.
......@@ -120,7 +136,7 @@ func createContextWithAuthorization(sessionToken string) context.Context {
}
```
The code in your app:
An example on how the code in your app to register a MNE in the controller could look like:
```golang
import "code.fbi.h-da.de/danet/gosdn/controller/api"
......@@ -130,20 +146,11 @@ ctx := createContextWithAuthorization(sessionToken)
_, err := api.AddNetworkElement(i.sessionContext, i.controllerAddress, networkElement.Name, networkElement.UUID, &transportOptions, pluginUUID, pndUUID, []string{})
```
The gRPC services can also be reached using HTTP requests via the gRPC-Gateway. The fitting OpenAPI definitions can be found [here](https://code.fbi.h-da.de/danet/gosdn/-/tree/master/api/openapiv2?ref_type=heads).
The gRPC services can also be reached using HTTP requests via the gRPC-Gateway. The fitting OpenAPI definitions can be found [here](https://code.fbi.h-da.de/danet/gosdn/-/tree/master/api/openapiv2?ref_type=heads). Note, that this is experimental and tested less well. If you want to use the controller in secure mode which implies it's mandatory to login and provide the received token in other requests via the HTTP header with the key-value pair:
Note, that this is experimental and tested less well. If you want to use the controller in secure mode it is mandatory to login and provide the received token in other requests via the HTTP header with the key-value pair:
`"authorize: token"`.
### Principal Networking Domain (PND)
The PND is the single source of truth within a network. Its state is held and
maintained by the controller. Any configuration of an MNE has to be applied by
the PND.
### Managed Network Element (MNE)
Any network element directly configured by `goSDN`
---
## Launch goSDN Controller local
In this chapter, you learn how to launch the goSDN controller.
......@@ -152,11 +159,9 @@ Firstly, make sure that you're located in the root directory of goSDN.
`goSDN` provides a `Makefile` for all common use cases.
```sh
# build the application files for goSDN, cSBI orchestrator, gosdnc (CLI).
make build
# build the application files for goSDN, cSBI orchestrator (currently not in use), gosdnc (CLI).
# build the Dockerfiles for goSDN, cSBI orchestrator, gosdnc (CLI).
make containerize-all
make build
```
Depending on how you want to use the controller, you might need to edit the example config files and provide the path to this or a new config file.
......@@ -175,12 +180,20 @@ running `./gosdn` from the shell:
---
## Getting Started
## Getting Started using containers
If you want to use the [playground](#playground) you have to make sure you
have [containerlab](https://containerlab.dev/install/) installed on your
system.
It is possible to build all the docker containers using the following command from the root directory of the project:
```sh
# build the Docker containers for goSDN, cSBI orchestrator (currently not in use), gosdnc (CLI) as well as some of the provided example applications.
make containerize-all
```
Note that this command is included as a step when using any of the following enviroments like the playgrounds, labs etc. and therefore can be skipped.
### Playground
With the help of [containerlab](https://containerlab.dev/) we provide simple test environments to play around with.
......
......@@ -89,7 +89,13 @@ func TestAuth_Login(t *testing.T) {
wantErr: true,
validationErrors: []*validate.Violation{
{
FieldPath: stringToPointer("username"),
Field: &validate.FieldPath{
Elements: []*validate.FieldPathElement{
{
FieldName: stringToPointer("username"),
},
},
},
ConstraintId: stringToPointer("required"),
Message: stringToPointer("value is required"),
}},
......@@ -160,7 +166,13 @@ func TestAuth_Logout(t *testing.T) {
wantErr: true,
validationErrors: []*validate.Violation{
{
FieldPath: stringToPointer("username"),
Field: &validate.FieldPath{
Elements: []*validate.FieldPathElement{
{
FieldName: stringToPointer("username"),
},
},
},
ConstraintId: stringToPointer("required"),
Message: stringToPointer("value is required"),
}},
......
......@@ -84,10 +84,20 @@ func TestRole_CreateRoles(t *testing.T) {
wantErr: true,
validationErrors: []*validate.Violation{
{
FieldPath: stringToPointer("roles[0].name"),
Field: &validate.FieldPath{
Elements: []*validate.FieldPathElement{
{
FieldName: stringToPointer("roles"),
},
{
FieldName: stringToPointer("name"),
},
},
},
ConstraintId: stringToPointer("string.min_len"),
Message: stringToPointer("value length must be at least 3 characters"),
}},
},
},
},
{
name: "role with too short description should fail",
......@@ -105,7 +115,16 @@ func TestRole_CreateRoles(t *testing.T) {
want: &apb.CreateRolesResponse{},
wantErr: true,
validationErrors: []*validate.Violation{{
FieldPath: stringToPointer("roles[0].description"),
Field: &validate.FieldPath{
Elements: []*validate.FieldPathElement{
{
FieldName: stringToPointer("roles"),
},
{
FieldName: stringToPointer("description"),
},
},
},
ConstraintId: stringToPointer("string.min_len"),
Message: stringToPointer("value length must be at least 3 characters"),
}},
......@@ -181,7 +200,13 @@ func TestRole_GetRole(t *testing.T) {
wantErr: true,
validationErrors: []*validate.Violation{
{
FieldPath: stringToPointer("role_name"),
Field: &validate.FieldPath{
Elements: []*validate.FieldPathElement{
{
FieldName: stringToPointer("role_name"),
},
},
},
ConstraintId: stringToPointer("required"),
Message: stringToPointer("value is required"),
},
......@@ -355,7 +380,16 @@ func TestRole_UpdateRoles(t *testing.T) {
wantErr: true,
validationErrors: []*validate.Violation{
{
FieldPath: stringToPointer("roles[0].name"),
Field: &validate.FieldPath{
Elements: []*validate.FieldPathElement{
{
FieldName: stringToPointer("roles"),
},
{
FieldName: stringToPointer("name"),
},
},
},
ConstraintId: stringToPointer("string.min_len"),
Message: stringToPointer("value length must be at least 3 characters"),
},
......@@ -379,7 +413,16 @@ func TestRole_UpdateRoles(t *testing.T) {
wantErr: true,
validationErrors: []*validate.Violation{
{
FieldPath: stringToPointer("roles[0].description"),
Field: &validate.FieldPath{
Elements: []*validate.FieldPathElement{
{
FieldName: stringToPointer("roles"),
},
{
FieldName: stringToPointer("description"),
},
},
},
ConstraintId: stringToPointer("string.min_len"),
Message: stringToPointer("value length must be at least 3 characters"),
},
......@@ -456,12 +499,24 @@ func TestRole_DeletePermissionsForRole(t *testing.T) {
wantErr: true,
validationErrors: []*validate.Violation{
{
FieldPath: stringToPointer("role_name"),
Field: &validate.FieldPath{
Elements: []*validate.FieldPathElement{
{
FieldName: stringToPointer("role_name"),
},
},
},
ConstraintId: stringToPointer("required"),
Message: stringToPointer("value is required"),
},
{
FieldPath: stringToPointer("permissions_to_delete"),
Field: &validate.FieldPath{
Elements: []*validate.FieldPathElement{
{
FieldName: stringToPointer("permissions_to_delete"),
},
},
},
ConstraintId: stringToPointer("required"),
Message: stringToPointer("value is required"),
},
......
......@@ -285,27 +285,72 @@ func TestTopology_AddLink(t *testing.T) {
wantErr: true,
validationErrors: []*validate.Violation{
{
FieldPath: stringToPointer("link.name"),
Field: &validate.FieldPath{
Elements: []*validate.FieldPathElement{
{
FieldName: stringToPointer("link"),
},
{
FieldName: stringToPointer("name"),
},
},
},
ConstraintId: stringToPointer("string.min_len"),
Message: stringToPointer("value length must be at least 1 characters"),
},
{
FieldPath: stringToPointer("link.sourceNode"),
Field: &validate.FieldPath{
Elements: []*validate.FieldPathElement{
{
FieldName: stringToPointer("link"),
},
{
FieldName: stringToPointer("sourceNode"),
},
},
},
ConstraintId: stringToPointer("required"),
Message: stringToPointer("value is required"),
},
{
FieldPath: stringToPointer("link.targetNode"),
Field: &validate.FieldPath{
Elements: []*validate.FieldPathElement{
{
FieldName: stringToPointer("link"),
},
{
FieldName: stringToPointer("targetNode"),
},
},
},
ConstraintId: stringToPointer("required"),
Message: stringToPointer("value is required"),
},
{
FieldPath: stringToPointer("link.sourcePort"),
Field: &validate.FieldPath{
Elements: []*validate.FieldPathElement{
{
FieldName: stringToPointer("link"),
},
{
FieldName: stringToPointer("sourcePort"),
},
},
},
ConstraintId: stringToPointer("required"),
Message: stringToPointer("value is required"),
},
{
FieldPath: stringToPointer("link.targetPort"),
Field: &validate.FieldPath{
Elements: []*validate.FieldPathElement{
{
FieldName: stringToPointer("link"),
},
{
FieldName: stringToPointer("targetPort"),
},
},
},
ConstraintId: stringToPointer("required"),
Message: stringToPointer("value is required"),
},
......@@ -461,7 +506,13 @@ func TestTopology_DeleteLink(t *testing.T) {
wantErr: true,
validationErrors: []*validate.Violation{
{
FieldPath: stringToPointer("id"),
Field: &validate.FieldPath{
Elements: []*validate.FieldPathElement{
{
FieldName: stringToPointer("id"),
},
},
},
ConstraintId: stringToPointer("required"),
Message: stringToPointer("value is required"),
}},
......
......@@ -90,7 +90,16 @@ func TestUser_CreateUsers(t *testing.T) {
wantErr: true,
validationErrors: []*validate.Violation{
{
FieldPath: stringToPointer("user[0].password"),
Field: &validate.FieldPath{
Elements: []*validate.FieldPathElement{
{
FieldName: stringToPointer("user"),
},
{
FieldName: stringToPointer("password"),
},
},
},
ConstraintId: stringToPointer("string.min_len"),
Message: stringToPointer("value length must be at least 5 characters"),
}},
......@@ -116,7 +125,16 @@ func TestUser_CreateUsers(t *testing.T) {
wantErr: true,
validationErrors: []*validate.Violation{
{
FieldPath: stringToPointer("user[0].name"),
Field: &validate.FieldPath{
Elements: []*validate.FieldPathElement{
{
FieldName: stringToPointer("user"),
},
{
FieldName: stringToPointer("name"),
},
},
},
ConstraintId: stringToPointer("string.min_len"),
Message: stringToPointer("value length must be at least 3 characters"),
}},
......@@ -190,7 +208,13 @@ func TestUser_GetUser(t *testing.T) {
wantErr: true,
validationErrors: []*validate.Violation{
{
FieldPath: stringToPointer("name"),
Field: &validate.FieldPath{
Elements: []*validate.FieldPathElement{
{
FieldName: stringToPointer("name"),
},
},
},
ConstraintId: stringToPointer("required"),
Message: stringToPointer("value is required"),
}},
......
......@@ -23,7 +23,8 @@ func isEqualFieldPaths(violationFieldPath, errFieldPath *validate.FieldPath) boo
}
for i, elem := range violationFieldPath.GetElements() {
if elem != errFieldPath.GetElements()[i] {
errElem := errFieldPath.GetElements()[i]
if *elem.FieldName != *errElem.FieldName {
return false
}
}
......
......@@ -200,6 +200,8 @@ func (n *NetworkElementWatcher) StopAndRemoveNetworkElementSubscription(subID uu
// handleSubscribeResponse takes the subscribe response and additional information about the network element to distinguish
// from which network element a subscribe response was sent including improved error handling.
func (n *NetworkElementWatcher) handleSubscribeResponse(subscriptionInfo *transport.SubscriptionInformation, workerName string) {
log.Debugf("Received Subscribe response: MNE ID: %s, MNE Name: %s, SubResponse: %v", subscriptionInfo.NetworkElementID, subscriptionInfo.NetworkElementName, subscriptionInfo.SubResponse)
if subscriptionInfo.SubResponse == nil {
// Note: This needs proper error handling, no idea how yet. Simply logging would lead to spam in the console
// if the target that was subscribed to is not reachable anymore.
......@@ -232,6 +234,11 @@ func (n *NetworkElementWatcher) handleSubscribeResponse(subscriptionInfo *transp
func (n *NetworkElementWatcher) handleSubscribeResponseUpdate(resp *gpb.SubscribeResponse_Update, subscriptionInfo *transport.SubscriptionInformation) {
pathsAndValues := make(map[string]string, len(resp.Update.Update))
if resp.Update == nil || len(resp.Update.Update) == 0 {
log.Debugf("handleSubscribeResponseUpdate empty update or updates; Update: %v, InnerUpdates: %v", resp.Update, resp.Update.Update)
return
}
for _, update := range resp.Update.Update {
pathString, err := ygot.PathToString(update.Path)
if err != nil {
......
......@@ -58,7 +58,7 @@ func ensureFileSystemStoreExists(pathToStore string) error {
func ensureDirExists(fileName string) error {
dirName := filepath.Dir(fileName)
if _, serr := os.Stat(dirName); serr != nil {
merr := os.MkdirAll(dirName, 0600)
merr := os.MkdirAll(dirName, 0700)
if merr != nil {
return merr
}
......
module code.fbi.h-da.de/danet/gosdn
go 1.23
go 1.23.4
toolchain go1.23.6
require (
github.com/aristanetworks/goarista v0.0.0-20241115153057-bd75d7f26a44
github.com/aristanetworks/goarista v0.0.0-20250211154211-46edb1645c7a
github.com/c-bata/go-prompt v0.2.6
github.com/docker/docker v24.0.9+incompatible
github.com/google/go-cmp v0.6.0
github.com/google/uuid v1.6.0
github.com/grpc-ecosystem/grpc-gateway/v2 v2.24.0
github.com/grpc-ecosystem/grpc-gateway/v2 v2.26.1
github.com/mitchellh/go-homedir v1.1.0
github.com/openconfig/gnmi v0.11.0
github.com/openconfig/gnmi v0.13.0
github.com/openconfig/goyang v1.6.0
github.com/openconfig/ygot v0.29.20
github.com/prometheus/client_golang v1.20.5
......@@ -18,14 +20,14 @@ require (
github.com/sethvargo/go-password v0.3.1
github.com/sirupsen/logrus v1.9.3
github.com/spf13/cobra v1.8.1
github.com/spf13/pflag v1.0.5
github.com/spf13/pflag v1.0.6
github.com/spf13/viper v1.19.0
github.com/stretchr/objx v0.5.2 // indirect
github.com/stretchr/testify v1.10.0
go.mongodb.org/mongo-driver v1.17.1
golang.org/x/sync v0.9.0
google.golang.org/grpc v1.68.1
google.golang.org/protobuf v1.35.2
go.mongodb.org/mongo-driver/v2 v2.0.0
golang.org/x/sync v0.11.0
google.golang.org/grpc v1.70.0
google.golang.org/protobuf v1.36.5
gopkg.in/yaml.v3 v3.0.1
)
......@@ -41,13 +43,13 @@ require (
github.com/fsnotify/fsnotify v1.7.0 // indirect
github.com/gogo/protobuf v1.3.2 // indirect
github.com/golang-jwt/jwt v3.2.2+incompatible
github.com/golang/glog v1.2.3
github.com/golang/glog v1.2.4
github.com/golang/protobuf v1.5.4
github.com/golang/snappy v0.0.4 // indirect
github.com/gookit/color v1.5.4 // indirect
github.com/hashicorp/hcl v1.0.0 // indirect
github.com/inconshreveable/mousetrap v1.1.0 // indirect
github.com/klauspost/compress v1.17.9 // indirect
github.com/klauspost/compress v1.17.11 // indirect
github.com/kylelemons/godebug v1.1.0 // indirect
github.com/magiconair/properties v1.8.7 // indirect
github.com/mattn/go-colorable v0.1.13 // indirect
......@@ -64,7 +66,7 @@ require (
github.com/pkg/term v1.2.0-beta.2 // indirect
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect
github.com/prometheus/client_model v0.6.1 // indirect
github.com/prometheus/common v0.57.0 // indirect
github.com/prometheus/common v0.61.0 // indirect
github.com/prometheus/procfs v0.15.1 // indirect
github.com/rabbitmq/amqp091-go v1.10.0
github.com/rivo/uniseg v0.4.4 // indirect
......@@ -77,37 +79,38 @@ require (
github.com/xdg-go/stringprep v1.0.4 // indirect
github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e // indirect
github.com/youmark/pkcs8 v0.0.0-20240726163527-a2c0da244d78 // indirect
golang.org/x/crypto v0.29.0
golang.org/x/net v0.31.0
golang.org/x/sys v0.27.0 // indirect
golang.org/x/term v0.26.0 // indirect
golang.org/x/text v0.20.0 // indirect
golang.org/x/crypto v0.33.0
golang.org/x/net v0.35.0
golang.org/x/sys v0.30.0 // indirect
golang.org/x/term v0.29.0 // indirect
golang.org/x/text v0.22.0 // indirect
gopkg.in/ini.v1 v1.67.0 // indirect
)
require (
buf.build/gen/go/bufbuild/protovalidate/protocolbuffers/go v1.35.2-20241127180247-a33202765966.1
github.com/bufbuild/protovalidate-go v0.7.3
buf.build/gen/go/bufbuild/protovalidate/protocolbuffers/go v1.36.4-20241127180247-a33202765966.1
github.com/bufbuild/protovalidate-go v0.8.2
github.com/hashicorp/go-multierror v1.1.1
github.com/hashicorp/go-plugin v1.4.10
github.com/lesismal/nbio v1.5.12
google.golang.org/genproto/googleapis/api v0.0.0-20241202173237-19429a94021a
github.com/lesismal/nbio v1.6.2
go.mongodb.org/mongo-driver v1.17.2
google.golang.org/genproto/googleapis/api v0.0.0-20250207221924-e9438ea467c6
)
require (
atomicgo.dev/cursor v0.2.0 // indirect
atomicgo.dev/keyboard v0.2.9 // indirect
atomicgo.dev/schedule v0.1.0 // indirect
cel.dev/expr v0.18.0 // indirect
cel.dev/expr v0.19.0 // indirect
github.com/Microsoft/hcsshim v0.12.3 // indirect
github.com/antlr4-go/antlr/v4 v4.13.0 // indirect
github.com/containerd/console v1.0.3 // indirect
github.com/fatih/color v1.15.0 // indirect
github.com/google/cel-go v0.22.0 // indirect
github.com/google/cel-go v0.22.1 // indirect
github.com/hashicorp/errwrap v1.1.0 // indirect
github.com/hashicorp/go-hclog v1.5.0 // indirect
github.com/hashicorp/yamux v0.1.1 // indirect
github.com/lesismal/llib v1.1.13 // indirect
github.com/lesismal/llib v1.2.1 // indirect
github.com/lithammer/fuzzysearch v1.1.8 // indirect
github.com/mitchellh/go-testing-interface v1.14.1 // indirect
github.com/moby/patternmatcher v0.5.0 // indirect
......@@ -122,7 +125,7 @@ require (
github.com/stoewer/go-strcase v1.3.0 // indirect
go.uber.org/atomic v1.9.0 // indirect
go.uber.org/multierr v1.9.0 // indirect
golang.org/x/exp v0.0.0-20240716175740-e3f259677ff7 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20241118233622-e639e219e697 // indirect
golang.org/x/exp v0.0.0-20241009180824-f66d83c29e7c // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20250204164813-702378808489 // indirect
gotest.tools/v3 v3.5.1 // indirect
)
This diff is collapsed.
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment