README.md

# uc-monitor-go-test [![Go Report Card](https://goreportcard.com/badge/lightkone.guifi.net/lightkone/uc-monitor-go-test)](https://goreportcard.com/report/lightkone.guifi.net/lightkone/uc-monitor-go-test)

A proof-of-concept application that leverages [AntidoteDB](https://syncfree.github.io/antidote/) to orchestrate the Guifi.net network nodes monitoring system for the [LightKone](https://www.lightkone.eu/) project.

## Description
The monitoring application is distributed in three main blocks:
 - Network description fetching and feeding (functional)
 - Nodes assignment among the different monitoring servers [WiP]
 - Actual nodes monitorisation [WiP]

This proof of concept takes advantage of the [AntidoteDB Java tutorial](https://github.com/SyncFree/antidote-java-tutorial) by [Deepthi Akkoorath (@deepthidevaki)](https://github.com/deepthidevaki) and uses [Mathias Weber (@mweberUKL)](https://github.com/mweberUKL)'s Go client for AntidoteDB (previously, [João Neto (@joaomlneto)](https://github.com/joaomlneto)'s [HTTP/HTTPS REST API for AntidoteDB](https://github.com/LightKone/antidote-rest-server) was used).

## Installation

### Docker
Install `docker-ce` using your

### Go and required libraries
Install Go using your operating system's package manager or follow the [instructions here](https://golang.org/doc/install).

After installing Go, download and install the folowing external libraries needed:
 - golang/glog:

 `go get github.com/golang/glog`

 - sparrc/go-ping:

 `go get github.com/sparrc/go-ping`

  - antidote-go-client:

 `go get github.com/AntidoteDB/antidote-go-client`

### AntidoteDB Java tutorial
Get the tutorial's source code [here](https://github.com/SyncFree/antidote-java-tutorial) to download the code and [start the two AntidoteDB nodes](https://github.com/SyncFree/antidote-java-tutorial#starting-antidote-nodes).

### HTTP/HTTPS REST API

Install the AntidoteDB REST API server following the [instructions here](https://github.com/LightKone/antidote-rest-server). Once everything is installed, start the server with the `antidote-rest-server` command. The server connects to one of the AntidoteDB instances (the one running on port 8087) from the Java tutorial.

## Building and running the application

### Fetch the network description and feed it to AntidoteDB
The whole Guifi.net network description is included in the `cnml.xml` file, which can be downloaded from the Guifi.net website, and is used by default:
```
$ cd src/monitor/fetch/
$ go run monitor-fetch.go
57642 nodes read from cnml.xml
53461 devices read from cnml.xml
39988 devices exported to /tmp/gmonitor2/devs.json
0 devices removed from AntidoteDB (0 success, 0 fail)
0 IPv4 addresses removed from AntidoteDB (0 success, 0 fail)
10634 devices added to AntidoteDB (10634 success, 0 fail) ...
```
etc. Smaller subnetworks can be used instead, like the Guifi-UPC subnetwork, for testing purposes:
```
$ go run monitor-fetch.go -cnml_file upc.xml
19 nodes read from upc.xml
0 devices removed from AntidoteDB (0 success, 0 fail)
0 IPv4 addresses removed from AntidoteDB (0 success, 0 fail)
49 devices added to AntidoteDB (49 success, 0 fail)
67 IPv4 addresses added or updated to AntidoteDB (67 success, 0 fail)
```

### Assign network nodes to monitoring servers [WiP]
```
go run monitor-assign.go
```

### Monitor the nodes [WiP]
```
go run monitor-ping.go  <host>
```
### Note
Previously as of this note on Linux support: _this library attempts to send an "unprivileged" ping via UDP. On Linux, this must be enabled by setting_:

```sudo sysctl -w net.ipv4.ping_group_range="0 2147483647"```

## Data structures in AntidoteDB

### Devices/Monitors assignation
The primary data source for this application is the CNML file. The `monitor-fetch` application parses the specified CNML file and pushes its contents to AntidoteDB. There is a single `monitor-fetch` instance, and its writes/updates are __authoritative__.

In AntidoteDB, the data are structured as follows:

#### guifi (bucket)
The `guifi` bucket contains the lists of monitors and network devices to be monitored:
##### guifi (bucket) => devices (set)
The `devices` *set* in the `guifi` *bucket* is an `array` of `strings`, each `string` containing the `ID` of a Guifi.net device. For example:
```bash
$ curl localhost:3000/set/read/guifi/devices
  ["22110","26932","38720","40605","40962","41175","42331","42626","42627","42628",
   "46654","46656","47103","48030","51580","57728","59001","60415","64962","64963",
   "64965","64966","65291","65720","72843","73952","79715","81297","82096","82097",
   "82098","82099","82103","82104","82105","82111","83865","85877","87503","90228",
   "92032","92802","92803","92804","94210","94965","96225","96676","96684"]
```

##### guifi (bucket) => monitors (set)
The `monitors` *set* in the `guifi` *bucket* is an `array` of `strings`, each `string` containing the `ID` of a Guifi.net monitor. For example:
```bash
$ curl localhost:3000/set/read/guifi/monitors
  ["a45632","a47363", "21435"]
```
When started, each of the monitoring instances register to the system by adding their `ID` to this *set*.

##### guifi (bucket) => checksum (LWW<sup>1</sup> register)
The `checksum` *LWW register* in the `guifi` *bucket* is a `strings` containing the *SHA256 checksum* of the CNML data fetched from the Guifi.net website and pushed to the database. For example:
```bash
$ curl localhost:3000/register/read/guifi/checksum
  35aaa826b841ed412897691bb1f50278d742ef9a76da9750a8ae509d3b01f8ee
```

#### device-i (bucket)
The `device-i` bucket, where `i` is the numeric `ID` of a device in the `guifi/devices` set, contains the information about a Guifi.net device:

##### device-i (bucket) => ipv4s (set)
The `ipv4s` *set* in the `device-i` *bucket* is an `array` of `strings`, each `string` containing an IPv4 address of the device. For example:
```bash
$ curl localhost:3000/set/read/device-26932/ipv4s
  ["10.139.37.226","172.25.40.188","172.25.40.189"]
```

##### device-i (bucket) => monitors (set)
The `monitors` *set* in the `device-i` *bucket* is an `array` of `strings`, each `string` containing the ID of a monitor the device is assigned to (i.e. the `ID` of a monitor that is in charge of monitoring the device). For example:
```bash
$ curl localhost:3000/set/read/device-26932/monitors
  ["a45632","a47363", "21435"]
```

##### device-i (bucket) => graphserver (LWW register)
The `graphserver` *LWW register* in the `device-i` *bucket* is a `string` containing the ID of a monitor the device is assigned to **in the Guifi.net** website (i.e., not automatically assigned by the monitoring application, but done manually on the Guifi.net website, and included in the CNML). For example:

```bash
$ curl localhost:3000/register/read/device-26932/graphserver
  71808
```

---
<sup>1</sup> LWW: last writer wins