# uc-monitor-go-test [![Go Report Card](https://goreportcard.com/badge/lightkone.guifi.net/lightkone/uc-monitor-go-test)](https://goreportcard.com/report/lightkone.guifi.net/lightkone/uc-monitor-go-test) A proof-of-concept application that leverages [AntidoteDB](https://syncfree.github.io/antidote/) to orchestrate the Guifi.net network nodes monitoring system for the [LightKone](https://www.lightkone.eu/) project. ## Description The monitoring application is distributed in three main blocks: - Network description fetching and feeding (functional) - Nodes assignment among the different monitoring servers [WiP] - Actual nodes monitorisation [WiP] This proof of concept takes advantage of the [AntidoteDB Java tutorial](https://github.com/SyncFree/antidote-java-tutorial) by [Deepthi Akkoorath (@deepthidevaki)](https://github.com/deepthidevaki) and uses [Mathias Weber (@mweberUKL)](https://github.com/mweberUKL)'s Go client for AntidoteDB (previously, [João Neto (@joaomlneto)](https://github.com/joaomlneto)'s [HTTP/HTTPS REST API for AntidoteDB](https://github.com/LightKone/antidote-rest-server) was used). ## Installation ### Docker Install `docker-ce` using your ### Go and required libraries Install Go using your operating system's package manager or follow the [instructions here](https://golang.org/doc/install). After installing Go, download and install the folowing external libraries needed: - golang/glog: `go get github.com/golang/glog` - sparrc/go-ping: `go get github.com/sparrc/go-ping` - antidote-go-client: `go get github.com/AntidoteDB/antidote-go-client` ### AntidoteDB Java tutorial Get the tutorial's source code [here](https://github.com/SyncFree/antidote-java-tutorial) to download the code and [start the two AntidoteDB nodes](https://github.com/SyncFree/antidote-java-tutorial#starting-antidote-nodes). ### HTTP/HTTPS REST API Install the AntidoteDB REST API server following the [instructions here](https://github.com/LightKone/antidote-rest-server). Once everything is installed, start the server with the `antidote-rest-server` command. The server connects to one of the AntidoteDB instances (the one running on port 8087) from the Java tutorial. ## Building and running the application ### Fetch the network description and feed it to AntidoteDB The whole Guifi.net network description is included in the `cnml.xml` file, which can be downloaded from the Guifi.net website, and is used by default: ``` $ cd src/monitor/fetch/ $ go run monitor-fetch.go 57642 nodes read from cnml.xml 53461 devices read from cnml.xml 39988 devices exported to /tmp/gmonitor2/devs.json 0 devices removed from AntidoteDB (0 success, 0 fail) 0 IPv4 addresses removed from AntidoteDB (0 success, 0 fail) 10634 devices added to AntidoteDB (10634 success, 0 fail) ... ``` etc. Smaller subnetworks can be used instead, like the Guifi-UPC subnetwork, for testing purposes: ``` $ go run monitor-fetch.go -cnml_file upc.xml 19 nodes read from upc.xml 0 devices removed from AntidoteDB (0 success, 0 fail) 0 IPv4 addresses removed from AntidoteDB (0 success, 0 fail) 49 devices added to AntidoteDB (49 success, 0 fail) 67 IPv4 addresses added or updated to AntidoteDB (67 success, 0 fail) ``` ### Assign network nodes to monitoring servers [WiP] ``` go run monitor-assign.go ``` ### Monitor the nodes [WiP] ``` go run monitor-ping.go ``` ### Note Previously as of this note on Linux support: _this library attempts to send an "unprivileged" ping via UDP. On Linux, this must be enabled by setting_: ```sudo sysctl -w net.ipv4.ping_group_range="0 2147483647"``` ## Data structures in AntidoteDB ### Devices/Monitors assignation The primary data source for this application is the CNML file. The `monitor-fetch` application parses the specified CNML file and pushes its contents to AntidoteDB. There is a single `monitor-fetch` instance, and its writes/updates are __authoritative__. In AntidoteDB, the data are structured as follows: #### guifi (bucket) The `guifi` bucket contains the lists of monitors and network devices to be monitored: ##### guifi (bucket) => devices (set) The `devices` *set* in the `guifi` *bucket* is an `array` of `strings`, each `string` containing the `ID` of a Guifi.net device. For example: ```bash $ curl localhost:3000/set/read/guifi/devices ["22110","26932","38720","40605","40962","41175","42331","42626","42627","42628", "46654","46656","47103","48030","51580","57728","59001","60415","64962","64963", "64965","64966","65291","65720","72843","73952","79715","81297","82096","82097", "82098","82099","82103","82104","82105","82111","83865","85877","87503","90228", "92032","92802","92803","92804","94210","94965","96225","96676","96684"] ``` ##### guifi (bucket) => monitors (set) The `monitors` *set* in the `guifi` *bucket* is an `array` of `strings`, each `string` containing the `ID` of a Guifi.net monitor. For example: ```bash $ curl localhost:3000/set/read/guifi/monitors ["a45632","a47363", "21435"] ``` When started, each of the monitoring instances register to the system by adding their `ID` to this *set*. ##### guifi (bucket) => checksum (LWW1 register) The `checksum` *LWW register* in the `guifi` *bucket* is a `strings` containing the *SHA256 checksum* of the CNML data fetched from the Guifi.net website and pushed to the database. For example: ```bash $ curl localhost:3000/register/read/guifi/checksum 35aaa826b841ed412897691bb1f50278d742ef9a76da9750a8ae509d3b01f8ee ``` #### device-i (bucket) The `device-i` bucket, where `i` is the numeric `ID` of a device in the `guifi/devices` set, contains the information about a Guifi.net device: ##### device-i (bucket) => ipv4s (set) The `ipv4s` *set* in the `device-i` *bucket* is an `array` of `strings`, each `string` containing an IPv4 address of the device. For example: ```bash $ curl localhost:3000/set/read/device-26932/ipv4s ["10.139.37.226","172.25.40.188","172.25.40.189"] ``` ##### device-i (bucket) => monitors (set) The `monitors` *set* in the `device-i` *bucket* is an `array` of `strings`, each `string` containing the ID of a monitor the device is assigned to (i.e. the `ID` of a monitor that is in charge of monitoring the device). For example: ```bash $ curl localhost:3000/set/read/device-26932/monitors ["a45632","a47363", "21435"] ``` ##### device-i (bucket) => graphserver (LWW register) The `graphserver` *LWW register* in the `device-i` *bucket* is a `string` containing the ID of a monitor the device is assigned to **in the Guifi.net** website (i.e., not automatically assigned by the monitoring application, but done manually on the Guifi.net website, and included in the CNML). For example: ```bash $ curl localhost:3000/register/read/device-26932/graphserver 71808 ``` --- 1 LWW: last writer wins