README.md

[PACKAGE]: ../../raw/master/create_topology_data-0.0.2-20231012.mkp "create_topology_data-0.0.2-20231012.mkp"
# PoC for creating topology_data.json from inventory data

This script creates the topology data file needed for the [Checkmk Exchange Network visualization](https://forum.checkmk.com/u/schnetz) plugin by Andreas Boesl and [schnetz](https://exchange.checkmk.com/u/schnetz). For more information see [Checkmk forum Network Visualization](https://forum.checkmk.com/t/network-visualization/41680). 

The network topology data is read from the Checkmk HW/SW inventory. The necessary inventory data can be created with my CDP/LLDP inventory plugins.

CDP: [CDP inventory plugin](https://thl-cmk.hopto.org/gitlab/checkmk/vendor-independent/inventory/inv_cdp_cache/)\
CDP path-in-inventory: `"networking,cdp_cache"`\
CDP inventory-columns: `"device_id,local_port,device_port"`

LLDP: [LLDP inventory plugin](https://thl-cmk.hopto.org/gitlab/checkmk/vendor-independent/inventory/inv_lldp_cache/)\
LLDP path-in-inventory: `"networking,lldp_cache"`\
LLDP inventory-columns: `"system_name,local_port_num,port_id"`

Usage with CDP (this is the default):
```
~/local/lib/topology_data/create_topology_data.py -s CORE01 -m
```

Usage with LLDP:
```
~/local/lib/topology_data/create_topology_data.py -s CORE01 -m -p "networking,lldp_cache" -c "system_name,local_port_num,port_id" -d inv_LLDP
```

---
### Download
* [Download latest mkp file][PACKAGE]
                        
---
### Installation

You can install the package by uploading it to your CheckMK site and as site user run 
```
mkp add PAKAGE_NAME.mkp
mkp enable PAKAGE_NAME VERSION
```

In the Enterprise/Free/Cloud edition of CheckMK you can use the GUI to install the package (_Setup_ -> _Extension Packages_ -> _Upload package_)

---
### Want to Contribute?

Nice ;-) Have a look at the [contribution guidelines](CONTRIBUTING.md "Contributing")

---
### Usage

```
OMD[build]:~$ ./local/bin/topology_data/create_topology_data.py -h
usage: create_topology_data.py [-h] [-s SEED_DEVICES [SEED_DEVICES ...]] [-v] [-p PATH_IN_INVENTORY [PATH_IN_INVENTORY ...]] [-c INVENTORY_COLUMNS] [-d DATA_SOURCE] [-t TIME_FORMAT] [-o OUTPUT_DIRECTORY] [-m] [-k] [-u | -l]

This script creates the topology data file needed for the Checkmk "network_visualization" plugin by Andreas Boesl and schnetz.
For more information see https://forum.checkmk.com/t/network-visualization/41680 and https://exchange.checkmk.com/p/network-visualization

The inventory data could be created with my inventory plugins:
CDP: https://thl-cmk.hopto.org/gitlab/checkmk/vendor-independent/inventory/inv_cdp_cache
LLDP: https://thl-cmk.hopto.org/gitlab/checkmk/vendor-independent/inventory/inv_lldp_cache

Version: 0.0.2-20231012 | Written by: thl-cmk, for more information see: https://thl-cmk.hopto.org

options:
  -h, --help            show this help message and exit
  -s SEED_DEVICES [SEED_DEVICES ...], --seed-devices SEED_DEVICES [SEED_DEVICES ...]
                        List of devices to start the topology discovery from. I.e. Core01 Core02
  -v, --version         Print version of this script and exit
  -p PATH_IN_INVENTORY [PATH_IN_INVENTORY ...], --path-in-inventory PATH_IN_INVENTORY [PATH_IN_INVENTORY ...]
                        Checkmk inventory path to the topology data. I.e. "networking,cdp_cache"
  -c INVENTORY_COLUMNS, --inventory-columns INVENTORY_COLUMNS
                        Columns used from the inventory data. I.e. "device_id,local_port,device_port"
                        NOTE: the columns must be in the order: neighbour, local_port, neighbour_port
  -d DATA_SOURCE, --data-source DATA_SOURCE
                        The source from which the topology data originates. I.e. inv_CDP for CDP data from the inventory. NOTE: right now this only an unused label.
  -t TIME_FORMAT, --time-format TIME_FORMAT
                        Format string to render the time. (default: %Y-%m-%dT%H:%M:%S.%m)
  -o OUTPUT_DIRECTORY, --output-directory OUTPUT_DIRECTORY
                        Directory name where to save the topology data. I.e.: my_topology. Default is the actual date/time in "--time-format" format.
                        NOTE: the directory is a sub directory under "~/var/topology_data/"
  -m, --make-default    Set the created topology data as default
  -k, --keep-domain     Do not remove the domain name from the neighbor name
  -u, --uppercase       Change neighbour names to all upper case
  -l, --lowercase       Change neighbour names to all lower case

Usage:
for CDP (the default):
~/local/bin/network-topology/create_topology_data.py -s Core01 Core02 -m
or
~/local/bin/network-topology/create_topology_data.py -s Core01 Core02 -m -p "networking,cdp_cache" -c "device_id,local_port,device_port" -d inv_CDP

for LLDP:
~/local/bin/network-topology/create_topology_data.py -s Core01 Core02 -m -p "networking,lldp_cache" -c "system_name,local_port_num,port_id" -d inv_LLDP
```

---
### Limitations

- neighbour names in CD/LLDP data have to match the host names in Checkmk
- local/neighbour interface names have to match the service names in Checkmk

If neighbour names in CD/LLDP data and Checkmk host names don't match, you can configure a mapping in the `create_topology_data.py` file. I.e.:

```
# map inventory neighbour name to CMK host name
HOST_MAP = {
    'neighbour': 'cmk_host',,
    'nexus01': 'NX01',
    'nexus02': 'NX02',
    'nexus03': 'NX03',
}
```
**NOTE**: the script will automatically remove the domain name form the neighbour names. See Usage option `-k` / `--keep-domain`

At the same place you can exclude invalid hosts from the topology data
```
# drop neighbours with invalid names
DROP_HOSTS = [
    'not advertised',
]
```

---
### Sample Output

Sample output

```
OMD[build]:~$ ~/local/bin/topology_data/create_topology_data.py -s C9540-7-1 -m
Start time: 2023-10-12T16:31:32.10
Device: lr-cos-ap-010: not found in auto checks path!
Device: lr-cos-ap-010: interface (Gi0) not found in services
Devices added: 56, source inv_CDP
time taken: 0.532150752/s
End time: 2023-10-12T16:31:33.10
OMD[build]:~$ 
```

Sample network topology

![sample output details](/img/sample-details.png?raw=true "sample output details")