From 65e328cd0fa48fbcbf5ad03d6bef0b1ed22cda3e Mon Sep 17 00:00:00 2001
From: thl-cmk <thl-cmk@outlook.com>
Date: Sun, 17 Nov 2024 16:24:42 +0000
Subject: [PATCH] Update README.md
---
README.md | 90 +++++++++++++++++++++++++++----------------------------
1 file changed, 44 insertions(+), 46 deletions(-)
diff --git a/README.md b/README.md
index 567c5c3..61688c3 100644
--- a/README.md
+++ b/README.md
@@ -1,4 +1,4 @@
-[PACKAGE]: ../../raw/master/mkp/nvdct-0.9.2-20241116.mkp "nvdct-0.9.2-20241116.mkp"
+[PACKAGE]: ../../raw/master/mkp/nvdct-0.9.2-20241117.mkp "nvdct-0.9.2-20241117.mkp"
# Network Visualization Data Creation Tool (NVDCT)
This script creates the topology data file needed for the [Checkmk Exchange Network visualization](https://exchange.checkmk.com/p/network-visualization) plugin.\
@@ -79,27 +79,23 @@ and let Checkmk run the HW/SW inventory for each device at least once
Usage with CDP and STATIC (this is the default):
```
-~/local/bin/nvdct/nvdct.py -s Core01 -d
+~/local/bin/nvdct/nvdct.py -u ~/local/bin/conf/my_nvdct.toml -d
```
Usage with LLDP and STATIC:
```
-~/local/bin/nvdct/nvdct.py -s Core01 -d -l LLDP STATIC
+~/local/bin/nvdct/nvdct.py -u ~/local/bin/conf/my_nvdct.toml -d -l LLDP STATIC
```
<details><summary>Help output from nvdct.py --help</summary>
```
OMD[build]:~$ ~/local/bin/nvdct/nvdct.py -h
-usage: nvdct.py [-h] [-b {LIVESTATUS,MULTISITE,RESTAPI}] [-d] [-o OUTPUT_DIRECTORY]
- [-s SEED_DEVICES [SEED_DEVICES ...]] [-p PREFIX]
- [-l {CDP,CUSTOM,LLDP,STATIC,L3v4} [{CDP,CUSTOM,LLDP,STATIC,L3v4} ...]]
- [-u USER_DATA_FILE] [-v] [--api-port API_PORT] [--case {LOWER,UPPER}]
- [--check-user-data-only] [--log-file LOG_FILE]
- [--log-level {CRITICAL,FATAL,ERROR,WARNING,INFO,DEBUG,OFF}] [--log-to-stdout]
- [--dont-compare] [--filter-customers {INCLUDE,EXCLUDE}]
- [--filter-sites {INCLUDE,EXCLUDE}] [--keep-domain] [--keep KEEP]
- [--min-age MIN_AGE] [--pre-fetch] [--quiet] [--skip-l3-if] [--skip-l3-ip]
- [--time-format TIME_FORMAT]
+usage: nvdct.py [-h] [-b {LIVESTATUS,MULTISITE,RESTAPI}] [-d] [-o OUTPUT_DIRECTORY] [-p PREFIX]
+ [-l {CDP,CUSTOM,LLDP,STATIC,L3v4} [{CDP,CUSTOM,LLDP,STATIC,L3v4} ...]] [-u USER_DATA_FILE] [-v]
+ [--api-port API_PORT] [--case {LOWER,UPPER}] [--check-user-data-only] [--log-file LOG_FILE]
+ [--log-level {CRITICAL,FATAL,ERROR,WARNING,INFO,DEBUG,OFF}] [--log-to-stdout] [--dont-compare]
+ [--filter-customers {INCLUDE,EXCLUDE}] [--filter-sites {INCLUDE,EXCLUDE}] [--remove-domain] [--keep KEEP]
+ [--min-age MIN_AGE] [--pre-fetch] [--quiet] [--skip-l3-if] [--skip-l3-ip] [--time-format TIME_FORMAT]
This script creates the topology data file needed for the Checkmk "network_visualization"
plugin by Andreas Boesl and schnetz. For more information see
@@ -111,7 +107,7 @@ CDP: https://thl-cmk.hopto.org/gitlab/checkmk/vendor-independent/inventory/inv_c
LLDP: https://thl-cmk.hopto.org/gitlab/checkmk/vendor-independent/inventory/inv_lldp_cache
L3v4: https://thl-cmk.hopto.org/gitlab/checkmk/vendor-independent/inventory/inv_ipv4_addresses
-Version: 0.9.0-20240923 | Written by: thl-cmk
+Version: 0.9.2-20241117 | Written by: thl-cmk
for more information see: https://thl-cmk.hopto.org/gitlab/checkmk/vendor-independent/nvdct
options:
@@ -128,19 +124,16 @@ options:
I.e.: my_topology. Default is the actual date/time
in "--time-format" format.
NOTE: the directory is a sub directory under "~/var/check_mk/topology/data/"
- -s SEED_DEVICES [SEED_DEVICES ...], --seed-devices SEED_DEVICES [SEED_DEVICES ...]
- List of devices to start the topology discovery from.
- I.e. Core01 Core02
-p PREFIX, --prefix PREFIX
Prepends each host with the prefix. (Needs testing)
-l {CDP,CUSTOM,LLDP,STATIC,L3v4} [{CDP,CUSTOM,LLDP,STATIC,L3v4} ...], --layers {CDP,CUSTOM,LLDP,STATIC,L3v4} [{CDP,CUSTOM,LLDP,STATIC,L3v4} ...]
- CDP : needs inv_cdp_cache package at least in version 0.7.1-20240320
- LLDP : needs inv_lldp_cache package at least in version 0.9.3-20240320
- L3v4 : needs inv_ipv4_addresses package at least in version 0.0.3-20240320
- adds, layer 3 topology fpr IPv4
+ adds, layer 3 topology fpr IPv4
-u USER_DATA_FILE, --user-data-file USER_DATA_FILE
Set the name uf the user provided data file
- Default is ~/local/bin/nvdct/conf/conf/nvdct.toml
+ Default is ~/local/bin/nvdct/conf/nvdct.toml
-v, --version Print version of this script and exit
--api-port API_PORT TCP Port to access the REST API. Default is 80. NVDCT will try to automatically
detect the site apache port.
@@ -162,7 +155,7 @@ options:
--filter-sites {INCLUDE,EXCLUDE}
INCLUDE/EXCLUDE site list from config file.
Note: MULTISITE backend only.
- --keep-domain Do not remove the domain name from the neighbor name
+ --remove-domain Remove the domain name from the neighbor name
--keep KEEP Number of topologies to keep. The oldest topologies above keep
max will be deleted.
NOTE: The default topologies will be always kept.
@@ -182,7 +175,7 @@ Exit codes:
4 - Automation secret not found
Usage:
-~/local/bin/nvdct/nvdct.py -s Core01 Core02 -d
+~/local/bin/nvdct/nvdct.py -u ~/local/bin/nvdct/conf/nvdct.toml
OMD[build]:~$
```
@@ -192,7 +185,6 @@ OMD[build]:~$
---
### Known limitations
-- the backend `FILESYSTEM` can only acces the local autochheck files, this might lead in ditributed environments to incomplete data, use `MUTLISITE` or `RESTAPI` instead
---
### FAQ
@@ -203,7 +195,7 @@ OMD[build]:~$
To make the topology created by this tool the default use the option `-d` / `--default`
```
-$ ~/local/bin/nvdct/nvdct.py -s Core01 -d
+$ ~/local/bin/nvdct/nvdct.py -u ~/local/bin/conf/my_nvdct.toml -d
```
</details>
@@ -214,7 +206,7 @@ $ ~/local/bin/nvdct/nvdct.py -s Core01 -d
Yes. Add a cron job for the site user in `~/etc/cron.d`. I.e. to create a topology every hour create the file `nvdct` in `~/etc/cron.d`
```
-0 * * * * $OMD_ROOT/local/bin/nvdct/nvdct.py -s Core01 -d --keep 10 --min-age 14
+0 * * * * $OMD_ROOT/local/bin/nvdct/nvdct.py $OMD_ROOT/local/bin/nvdct/conf_my_nvdct.toml -d --keep 10 --min-age 14
```
This will run the tool every hour and create a new topology if there was a changee. It keeps 10 topologies and deletes old topologies only if they are older than 14 days.
@@ -235,7 +227,7 @@ With crontab -l you get the list of active cron jobs
OMD[build]:~$ crontab -l | grep -A 3 -B 1 nvdct
# ------------------------------------------------------------
# /omd/sites/build/etc/cron.d/nvdct
-0 * * * * $OMD_ROOT/local/bin/nvdct/nvdct.py -s Core01 d --keep 10
+0 * * * * $OMD_ROOT/local/bin/nvdct/nvdct.py $OMD_ROOT/local/bin/nvdct/conf_my_nvdct.toml d --keep 10
# ------------------------------------------------------------
```
@@ -250,7 +242,7 @@ OMD[build]:~$ crontab -l | grep -A 3 -B 1 nvdct
Yes. If you use the option --keep <number of topologies to keep> the tool will delete all topologies over this number. I.e.
```
-$ ~/local/bin/nvdct/nvdct.py -s Core01 --keep 10
+$ ~/local/bin/nvdct/nvdct.py -u ~/local/bin/conf/my_nvdct.toml --keep 10
```
will delete the oldest topologies until there only 10 topologies left.
@@ -262,7 +254,7 @@ will delete the oldest topologies until there only 10 topologies left.
\
No problem. Just use the option `--min-age` togeher with `--keep`. I.e.
```
-$ ~/local/bin/nvdct/nvdct.py -s Core01 --keep 10 --min-age 30
+$ ~/local/bin/nvdct/nvdct.py -u ~/local/bin/conf/my_nvdct.toml --keep 10 --min-age 30
```
will only delete a topology if it is older than 30 days.
</details>
@@ -270,12 +262,12 @@ will only delete a topology if it is older than 30 days.
<details><summary>Can the topology data creation be customized</summary>
\
-Yes. You can customize the topology data creation by modifying the `~/local/bin/nvdct/conf/nvdct.toml` file. Or provide your own data file (see Option -u, --user-data-file)
+Yes you can and you sould. Create a copy of the default `~/local/bin/nvdct/conf/nvdct.toml` file. Or provide your own data file (see Option -u, --user-data-file).
**Note**: This file uses [Tom's Obvious Minimal Language](https://toml.io/en/)
```
-~/local/bin/nvdct/nvdct.py -u ~/local/bin/nvdct/conf/my.toml
+~/local/bin/nvdct/nvdct.py -u ~/local/bin/nvdct/conf/my_nvdct.toml
```
</details>
@@ -287,28 +279,35 @@ If the neighbor names from the inventory and the checkmk host names do not match
```
# map inventory neighbour name to Checkmk host name
-[HOST_MAP]
+[L2_HOST_MAP]
inventory_neighbour1 = "cmk_host1"
inventory_neighbour2 = "cmk_host2"
inventory_neighbour3 = "cmk_host3"
```
-**NOTE**: The script will automatically remove the domain name form the neighbour names. Use the option `--keep-domain` if you want to keep the domain name.
</details>
<details><summary>In my inventory are invalid neighbor names</summary>
\
-If you have invalid neighbor names in the inventory (i.e. "not advertised"), you can drop them by adding them to the **DROP_HOSTS** list in the data file (by default `nvdct.toml`). I.e..
+If you have invalid neighbor names in the inventory (i.e. "not advertised"), you can drop them by adding them to the **L2_DROP_HOSTS** list in the data file (by default `nvdct.toml`). I.e..
```
# drop neighbours with invalid names
-DROP_HOSTS = [
+L2_DROP_HOSTS = [
"not advertised",
"a nother invalid name",
]
```
+If you need a more advanced option to remove neighbours from the topology, you can also use the `L2_NEIGHBOUR_REPLACE_REGEX` section.
+Ie. to remove all neighbours advertised as MAC-address only use:
+
+```
+[L2_NEIGHBOUR_REPLACE_REGEX]
+"^(([0-9a-fA-F]){2}[:.-]?){5}([0-9a-fA-F]){2}$" = ""
+```
+
</details>
@@ -341,7 +340,7 @@ CUSTOM_LAYERS = [
```
then run the tool with the option `-l CUSTOM`
```
-~/local/bin/network-topology/nvdct.py -s Core01 -l CUSTOM
+~/local/bin/network-topology/nvdct.py -u ~/local/bin/conf/my_nvdct.toml -l CUSTOM
```
</details>
@@ -362,30 +361,29 @@ First check your HW/SW inventory if it not contains the data needed to create th
Sets the loglevel. The default is "WARN"
* --log-to-stdout Send log to stdout.
+For more troubleshooting see [TROUBLESHOOTING.md](/TROUBLESHOOTING.md)
+
</details>
---
### Sample Output
-Sample output of NVDCT.
-
```
-OMD[build]:~$ ~/local/bin/nvdct/nvdct.py -u ~/local/bin/nvdct/conf/my.toml
+OMD[build]:~$ ~/local/bin/nvdct/nvdct.py -u ~/local/bin/nvdct/conf/lrm.toml -d
Network Visualisation Data Creation Tool (NVDCT)
-by thl-cmk[at]outlook[dot]com, version 0.8.7-20240430
+by thl-cmk[at]outlook[dot]com, version 0.9.2-20241117
see https://thl-cmk.hopto.org/gitlab/checkmk/vendor-independent/nvdct
-Start time....: 2024-04-30T14:17:02.04
-Devices added.: 16, source lldp
-Devices added.: 72, source cdp
-Devices added.: 29, source STATIC
-Devices added.: 59, source L3v4
-Time taken....: 3.225333135/s
-End time......: 2024-04-30T14:17:05.04
+Start time....: 2024-11-17T17:20:54.11
+Source lldp...: Devices/Objects/Connections added 8/18/15
+Source cdp....: Devices/Objects/Connections added 28/144/174
+Source static.: Devices/Objects/Connections added 29/85/84
+Source l3v4...: Devices/Objects/Connections added 2/141/140
+Time taken....: 13.360647338/s
+End time......: 2024-11-17T17:21:08.11
OMD[build]:~$
-
```
Sample network topology CDP/LLDP
--
GitLab