From 1a7ee52e3bcbaa2b7e5ef19c1001f5e88218e2f1 Mon Sep 17 00:00:00 2001
From: thl-cmk <thl-cmk@outlook.com>
Date: Wed, 5 Feb 2025 19:41:42 +0000
Subject: [PATCH] Edit README.md
---
README.md | 158 ++++++++++++++++++++++++++++--------------------------
1 file changed, 81 insertions(+), 77 deletions(-)
diff --git a/README.md b/README.md
index ac47e7c..e302456 100644
--- a/README.md
+++ b/README.md
@@ -88,15 +88,16 @@ Usage with LLDP:
<details><summary>Help output from nvdct.py --help</summary>
```
-OMD[build]:~$ ./local/bin/nvdct/nvdct.py --help
-usage: nvdct.py [-h] [-b {LIVESTATUS,MULTISITE,RESTAPI}] [-d] [-o OUTPUT_DIRECTORY] [-p PREFIX]
- [-l {CDP,LLDP,L3v4,STATIC} [{CDP,LLDP,L3v4,STATIC} ...]] [-u USER_DATA_FILE] [-v] [--adjust-toml]
- [--api-port API_PORT] [--case {LOWER,UPPER}] [--check-user-data-only] [--log-file LOG_FILE]
- [--log-level {CRITICAL,FATAL,ERROR,WARNING,INFO,DEBUG,OFF}] [--log-to-stdout] [--display-l2-neighbours]
- [--dont-compare] [--filter-customers {INCLUDE,EXCLUDE}] [--filter-sites {EXCLUDE,EXCLUDE}] [--include-l3-hosts]
- [--include-l3-loopback] [--remove-domain] [--keep KEEP] [--min-age MIN_AGE] [--pre-fetch] [--quiet]
- [--skip-l3-cidr-0] [--skip-l3-cidr-32-128] [--skip-l3-if] [--skip-l3-ip] [--skip-l3-public]
- [--time-format TIME_FORMAT]
+OMD[build]:~$ ~/local/bin/nvdct/nvdct.py --help
+usage: nvdct.py [-h] [-b {LIVESTATUS,MULTISITE,RESTAPI}] [-c CONFIG] [-d]
+ [-l {CDP,LLDP,L3v4,STATIC} [{CDP,LLDP,L3v4,STATIC} ...]] [-o OUTPUT_DIRECTORY] [-v] [--api-port API_PORT]
+ [--check-config] [--dont-compare] [--filter-customers {INCLUDE,EXCLUDE}] [--filter-sites {EXCLUDE,EXCLUDE}]
+ [--keep-max-topologies KEEP_MAX_TOPOLOGIES] [--l2-case {INSENSITIVE,LOWER,UPPER,AUTO,OFF}] [--l2-display-ports]
+ [--l2-display-neighbours] [--l2-prefix L2_PREFIX] [--l2-remove-domain {OFF,ON,AUTO}] [--l2-skip-external]
+ [--l3-display-devices] [--l3-include-hosts] [--l3-include-loopback] [--l3-skip-cidr-0] [--l3-skip-cidr-32-128]
+ [--l3-skip-if] [--l3-skip-ip] [--l3-skip-public] [--log-file LOG_FILE]
+ [--log-level {CRITICAL,FATAL,ERROR,WARNING,INFO,DEBUG,OFF}] [--log-to-stdout]
+ [--min-topology-age MIN_TOPOLOGY_AGE] [--pre-fetch] [--quiet] [--time-format TIME_FORMAT] [--update-config]
This script creates the topology data file needed for the Checkmk Network Visualization,
For more information see the announcement from schnetz in the Checkmk forum:
@@ -105,7 +106,7 @@ https://forum.checkmk.com/t/network-visualization/41680
The required plugins to create the inventory data can be found here:
https://thl-cmk.hopto.org/gitlab/explore/projects/topics/Network%20Visualization
-Version: 0.9.7-20241230 | Written by: thl-cmk
+Version: 0.9.8-20250205 | Written by: thl-cmk
for more information see: https://thl-cmk.hopto.org/gitlab/checkmk/vendor-independent/nvdct
options:
@@ -114,68 +115,85 @@ options:
Backend used to retrieve the topology data
- LIVESTATUS : fetches data via local Livestatus (local site only)
- MULTISITE : like LIVESTATUS but for distributed environments (default)
- - RESTAPI : uses the CMK REST API.
+ - RESTAPI : uses the Checkmk REST API.
+ -c CONFIG, --config CONFIG
+ Set the name of the config file.
+ Default is ~/local/bin/nvdct/conf/nvdct.toml
-d, --default Set the created topology data as default. Will be created automatically
if it doesnt exists.
+ -l {CDP,LLDP,L3v4,STATIC} [{CDP,LLDP,L3v4,STATIC} ...], --layers {CDP,LLDP,L3v4,STATIC} [{CDP,LLDP,L3v4,STATIC} ...]
+ - CDP : needs inv_cdp_cache package at least in version 0.7.1-20240320
+ - LLDP : needs inv_lldp_cache package at least in version 0.9.3-20240320
+ - L3v4 : needs inv_ip_address package at least in version 0.0.6-20241210 for SNMP based hosts
+ for Linux based hosts inv_lnx_ip_if in version 0.0.4-20241210
+ for Windows based hosts inv_win_ip_if in version 0.0.3-20241210
+ - STATIC : creates a topology base on the "[STATIC_CONNECTIONS]" section in the TOML file
-o OUTPUT_DIRECTORY, --output-directory OUTPUT_DIRECTORY
Directory name where to save the topology data.
I.e.: my_topology. Default is the actual date/time
in "--time-format" format.
NOTE: the directory is a sub directory under "~/var/check_mk/topology/data/"
- -p PREFIX, --prefix PREFIX
- Prepends each host with the prefix. (Needs more testing)
- -l {CDP,LLDP,L3v4,STATIC} [{CDP,LLDP,L3v4,STATIC} ...], --layers {CDP,LLDP,L3v4,STATIC} [{CDP,LLDP,L3v4,STATIC} ...]
- - CDP : needs inv_cdp_cache package at least in version 0.7.1-20240320
- - LLDP : needs inv_lldp_cache package at least in version 0.9.3-20240320
- - L3v4 : needs inv_ip_address package at least in version 0.0.6-20241210 for SNMP based hosts
- for Linux based hosts inv_lnx_ip_if in version 0.0.4-20241210
- for Windows based hosts inv_win_ip_if in version 0.0.3-20241210
- - STATIC : creates a topology base on the "[STATIC_CONNECTIONS]" section in the TOML file
- -u USER_DATA_FILE, --user-data-file USER_DATA_FILE
- Set the name of the user provided data file
- Default is ~/local/bin/nvdct/conf/nvdct.toml
-v, --version Print version of this script and exit
- --adjust-toml Adjusts old options in TOML file.
--api-port API_PORT TCP Port to access the REST API. By NVDCT will try to automatically
detect the site apache port.
- --case {LOWER,UPPER} Change L2 neighbour name to all lower/upper case before matching to CMK host
- --check-user-data-only
- Only tries to read/parse the user data from nvdct.toml and exits.
- --log-file LOG_FILE Set the log file. Default is "~/var/log/nvdct.log"
- --log-level {CRITICAL,FATAL,ERROR,WARNING,INFO,DEBUG,OFF}
- Sets the log level. The default is "WARNING"
- --log-to-stdout Send log to stdout.
- --display-l2-neighbours
- Use L2 neighbour name as display name in L2 topologies
+ --check-config Only tries to read/parse the config file from nvdct.toml and exits.
--dont-compare Do not compare the actual topology data with the default topology
data. By default, the actual topology is compared with the default
topology. If the data matches, the actual topology is not saved.
So, if you run this tool in a cron job, a new topology will be
created only if there was a change, unless you use "--dont-compare".
--filter-customers {INCLUDE,EXCLUDE}
- INCLUDE/EXCLUDE customer list "[CUSTOMERS]" from TOML file.
- Note: MULTISITE backend only.
+ INCLUDE/EXCLUDE customer list "[FILTER_BY_CUSTOMER]" from TOML file.NOTE: MULTISITE backend only.
--filter-sites {EXCLUDE,EXCLUDE}
- INCLUDE/EXCLUDE site list "[SITES]" from TOML file.
- --include-l3-hosts Include hosts (single IP objects) in layer 3 topologies
- --include-l3-loopback
- Include loopback ip-addresses in layer 3 topologies
- --remove-domain Remove the domain name from the L2 neighbor name before matching CMK host.
- --keep KEEP Number of topologies to keep. The oldest topologies above keep
+ INCLUDE/EXCLUDE site list "[FILTER_BY_SITE]" from TOML file.
+ --keep-max-topologies KEEP_MAX_TOPOLOGIES
+ Number of topologies to keep. The oldest topologies above keep
will be deleted.
NOTE: The default/protected topologies will be kept always.
- --min-age MIN_AGE The minimum number of days before a topology is deleted by "--keep"
+ --l2-case {INSENSITIVE,LOWER,UPPER,AUTO,OFF}
+ Change L2 neighbour name case before matching to Checkmk host
+ - OFF : Do not change the case of the neighbour name
+ - INSENSITIVE : search for a matching host by ignoring the case of neighbour name and host name
+ - LOWER : change to all lower case
+ - UPPER : change to all upper case, without the domain part of the neighbour name
+ - AUTO : try all the above variants
+ Default is let the case of the neighbour name untouched ("OFF").
+ Takes place after "--l2-remove-domain" and before "--l2-prefix"
+ --l2-display-ports Use L2 port names as display name for interfaces in L2 topologies
+ --l2-display-neighbours
+ Use L2 neighbour name as display name in L2 topologies
+ --l2-prefix L2_PREFIX
+ Prepends each L2 neighbour name with the prefix before matching to a Checkmk host name.
+ Takes place after "--l2-remove-domain" and "--l2-case"
+ --l2-remove-domain {OFF,ON,AUTO}
+ Handle the the domain name part of a neighbour name before matching it to a Checkmk host.
+ - OFF : dont touch the neighbour name, keep host name and domain part
+ - ON : will remove the domain part from the neighbour name, keep only the host name part
+ - AUTO : try all of the above variants
+ Default: "OFF". Takes place after "--l2-remove-domain" and before "--l2-prefix"
+ --l2-skip-external Skip L2 neighbours external to Checkmk (that have no matching host)
+ --l3-display-devices Use L3 device names as display name for interfaces in L3 topologies
+ --l3-include-hosts Include hosts (single IP objects) in layer 3 topologies
+ --l3-include-loopback
+ Include loopback ip-addresses in layer 3 topologies
+ --l3-skip-cidr-0 Skip ip-address with CIDR "/0" in layer 3 topologies
+ --l3-skip-cidr-32-128
+ Skip ip-address with CIDR "/32" or "/128" in layer 3 topologies
+ --l3-skip-if Dont show interface in layer 3 topologies
+ --l3-skip-ip Dont show ip-addresses in layer 3 topologies
+ --l3-skip-public Skip public ip-addresses in layer 3 topologies
+ --log-file LOG_FILE Set the log file. Default is "~/var/log/nvdct.log"
+ --log-level {CRITICAL,FATAL,ERROR,WARNING,INFO,DEBUG,OFF}
+ Sets the log level. The default is "WARNING"
+ --log-to-stdout Send log to stdout.
+ --min-topology-age MIN_TOPOLOGY_AGE
+ The minimum number of days before a topology is deleted by "--keep-max-topologies"
--pre-fetch Try to fetch host data, with less API calls. Can improve RESTAPI backend
performance
--quiet Suppress all output to stdtout
- --skip-l3-cidr-0 Skip ip-address with CIDR "/0" in layer 3 topologies
- --skip-l3-cidr-32-128
- Skip ip-address with CIDR "/32" or "/128" in layer 3 topologies
- --skip-l3-if Dont show interface in layer 3 topologies
- --skip-l3-ip Dont show ip-addresses in layer 3 topologies
- --skip-l3-public Skip public ip-addresses in layer 3 topologies
--time-format TIME_FORMAT
Format string to render the time. (default: "%Y-%m-%dT%H:%M:%S.%m")
+ --update-config Adjusts options in config file.
Exit codes:
0 - No error
@@ -186,7 +204,7 @@ Exit codes:
5 - No layer to work on
Usage:
-~/local/bin/nvdct/nvdct.py -u ~/local/bin/nvdct/conf/my_nvdct.toml
+~/local/bin/nvdct/nvdct.py -c ~/local/bin/nvdct/conf/my_nvdct.toml
OMD[build]:~$
```
@@ -205,7 +223,7 @@ OMD[build]:~$
To make the topology created by this tool the default use the option `-d` / `--default`
```
-$ ~/local/bin/nvdct/nvdct.py -u ~/local/bin/conf/my_nvdct.toml -d
+$ ~/local/bin/nvdct/nvdct.py -c ~/local/bin/conf/my_nvdct.toml -d
```
</details>
@@ -216,7 +234,7 @@ $ ~/local/bin/nvdct/nvdct.py -u ~/local/bin/conf/my_nvdct.toml -d
Yes. Add a cron job for the site user in `~/etc/cron.d`. I.e. to create a topology every hour create the file `nvdct` in `~/etc/cron.d`
```
-0 * * * * $OMD_ROOT/local/bin/nvdct/nvdct.py $OMD_ROOT/local/bin/nvdct/conf_my_nvdct.toml -d --keep 10 --min-age 14
+0 * * * * $OMD_ROOT/local/bin/nvdct/nvdct.py -c $OMD_ROOT/local/bin/nvdct/conf/my_nvdct.toml -d --keep-max-topologies 10 --min-topology-age 14
```
This will run the tool every hour and create a new topology if there was a changee. It keeps 10 topologies and deletes old topologies only if they are older than 14 days.
@@ -237,7 +255,7 @@ With crontab -l you get the list of active cron jobs
OMD[build]:~$ crontab -l | grep -A 3 -B 1 nvdct
# ------------------------------------------------------------
# /omd/sites/build/etc/cron.d/nvdct
-0 * * * * $OMD_ROOT/local/bin/nvdct/nvdct.py -u $OMD_ROOT/local/bin/nvdct/conf_my_nvdct.toml d --keep 10
+0 * * * * $OMD_ROOT/local/bin/nvdct/nvdct.py -c $OMD_ROOT/local/bin/nvdct/conf/my_nvdct.toml -d --keep-max-topologies 10 --min-topology-age 14
# ------------------------------------------------------------
```
@@ -252,7 +270,7 @@ OMD[build]:~$ crontab -l | grep -A 3 -B 1 nvdct
Yes. If you use the option --keep <number of topologies to keep> the tool will delete all topologies over this number. I.e.
```
-$ ~/local/bin/nvdct/nvdct.py -u ~/local/bin/conf/my_nvdct.toml --keep 10
+$ ~/local/bin/nvdct/nvdct.py -c ~/local/bin/conf/my_nvdct.toml --keep-max-topologies 10
```
will delete the oldest topologies until there only 10 topologies left.
@@ -262,9 +280,9 @@ will delete the oldest topologies until there only 10 topologies left.
<details><summary>I will not delete topologies from the last X days</summary>
\
-No problem. Just use the option `--min-age` togeher with `--keep`. I.e.
+No problem. Just use the option `--min-topology-age` togeher with `--keep-max-topologies`. I.e.
```
-$ ~/local/bin/nvdct/nvdct.py -u ~/local/bin/conf/my_nvdct.toml --keep 10 --min-age 30
+$ ~/local/bin/nvdct/nvdct.py -c ~/local/bin/conf/my_nvdct.toml --keep-max-topologies 10 --min-topology-age 30
```
will only delete a topology if it is older than 30 days.
</details>
@@ -272,12 +290,12 @@ will only delete a topology if it is older than 30 days.
<details><summary>Can the topology data creation be customized</summary>
\
-Yes you can and you sould. Create a copy of the default `~/local/bin/nvdct/conf/nvdct.toml` file. Or provide your own data file (see Option -u, --user-data-file).
+Yes you can and you sould. Create a copy of the default `~/local/bin/nvdct/conf/nvdct.toml` file. Or provide your own config file (see Option -c, --config).
**Note**: This file uses [Tom's Obvious Minimal Language](https://toml.io/en/)
```
-~/local/bin/nvdct/nvdct.py -u ~/local/bin/nvdct/conf/my_nvdct.toml
+~/local/bin/nvdct/nvdct.py -c ~/local/bin/nvdct/conf/my_nvdct.toml
```
</details>
@@ -285,11 +303,11 @@ Yes you can and you sould. Create a copy of the default `~/local/bin/nvdct/conf/
<details><summary>My Neighbor names from the inventory and checkmk host names do not match</summary>
\
-If the neighbor names from the inventory and the checkmk host names do not match, the topology does not work. To fix this, add a mapping between the neighbor name and the checkmk host name to the **HOST_MAP** in the data file (by default `nvdct.toml`). I.e.
+If the neighbor names from the inventory and the checkmk host names do not match, the topology does not work. To fix this, add a mapping between the neighbor name and the checkmk host name to the **HOST_MAP** in the config file (by default `nvdct.toml`). I.e.
```
# map inventory neighbour name to Checkmk host name
-[L2_HOST_MAP]
+[L2_NEIGHBOUR_TO_HOST_MAP]
inventory_neighbour1 = "cmk_host1"
inventory_neighbour2 = "cmk_host2"
inventory_neighbour3 = "cmk_host3"
@@ -300,11 +318,11 @@ inventory_neighbour3 = "cmk_host3"
<details><summary>In my inventory are invalid neighbor names</summary>
\
-If you have invalid neighbor names in the inventory (i.e. "not advertised"), you can drop them by adding them to the **L2_DROP_HOSTS** list in the data file (by default `nvdct.toml`). I.e..
+If you have invalid neighbor names in the inventory (i.e. "not advertised"), you can drop them by adding them to the **L2_DROP_NEIGHBOURS** list in the configuration file (by default `nvdct.toml`). I.e..
```
# drop neighbours with invalid names
-L2_DROP_HOSTS = [
+L2_DROP_NEIGHBOURS = [
"not advertised",
"a nother invalid name",
]
@@ -347,21 +365,7 @@ STATIC_CONNECTIONS = [
<details><summary>Can I use the tool with other inventory data plugins</summary>
\
-**This option is decrepated und will be removed soon.**
-
-
-Yes. If you don\`t use my plugins you can still use this tool to create your topology data, for this you need the path to your topology data in the inventory. For example for my CDP plugin this is _Networking_ -> _CDP Cache_ -> _CDP neighbours_ internally known as `networking` -> `cdp_cache` -> `neighbours`. The data must form a table in the inventory that contains at least the three columns for the neighbor name, neighbor port, and local port. For these columns you need the internal names. I.e. for my CDP plugin the columns _Neighbor name_, _Neighbor port_ and _Local port_ are known internally as `neighbour_name`, `local_port` and `neighbour_port`. With this information, you can edit the CUSTOM_LAYERS section in the user data file like this
-```
-# optional custom layers use option -l/--layers CUSTOM to include this layers
-CUSTOM_LAYERS = [
- { path = "networking,lldp_cache.neighbours", columns = "neighbour_name,local_port_num,neighbour_port", label = "custom_LLDP" },
- { path = "networking,cdp_cache.neighbours", columns = "neighbour_name,local_port,neighbour_port", label = "custom_CDP" },
-]
-```
-then run the tool with the option `-l CUSTOM`
-```
-~/local/bin/network-topology/nvdct.py -u ~/local/bin/conf/my_nvdct.toml -l CUSTOM
-```
+No.
</details>
@@ -389,7 +393,7 @@ For more troubleshooting see [TROUBLESHOOTING.md](/TROUBLESHOOTING.md)
### Sample Output
```
-OMD[build]:~$ ~/local/bin/nvdct/nvdct.py -u ~/local/bin/nvdct/conf/lrm.toml -d
+OMD[build]:~$ ~/local/bin/nvdct/nvdct.py -c ~/local/bin/nvdct/conf/my_nvdct.toml -d
Network Visualisation Data Creation Tool (NVDCT)
by thl-cmk[at]outlook[dot]com, version 0.9.2-20241117
--
GitLab