mirror of
https://github.com/tcivie/meshtastic-metrics-exporter.git
synced 2025-03-05 20:52:02 -08:00
Compare commits
21 commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
ecb19a4735 | ||
|
|
868c2bcab5 | ||
|
|
6a7aa6c112 | ||
|
|
60002b20e9 | ||
|
|
ae6f8b4160 | ||
|
|
b78deaf95a | ||
|
|
92210ee24a | ||
|
|
5183ade98b | ||
|
|
3fb222de43 | ||
|
0281d0b990 | ||
|
|
0650382405 | ||
|
|
2cfbcaa8cd | ||
|
|
b92b3da4cd | ||
|
|
7b32cb57da | ||
|
|
1fc4096772 | ||
|
|
577900faf3 | ||
|
|
8eb6184763 | ||
|
|
44549c75a6 | ||
|
|
bc4ff2d6f5 | ||
|
|
c46049f866 | ||
|
|
e73837ac6b |
2
.env
2
.env
|
|
@ -5,7 +5,7 @@ DATABASE_URL=postgres://postgres:postgres@postgres:5432/meshtastic
|
||||||
|
|
||||||
# Prometheus connection details
|
# Prometheus connection details
|
||||||
PROMETHEUS_COLLECTOR_PORT=9464
|
PROMETHEUS_COLLECTOR_PORT=9464
|
||||||
PROMETHEUS_JOB=example
|
PROMETHEUS_JOB=meshtastic
|
||||||
|
|
||||||
# MQTT connection details
|
# MQTT connection details
|
||||||
MQTT_HOST=mqtt.meshtastic.org
|
MQTT_HOST=mqtt.meshtastic.org
|
||||||
|
|
|
||||||
|
|
@ -9,5 +9,6 @@
|
||||||
</component>
|
</component>
|
||||||
<component name="PackageRequirementsSettings">
|
<component name="PackageRequirementsSettings">
|
||||||
<option name="versionSpecifier" value="Greater or equal (>=x.y.z)" />
|
<option name="versionSpecifier" value="Greater or equal (>=x.y.z)" />
|
||||||
|
<option name="keepMatchingSpecifier" value="false" />
|
||||||
</component>
|
</component>
|
||||||
</module>
|
</module>
|
||||||
237
README.md
237
README.md
|
|
@ -1,38 +1,141 @@
|
||||||
# Meshtastic Metrics Exporter
|
# Meshtastic Metrics Exporter
|
||||||
[](https://github.com/tcivie/meshtastic-metrics-exporter/actions/workflows/github-code-scanning/codeql)
|
[](https://github.com/tcivie/meshtastic-metrics-exporter/actions/workflows/github-code-scanning/codeql)
|
||||||
|
|
||||||
The `meshtastic-metrics-exporter` is a tool designed to export nearly all available data from an MQTT server to a
|
The `meshtastic-metrics-exporter` is a tool designed to export nearly all available data from an MQTT server
|
||||||
Prometheus server. It comes with a pre-configured Grafana dashboard connected to both data sources, allowing users to
|
to a Prometheus server. It comes with a pre-configured Grafana dashboard connected to both data sources,
|
||||||
start creating dashboards immediately.
|
allowing users to start creating dashboards immediately.
|
||||||
|
|
||||||
|
## Public Dashboards
|
||||||
|
|
||||||
|
You can explore these public instances to see the exporter in action:
|
||||||
|
|
||||||
|
- **Canadaverse Dashboard**: [dash.mt.gt](https://dash.mt.gt) (Guest access: username: `guest`, password: `guest`)
|
||||||
|
> This instance demonstrates the metrics exporter's capabilities in a production environment, maintained by [@tb0hdan](https://github.com/tb0hdan).
|
||||||
|
|
||||||
## Features
|
## Features
|
||||||
|
|
||||||
- Exports a comprehensive set of metrics from an MQTT server to Prometheus.
|
- Exports a comprehensive set of metrics from an MQTT server to Prometheus.
|
||||||
- Comes with a Grafana dashboard configured to connect to both Prometheus and Postgres data sources.
|
- Comes with a Grafana dashboard configured to connect to both Prometheus and Postgres data sources.
|
||||||
- Comes with some basic dashboards, see the section below for general view of the dashboards
|
- Comes with some basic dashboards, see the section below for general view of the dashboards
|
||||||
- Stores node details (ID, short/long name, hardware details, and client type) in a Postgres server, which is also part of
|
- Stores node details (ID, short/long name, hardware details, and client type) in a Postgres server, which is also part
|
||||||
the package.
|
of the package.
|
||||||
- Configuration via a `.env` file.
|
- Configuration via a `.env` file.
|
||||||
|
|
||||||
### Grafana Dashboards
|
### Database Structure
|
||||||
The project comes wtih 2 dashboards.
|
|
||||||
#### Main Dashboard
|
|
||||||
<img width="1514" alt="SCR-20240707-qgnn" src="https://github.com/tcivie/meshtastic-metrics-exporter/assets/87943721/9679c140-c5f7-4ea5-bfc6-0173b52fb28c">
|
|
||||||
|
|
||||||
> The dashboard has some basic data about the mesh network and it's data is temporarely updated (With new data coming in it would fill out the missing pieces automatically)
|
The system uses PostgreSQL with the following tables:
|
||||||
|
|
||||||
|
#### 1. messages
|
||||||
|
|
||||||
|
- Stores message IDs and timestamps
|
||||||
|
- Auto-expires messages older than 1 minute using a trigger
|
||||||
|
|
||||||
|
```sql
|
||||||
|
Columns:
|
||||||
|
- id (TEXT, PRIMARY KEY)
|
||||||
|
- received_at (TIMESTAMP)
|
||||||
|
```
|
||||||
|
|
||||||
|
#### 2. node_details
|
||||||
|
|
||||||
|
- Stores basic information about mesh nodes
|
||||||
|
|
||||||
|
```sql
|
||||||
|
Columns:
|
||||||
|
- node_id (VARCHAR, PRIMARY KEY)
|
||||||
|
- short_name (VARCHAR)
|
||||||
|
- long_name (VARCHAR)
|
||||||
|
- hardware_model (VARCHAR)
|
||||||
|
- role (VARCHAR)
|
||||||
|
- mqtt_status (VARCHAR, default 'none')
|
||||||
|
- longitude (INT)
|
||||||
|
- latitude (INT)
|
||||||
|
- altitude (INT)
|
||||||
|
- precision (INT)
|
||||||
|
- created_at (TIMESTAMP)
|
||||||
|
- updated_at (TIMESTAMP)
|
||||||
|
```
|
||||||
|
|
||||||
|
#### 3. node_neighbors
|
||||||
|
|
||||||
|
- Tracks connections between nodes
|
||||||
|
|
||||||
|
```sql
|
||||||
|
Columns:
|
||||||
|
- id (SERIAL, PRIMARY KEY)
|
||||||
|
- node_id (VARCHAR, FOREIGN KEY)
|
||||||
|
- neighbor_id (VARCHAR, FOREIGN KEY)
|
||||||
|
- snr (FLOAT)
|
||||||
|
```
|
||||||
|
|
||||||
|
#### 4. node_configurations
|
||||||
|
|
||||||
|
- Stores detailed configuration and timing information for nodes
|
||||||
|
|
||||||
|
```sql
|
||||||
|
Columns:
|
||||||
|
- node_id (VARCHAR, PRIMARY KEY)
|
||||||
|
- last_updated (TIMESTAMP)
|
||||||
|
- environment_update_interval (INTERVAL)
|
||||||
|
- environment_update_last_timestamp (TIMESTAMP)
|
||||||
|
- device_update_interval (INTERVAL)
|
||||||
|
- device_update_last_timestamp (TIMESTAMP)
|
||||||
|
- air_quality_update_interval (INTERVAL)
|
||||||
|
- air_quality_update_last_timestamp (TIMESTAMP)
|
||||||
|
- power_update_interval (INTERVAL)
|
||||||
|
- power_update_last_timestamp (TIMESTAMP)
|
||||||
|
- range_test_interval (INTERVAL)
|
||||||
|
- range_test_packets_total (INT)
|
||||||
|
- range_test_first_packet_timestamp (TIMESTAMP)
|
||||||
|
- range_test_last_packet_timestamp (TIMESTAMP)
|
||||||
|
- pax_counter_interval (INTERVAL)
|
||||||
|
- pax_counter_last_timestamp (TIMESTAMP)
|
||||||
|
- neighbor_info_interval (INTERVAL)
|
||||||
|
- neighbor_info_last_timestamp (TIMESTAMP)
|
||||||
|
- mqtt_encryption_enabled (BOOLEAN)
|
||||||
|
- mqtt_json_enabled (BOOLEAN)
|
||||||
|
- mqtt_json_message_timestamp (TIMESTAMP)
|
||||||
|
- mqtt_configured_root_topic (TEXT)
|
||||||
|
- mqtt_info_last_timestamp (TIMESTAMP)
|
||||||
|
- map_broadcast_interval (INTERVAL)
|
||||||
|
- map_broadcast_last_timestamp (TIMESTAMP)
|
||||||
|
```
|
||||||
|
|
||||||
|
### Grafana Dashboards
|
||||||
|
|
||||||
|
The project comes with 2 dashboards.
|
||||||
|
|
||||||
|
#### Main Dashboard
|
||||||
|
|
||||||
|
<img width="1470" alt="image" src="https://github.com/user-attachments/assets/09fe72e5-23eb-4516-9f34-19e2cc38b7dc">
|
||||||
|
|
||||||
|
> The dashboard has some basic data about the mesh network and its data is temporarily updated
|
||||||
|
> (With new data coming in it would fill out the missing pieces automatically)
|
||||||
|
|
||||||
|
**Note:** The dashboard contains links to nodes that target `localhost:3000`.
|
||||||
|
If you're accessing Grafana from a different host, you'll need to modify these links in the panel configuration
|
||||||
|
to match your Grafana server's address.
|
||||||
|
|
||||||
#### User Panel
|
#### User Panel
|
||||||
<img width="1470" alt="SCR-20240707-qhth" src="https://github.com/tcivie/meshtastic-metrics-exporter/assets/87943721/58f15190-127d-4481-b896-1c3e2121dea5">
|
|
||||||
|
|
||||||
> This panel can be reached from the "Node ID" link on the main dashboard (The table in the center) or you can go to it from the dashbaords tab in grafana and select the node you want to spectate. This board includes some telemetry data and basic information about the node.
|
|
||||||
|
|
||||||
|
|
||||||
|
> This panel can be reached from the "Node ID" link on the main dashboard (The table in the center)
|
||||||
|
> or you can go to it from the dashboards tab in Grafana and select the node you want to spectate.
|
||||||
|
> This board includes some telemetry data and basic information about the node.
|
||||||
|
|
||||||
#### The Node Graph
|
#### The Node Graph
|
||||||
|
|
||||||
<img width="585" alt="SCR-20240707-qjaj" src="https://github.com/tcivie/meshtastic-metrics-exporter/assets/87943721/d29b2ac4-6291-4095-9938-e6e63df15098">
|
<img width="585" alt="SCR-20240707-qjaj" src="https://github.com/tcivie/meshtastic-metrics-exporter/assets/87943721/d29b2ac4-6291-4095-9938-e6e63df15098">
|
||||||
|
|
||||||
> Both boards also include node graph which allows you to view nodes which are sending [Neighbour Info packets](https://meshtastic.org/docs/configuration/module/neighbor-info)
|
> Both boards also include node graph which allows you to view nodes which are sending [Neighbour Info packets](https://meshtastic.org/docs/configuration/module/neighbor-info)
|
||||||
> As long as we have some node which is connected to our MQTT server the data would be read buy the exporter and parsed as node graph. The line colors indicate the SNR value and the arrow is the direction of the flow captured (It can be two way). And the node circle color indicates which node is connected to MQTT (Green) which one is disconnected from MQTT (Red) and unknown (Gray - Never connected to the MQTT server)
|
> As long as we have some node which is connected to our MQTT server the data would be read by the exporter
|
||||||
|
> and parsed as node graph. The line colors indicate the SNR value and the arrow is the direction of the flow captured
|
||||||
|
> (It can be two way). And the node circle color indicates which node is connected to MQTT (Green)
|
||||||
|
> which one is disconnected from MQTT (Red) and unknown (Gray - Never connected to the MQTT server)
|
||||||
|
|
||||||
**I highly recomend giving the system to stabilize over 24 hours before seeking any useful information from it.**
|
**It is highly recommended to give the system 24 hours to stabilize before seeking any useful information from it.**
|
||||||
|
|
||||||
## Exported Metrics
|
## Exported Metrics
|
||||||
|
|
||||||
|
|
@ -43,65 +146,55 @@ Label Notation:
|
||||||
- 🏷️ (source): Indicates that all common labels are used, prefixed with "source_" (e.g., source_node_id, source_short_name, etc.).
|
- 🏷️ (source): Indicates that all common labels are used, prefixed with "source_" (e.g., source_node_id, source_short_name, etc.).
|
||||||
- 🏷️ (destination): Indicates that all common labels are used, prefixed with "destination_" (e.g., destination_node_id, destination_short_name, etc.).
|
- 🏷️ (destination): Indicates that all common labels are used, prefixed with "destination_" (e.g., destination_node_id, destination_short_name, etc.).
|
||||||
|
|
||||||
The following is a list of metrics exported by the `meshtastic-metrics-exporter`:
|
### Available Metrics
|
||||||
|
|
||||||
| Metric Name | Description | Type | Labels |
|
| Metric Name | Description | Type | Labels |
|
||||||
|-----------------------------------|------------------------------------------------------------------------------|-----------|--------------------------------------|
|
|------------------------------------------------|--------------------------------------------------|-----------|--------------------------------------------|
|
||||||
| text_message_app_length | Length of text messages processed by the app | Histogram | 🏷️ |
|
| text_message_app_length | Length of text messages processed by the app | Histogram | 🏷️ |
|
||||||
| device_latitude | Device latitude | Gauge | 🏷️ |
|
| device_latitude | Device latitude | Gauge | 🏷️ |
|
||||||
| device_longitude | Device longitude | Gauge | 🏷️ |
|
| device_longitude | Device longitude | Gauge | 🏷️ |
|
||||||
| device_altitude | Device altitude | Gauge | 🏷️ |
|
| device_altitude | Device altitude | Gauge | 🏷️ |
|
||||||
| device_position_precision | Device position precision | Gauge | 🏷️ |
|
| device_position_precision | Device position precision | Gauge | 🏷️ |
|
||||||
| telemetry_app_ch1_voltage | Voltage measured by the device on channel 1 | Gauge | 🏷️ |
|
| telemetry_app_ch[1-3]_voltage | Voltage measured by the device on channels 1-3 | Gauge | 🏷️ |
|
||||||
| telemetry_app_ch1_current | Current measured by the device on channel 1 | Gauge | 🏷️ |
|
| telemetry_app_ch[1-3]_current | Current measured by the device on channels 1-3 | Gauge | 🏷️ |
|
||||||
| telemetry_app_ch2_voltage | Voltage measured by the device on channel 2 | Gauge | 🏷️ |
|
| telemetry_app_pm[10/25/100]_standard | Concentration Units Standard PM1.0/2.5/10.0 | Gauge | 🏷️ |
|
||||||
| telemetry_app_ch2_current | Current measured by the device on channel 2 | Gauge | 🏷️ |
|
| telemetry_app_pm[10/25/100]_environmental | Concentration Units Environmental PM1.0/2.5/10.0 | Gauge | 🏷️ |
|
||||||
| telemetry_app_ch3_voltage | Voltage measured by the device on channel 3 | Gauge | 🏷️ |
|
| telemetry_app_particles_[03/05/10/25/50/100]um | Particle Count for different sizes | Gauge | 🏷️ |
|
||||||
| telemetry_app_ch3_current | Current measured by the device on channel 3 | Gauge | 🏷️ |
|
| telemetry_app_temperature | Temperature measured by the device | Gauge | 🏷️ |
|
||||||
| telemetry_app_pm10_standard | Concentration Units Standard PM1.0 | Gauge | 🏷️ |
|
| telemetry_app_relative_humidity | Relative humidity percent | Gauge | 🏷️ |
|
||||||
| telemetry_app_pm25_standard | Concentration Units Standard PM2.5 | Gauge | 🏷️ |
|
| telemetry_app_barometric_pressure | Barometric pressure in hPA | Gauge | 🏷️ |
|
||||||
| telemetry_app_pm100_standard | Concentration Units Standard PM10.0 | Gauge | 🏷️ |
|
| telemetry_app_gas_resistance | Gas resistance in MOhm | Gauge | 🏷️ |
|
||||||
| telemetry_app_pm10_environmental | Concentration Units Environmental PM1.0 | Gauge | 🏷️ |
|
| telemetry_app_iaq | IAQ value (0-500) | Gauge | 🏷️ |
|
||||||
| telemetry_app_pm25_environmental | Concentration Units Environmental PM2.5 | Gauge | 🏷️ |
|
| telemetry_app_distance | Distance in mm | Gauge | 🏷️ |
|
||||||
| telemetry_app_pm100_environmental | Concentration Units Environmental PM10.0 | Gauge | 🏷️ |
|
| telemetry_app_lux | Ambient light in Lux | Gauge | 🏷️ |
|
||||||
| telemetry_app_particles_03um | 0.3um Particle Count | Gauge | 🏷️ |
|
| telemetry_app_white_lux | White light in Lux | Gauge | 🏷️ |
|
||||||
| telemetry_app_particles_05um | 0.5um Particle Count | Gauge | 🏷️ |
|
| telemetry_app_ir_lux | Infrared light in Lux | Gauge | 🏷️ |
|
||||||
| telemetry_app_particles_10um | 1.0um Particle Count | Gauge | 🏷️ |
|
| telemetry_app_uv_lux | Ultraviolet light in Lux | Gauge | 🏷️ |
|
||||||
| telemetry_app_particles_25um | 2.5um Particle Count | Gauge | 🏷️ |
|
| telemetry_app_wind_direction | Wind direction in degrees | Gauge | 🏷️ |
|
||||||
| telemetry_app_particles_50um | 5.0um Particle Count | Gauge | 🏷️ |
|
| telemetry_app_wind_speed | Wind speed in m/s | Gauge | 🏷️ |
|
||||||
| telemetry_app_particles_100um | 10.0um Particle Count | Gauge | 🏷️ |
|
| telemetry_app_weight | Weight in KG | Gauge | 🏷️ |
|
||||||
| telemetry_app_temperature | Temperature measured by the device | Gauge | 🏷️ |
|
| telemetry_app_battery_level | Battery level (0-100, >100 means powered) | Gauge | 🏷️ |
|
||||||
| telemetry_app_relative_humidity | Relative humidity percent measured by the device | Gauge | 🏷️ |
|
| telemetry_app_voltage | Voltage | Gauge | 🏷️ |
|
||||||
| telemetry_app_barometric_pressure | Barometric pressure in hPA measured by the device | Gauge | 🏷️ |
|
| telemetry_app_channel_utilization | Channel utilization including TX, RX, and noise | Gauge | 🏷️ |
|
||||||
| telemetry_app_gas_resistance | Gas resistance in MOhm measured by the device | Gauge | 🏷️ |
|
| telemetry_app_air_util_tx | Airtime utilization for TX in last hour | Gauge | 🏷️ |
|
||||||
| telemetry_app_iaq | IAQ value measured by the device (0-500) | Gauge | 🏷️ |
|
| telemetry_app_uptime_seconds | Device uptime in seconds | Counter | 🏷️ |
|
||||||
| telemetry_app_distance | Distance measured by the device in mm | Gauge | 🏷️ |
|
| route_length | Number of nodes in route | Counter | 🏷️ |
|
||||||
| telemetry_app_lux | Ambient light measured by the device in Lux | Gauge | 🏷️ |
|
| route_response | Number of route discovery responses | Counter | 🏷️, response_type |
|
||||||
| telemetry_app_white_lux | White light measured by the device in Lux | Gauge | 🏷️ |
|
| mesh_packet_source_types | Mesh packet types by source | Counter | 🏷️ (source), portnum |
|
||||||
| telemetry_app_ir_lux | Infrared light measured by the device in Lux | Gauge | 🏷️ |
|
| mesh_packet_destination_types | Mesh packet types by destination | Counter | 🏷️ (destination), portnum |
|
||||||
| telemetry_app_uv_lux | Ultraviolet light measured by the device in Lux | Gauge | 🏷️ |
|
| mesh_packet_total | Total mesh packets processed | Counter | 🏷️ (source), 🏷️ (destination) |
|
||||||
| telemetry_app_wind_direction | Wind direction in degrees measured by the device | Gauge | 🏷️ |
|
| mesh_packet_rx_time | Packet receive time (seconds since 1970) | Histogram | 🏷️ (source), 🏷️ (destination) |
|
||||||
| telemetry_app_wind_speed | Wind speed in m/s measured by the device | Gauge | 🏷️ |
|
| mesh_packet_rx_snr | Packet receive SNR | Gauge | 🏷️ (source), 🏷️ (destination) |
|
||||||
| telemetry_app_weight | Weight in KG measured by the device | Gauge | 🏷️ |
|
| mesh_packet_hop_limit | Packet hop limit | Counter | 🏷️ (source), 🏷️ (destination) |
|
||||||
| telemetry_app_battery_level | Battery level of the device (0-100, >100 means powered) | Gauge | 🏷️ |
|
| mesh_packet_want_ack | Want ACK occurrences | Counter | 🏷️ (source), 🏷️ (destination) |
|
||||||
| telemetry_app_voltage | Voltage measured by the device | Gauge | 🏷️ |
|
| mesh_packet_via_mqtt | MQTT transmission occurrences | Counter | 🏷️ (source), 🏷️ (destination) |
|
||||||
| telemetry_app_channel_utilization | Utilization for the current channel, including well-formed TX, RX, and noise | Gauge | 🏷️ |
|
| mesh_packet_hop_start | Packet hop start | Gauge | 🏷️ (source), 🏷️ (destination) |
|
||||||
| telemetry_app_air_util_tx | Percent of airtime for transmission used within the last hour | Gauge | 🏷️ |
|
| mesh_packet_ids | Unique packet IDs | Counter | 🏷️ (source), 🏷️ (destination), packet_id |
|
||||||
| telemetry_app_uptime_seconds | How long the device has been running since the last reboot (in seconds) | Counter | 🏷️ |
|
| mesh_packet_channel | Packet channel | Counter | 🏷️ (source), 🏷️ (destination), channel |
|
||||||
| route_length | Number of nodes in the route | Counter | 🏷️ |
|
| mesh_packet_rx_rssi | Packet receive RSSI | Gauge | 🏷️ (source), 🏷️ (destination) |
|
||||||
| route_response | Number of responses to route discovery | Counter | 🏷️, response_type |
|
| pax_wifi | Number of Wifi devices (PAX) | Gauge | 🏷 |
|
||||||
| mesh_packet_source_types | Types of mesh packets processed by source | Counter | 🏷️ (source), portnum |
|
| pax_ble | Number of Bluetooth devices (PAX) | Gauge | 🏷 |
|
||||||
| mesh_packet_destination_types | Types of mesh packets processed by destination | Counter | 🏷️ (destination), portnum |
|
| pax_uptime | PAX device uptime | Gauge | 🏷 |
|
||||||
| mesh_packet_total | Total number of mesh packets processed | Counter | 🏷️ (source), 🏷️ (destination) |
|
|
||||||
| mesh_packet_rx_time | Receive time of mesh packets (seconds since 1970) | Histogram | 🏷️ (source), 🏷️ (destination) |
|
|
||||||
| mesh_packet_rx_snr | Receive SNR of mesh packets | Gauge | 🏷️ (source), 🏷️ (destination) |
|
|
||||||
| mesh_packet_hop_limit | Hop limit of mesh packets | Counter | 🏷️ (source), 🏷️ (destination) |
|
|
||||||
| mesh_packet_want_ack | Occurrences of want ACK for mesh packets | Counter | 🏷️ (source), 🏷️ (destination) |
|
|
||||||
| mesh_packet_via_mqtt | Occurrences of mesh packets sent via MQTT | Counter | 🏷️ (source), 🏷️ (destination) |
|
|
||||||
| mesh_packet_hop_start | Hop start of mesh packets | Gauge | 🏷️ (source), 🏷️ (destination) |
|
|
||||||
| mesh_packet_ids | Unique IDs for mesh packets | Counter | 🏷️ (source), 🏷️ (destination), packet_id |
|
|
||||||
| mesh_packet_channel | Channel used for mesh packets | Counter | 🏷️ (source), 🏷️ (destination), channel |
|
|
||||||
| mesh_packet_rx_rssi | Receive RSSI of mesh packets | Gauge | 🏷️ (source), 🏷️ (destination) |
|
|
||||||
|
|
||||||
## Configuration
|
## Configuration
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1,4 +1,5 @@
|
||||||
from datetime import datetime
|
from datetime import datetime
|
||||||
|
|
||||||
from prometheus_client import CollectorRegistry, Counter, Gauge
|
from prometheus_client import CollectorRegistry, Counter, Gauge
|
||||||
|
|
||||||
from exporter.client_details import ClientDetails
|
from exporter.client_details import ClientDetails
|
||||||
|
|
@ -35,6 +36,7 @@ class Metrics:
|
||||||
self._init_metrics_telemetry_air_quality()
|
self._init_metrics_telemetry_air_quality()
|
||||||
self._init_metrics_telemetry_power()
|
self._init_metrics_telemetry_power()
|
||||||
self._init_route_discovery_metrics()
|
self._init_route_discovery_metrics()
|
||||||
|
self._init_pax_counter_metrics()
|
||||||
|
|
||||||
def update_metrics_position(self, latitude, longitude, altitude, precision, client_details: ClientDetails):
|
def update_metrics_position(self, latitude, longitude, altitude, precision, client_details: ClientDetails):
|
||||||
# Could be used to calculate more complex data (Like distances etc..)
|
# Could be used to calculate more complex data (Like distances etc..)
|
||||||
|
|
@ -328,3 +330,23 @@ class Metrics:
|
||||||
self._get_common_labels() + ['response_type'],
|
self._get_common_labels() + ['response_type'],
|
||||||
registry=self._registry
|
registry=self._registry
|
||||||
)
|
)
|
||||||
|
|
||||||
|
def _init_pax_counter_metrics(self):
|
||||||
|
self.pax_wifi_gauge = Gauge(
|
||||||
|
'pax_wifi',
|
||||||
|
'Number of WiFi devices',
|
||||||
|
self._get_common_labels(),
|
||||||
|
registry=self._registry
|
||||||
|
)
|
||||||
|
self.pax_ble_gauge = Gauge(
|
||||||
|
'pax_ble',
|
||||||
|
'Number of BLE devices',
|
||||||
|
self._get_common_labels(),
|
||||||
|
registry=self._registry
|
||||||
|
)
|
||||||
|
self.pax_uptime_gauge = Gauge(
|
||||||
|
'pax_uptime',
|
||||||
|
'Uptime of the device',
|
||||||
|
self._get_common_labels(),
|
||||||
|
registry=self._registry
|
||||||
|
)
|
||||||
|
|
|
||||||
100
exporter/metric_cleanup_job.py
Normal file
100
exporter/metric_cleanup_job.py
Normal file
|
|
@ -0,0 +1,100 @@
|
||||||
|
from datetime import datetime, timedelta
|
||||||
|
from typing import Any, Dict, Tuple
|
||||||
|
|
||||||
|
from apscheduler.schedulers.background import BackgroundScheduler
|
||||||
|
from prometheus_client import CollectorRegistry
|
||||||
|
|
||||||
|
|
||||||
|
class TrackedMetricsDict(dict):
|
||||||
|
"""A dictionary that tracks updates for metrics"""
|
||||||
|
|
||||||
|
def __init__(self, collector, metric_tracker, *args, **kwargs):
|
||||||
|
super().__init__(*args, **kwargs)
|
||||||
|
self._collector = collector
|
||||||
|
self._metric_tracker = metric_tracker
|
||||||
|
|
||||||
|
def __setitem__(self, key, value):
|
||||||
|
super().__setitem__(key, value)
|
||||||
|
self._metric_tracker.update_metric_timestamp(self._collector, key)
|
||||||
|
|
||||||
|
|
||||||
|
class MetricTrackingRegistry(CollectorRegistry):
|
||||||
|
"""Extended CollectorRegistry that tracks metric updates"""
|
||||||
|
|
||||||
|
def __init__(self, *args, **kwargs):
|
||||||
|
super().__init__(*args, **kwargs)
|
||||||
|
self._metric_tracker = MetricCleanupJob(self)
|
||||||
|
self._metric_tracker.start()
|
||||||
|
|
||||||
|
def register(self, collector):
|
||||||
|
"""Override register to add update tracking to collectors"""
|
||||||
|
super().register(collector)
|
||||||
|
if hasattr(collector, '_metrics'):
|
||||||
|
# Replace the metrics dict with our tracking version
|
||||||
|
tracked_dict = TrackedMetricsDict(
|
||||||
|
collector,
|
||||||
|
self._metric_tracker,
|
||||||
|
collector._metrics
|
||||||
|
)
|
||||||
|
collector._metrics = tracked_dict
|
||||||
|
|
||||||
|
def __del__(self):
|
||||||
|
"""Ensure cleanup job is stopped when registry is destroyed"""
|
||||||
|
if hasattr(self, '_metric_tracker'):
|
||||||
|
self._metric_tracker.stop()
|
||||||
|
|
||||||
|
class MetricCleanupJob:
|
||||||
|
def __init__(self, registry: CollectorRegistry):
|
||||||
|
self.registry = registry
|
||||||
|
self.scheduler = BackgroundScheduler()
|
||||||
|
self.last_updates: Dict[Tuple[Any, Any], datetime] = {}
|
||||||
|
|
||||||
|
def start(self):
|
||||||
|
"""Start the cleanup job to run every hour"""
|
||||||
|
self.scheduler.add_job(
|
||||||
|
self.cleanup_stale_metrics,
|
||||||
|
'interval',
|
||||||
|
minutes=10,
|
||||||
|
next_run_time=datetime.now() + timedelta(minutes=1)
|
||||||
|
)
|
||||||
|
self.scheduler.start()
|
||||||
|
|
||||||
|
def stop(self):
|
||||||
|
"""Stop the cleanup job"""
|
||||||
|
self.scheduler.shutdown()
|
||||||
|
|
||||||
|
def update_metric_timestamp(self, collector, labels):
|
||||||
|
"""Update the last modification time for a metric"""
|
||||||
|
metric_key = (collector, labels)
|
||||||
|
self.last_updates[metric_key] = datetime.now()
|
||||||
|
|
||||||
|
def cleanup_stale_metrics(self):
|
||||||
|
"""Remove metric entries that haven't been updated in 24 hours"""
|
||||||
|
try:
|
||||||
|
current_time = datetime.now()
|
||||||
|
stale_threshold = current_time - timedelta(hours=24)
|
||||||
|
|
||||||
|
for collector, _ in list(self.registry._collector_to_names.items()):
|
||||||
|
if hasattr(collector, '_metrics'):
|
||||||
|
labels_to_remove = []
|
||||||
|
|
||||||
|
for labels, _ in list(collector._metrics.items()):
|
||||||
|
metric_key = (collector, labels)
|
||||||
|
last_update = self.last_updates.get(metric_key)
|
||||||
|
|
||||||
|
if last_update is None or last_update < stale_threshold:
|
||||||
|
labels_to_remove.append(labels)
|
||||||
|
|
||||||
|
for labels in labels_to_remove:
|
||||||
|
try:
|
||||||
|
del collector._metrics[labels]
|
||||||
|
metric_key = (collector, labels)
|
||||||
|
self.last_updates.pop(metric_key, None)
|
||||||
|
print(f"Removed stale metric entry with labels: {labels}")
|
||||||
|
except KeyError:
|
||||||
|
pass
|
||||||
|
except Exception as e:
|
||||||
|
print(f"Error removing metric entry: {e}")
|
||||||
|
|
||||||
|
except Exception as e:
|
||||||
|
print(f"Error during metric cleanup: {e}")
|
||||||
|
|
@ -5,16 +5,17 @@ import sys
|
||||||
|
|
||||||
from cryptography.hazmat.backends import default_backend
|
from cryptography.hazmat.backends import default_backend
|
||||||
from cryptography.hazmat.primitives.ciphers import Cipher, algorithms, modes
|
from cryptography.hazmat.primitives.ciphers import Cipher, algorithms, modes
|
||||||
from meshtastic.protobuf.mqtt_pb2 import ServiceEnvelope
|
|
||||||
|
|
||||||
from exporter.metric.node_configuration_metrics import NodeConfigurationMetrics
|
from exporter.metric.node_configuration_metrics import NodeConfigurationMetrics
|
||||||
|
|
||||||
try:
|
try:
|
||||||
from meshtastic.mesh_pb2 import MeshPacket, Data, HardwareModel
|
from meshtastic.mesh_pb2 import MeshPacket, Data, HardwareModel
|
||||||
from meshtastic.portnums_pb2 import PortNum
|
from meshtastic.portnums_pb2 import PortNum
|
||||||
|
from meshtastic.mqtt_pb2 import ServiceEnvelope
|
||||||
except ImportError:
|
except ImportError:
|
||||||
from meshtastic.protobuf.mesh_pb2 import MeshPacket, Data, HardwareModel
|
from meshtastic.protobuf.mesh_pb2 import MeshPacket, Data, HardwareModel
|
||||||
from meshtastic.protobuf.portnums_pb2 import PortNum
|
from meshtastic.protobuf.portnums_pb2 import PortNum
|
||||||
|
from meshtastic.protobuf.mqtt_pb2 import ServiceEnvelope
|
||||||
|
|
||||||
from prometheus_client import CollectorRegistry, Counter, Gauge
|
from prometheus_client import CollectorRegistry, Counter, Gauge
|
||||||
from psycopg_pool import ConnectionPool
|
from psycopg_pool import ConnectionPool
|
||||||
|
|
@ -149,14 +150,15 @@ class MessageProcessor:
|
||||||
def process_json_mqtt(message):
|
def process_json_mqtt(message):
|
||||||
topic = message.topic
|
topic = message.topic
|
||||||
json_packet = json.loads(message.payload)
|
json_packet = json.loads(message.payload)
|
||||||
if json_packet['sender'][0] == '!':
|
if 'sender' in json_packet:
|
||||||
gateway_node_id = str(int(json_packet['sender'][1:], 16))
|
if json_packet['sender'][0] == '!':
|
||||||
NodeConfigurationMetrics().process_mqtt_update(
|
gateway_node_id = str(int(json_packet['sender'][1:], 16))
|
||||||
node_id=gateway_node_id,
|
NodeConfigurationMetrics().process_mqtt_update(
|
||||||
mqtt_json_enabled=True,
|
node_id=gateway_node_id,
|
||||||
mqtt_encryption_enabled=json_packet.get('encrypted', False),
|
mqtt_json_enabled=True,
|
||||||
mqtt_configured_root_topic=topic
|
mqtt_encryption_enabled=json_packet.get('encrypted', False),
|
||||||
)
|
mqtt_configured_root_topic=topic
|
||||||
|
)
|
||||||
|
|
||||||
@staticmethod
|
@staticmethod
|
||||||
def process_mqtt(topic: str, service_envelope: ServiceEnvelope, mesh_packet: MeshPacket):
|
def process_mqtt(topic: str, service_envelope: ServiceEnvelope, mesh_packet: MeshPacket):
|
||||||
|
|
@ -308,6 +310,8 @@ class MessageProcessor:
|
||||||
).set(mesh_packet.rx_rssi)
|
).set(mesh_packet.rx_rssi)
|
||||||
|
|
||||||
def _get_client_details(self, node_id: int) -> ClientDetails:
|
def _get_client_details(self, node_id: int) -> ClientDetails:
|
||||||
|
if node_id == 4294967295 or node_id == 1: # FFFFFFFF or 1 (Broadcast)
|
||||||
|
return ClientDetails(node_id=node_id, short_name='Broadcast', long_name='Broadcast')
|
||||||
node_id_str = str(node_id) # Convert the integer to a string
|
node_id_str = str(node_id) # Convert the integer to a string
|
||||||
with self.db_pool.connection() as conn:
|
with self.db_pool.connection() as conn:
|
||||||
with conn.cursor() as cur:
|
with conn.cursor() as cur:
|
||||||
|
|
|
||||||
|
|
@ -267,6 +267,16 @@ class PaxCounterAppProcessor(Processor):
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
logger.error(f"Failed to parse PAXCOUNTER_APP packet: {e}")
|
logger.error(f"Failed to parse PAXCOUNTER_APP packet: {e}")
|
||||||
return
|
return
|
||||||
|
self.metrics.pax_wifi_gauge.labels(
|
||||||
|
**client_details.to_dict()
|
||||||
|
).set(paxcounter.wifi)
|
||||||
|
self.metrics.pax_ble_gauge.labels(
|
||||||
|
**client_details.to_dict()
|
||||||
|
).set(paxcounter.ble)
|
||||||
|
self.metrics.pax_uptime_gauge.labels(
|
||||||
|
**client_details.to_dict()
|
||||||
|
).set(paxcounter.uptime)
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
@ProcessorRegistry.register_processor(PortNum.SERIAL_APP)
|
@ProcessorRegistry.register_processor(PortNum.SERIAL_APP)
|
||||||
|
|
|
||||||
5
main.py
5
main.py
|
|
@ -7,6 +7,7 @@ from dotenv import load_dotenv
|
||||||
|
|
||||||
from constants import callback_api_version_map, protocol_map
|
from constants import callback_api_version_map, protocol_map
|
||||||
from exporter.metric.node_configuration_metrics import NodeConfigurationMetrics
|
from exporter.metric.node_configuration_metrics import NodeConfigurationMetrics
|
||||||
|
from exporter.metric_cleanup_job import MetricTrackingRegistry
|
||||||
|
|
||||||
try:
|
try:
|
||||||
from meshtastic.mesh_pb2 import MeshPacket
|
from meshtastic.mesh_pb2 import MeshPacket
|
||||||
|
|
@ -15,7 +16,7 @@ except ImportError:
|
||||||
from meshtastic.protobuf.mesh_pb2 import MeshPacket
|
from meshtastic.protobuf.mesh_pb2 import MeshPacket
|
||||||
from meshtastic.protobuf.mqtt_pb2 import ServiceEnvelope
|
from meshtastic.protobuf.mqtt_pb2 import ServiceEnvelope
|
||||||
|
|
||||||
from prometheus_client import CollectorRegistry, start_http_server
|
from prometheus_client import start_http_server
|
||||||
from psycopg_pool import ConnectionPool
|
from psycopg_pool import ConnectionPool
|
||||||
|
|
||||||
connection_pool = None
|
connection_pool = None
|
||||||
|
|
@ -103,7 +104,7 @@ if __name__ == "__main__":
|
||||||
node_conf_metrics = NodeConfigurationMetrics(connection_pool)
|
node_conf_metrics = NodeConfigurationMetrics(connection_pool)
|
||||||
|
|
||||||
# Configure Prometheus exporter
|
# Configure Prometheus exporter
|
||||||
registry = CollectorRegistry()
|
registry = MetricTrackingRegistry()
|
||||||
start_http_server(int(os.getenv('PROMETHEUS_COLLECTOR_PORT', 9464)), registry=registry)
|
start_http_server(int(os.getenv('PROMETHEUS_COLLECTOR_PORT', 9464)), registry=registry)
|
||||||
|
|
||||||
# Create an MQTT client
|
# Create an MQTT client
|
||||||
|
|
|
||||||
|
|
@ -1,10 +1,15 @@
|
||||||
paho-mqtt~=2.1.0
|
paho-mqtt>=2.1.0
|
||||||
python-dotenv~=1.0.1
|
python-dotenv>=1.0.1
|
||||||
prometheus_client~=0.20.0
|
prometheus_client>=0.21.1
|
||||||
unishox2-py3~=1.0.0
|
unishox2-py3~=1.0.0
|
||||||
cryptography~=42.0.8
|
cryptography>=44.0.1
|
||||||
psycopg~=3.1.19
|
psycopg>=3.2.5
|
||||||
psycopg_pool~=3.2.2
|
psycopg_pool~=3.2.2
|
||||||
meshtastic~=2.3.13
|
psycopg-binary>=3.2.5
|
||||||
psycopg-binary~=3.1.20
|
|
||||||
geopy>=2.4.1
|
geopy>=2.4.1
|
||||||
|
psycopg-pool>=3.2.5
|
||||||
|
APScheduler>=3.11.0
|
||||||
|
|
||||||
|
# Meshtastic Protocol Buffers
|
||||||
|
meshtastic-protobufs-protocolbuffers-python==29.3.0.1.20241006120827+cc36fd21e859
|
||||||
|
--extra-index-url https://buf.build/gen/python
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue