meshtastic/docs/software/integrations/mqtt/index.mdx
rcarteraz 376957d25b
Some checks are pending
CI / quality (push) Waiting to run
CI / build (push) Waiting to run
add position precision filtering
2024-11-01 12:34:13 -07:00

185 lines
11 KiB
Plaintext

---
id: mqtt
title: MQTT | Integrations Overview
sidebar_label: MQTT
sidebar_position: 6
description: Bridging mesh networks over the internet and integrating Meshtastic protocols with popular technologies such as Home Assistant, Node Red, and Adafruit IO.
---
## Public MQTT Server
![Common MQTT Layout](/img/software/mqtt/mqtt.webp)
The Meshtastic project provides a public MQTT service that users can connect to, with certain restrictions in place to ensure network stability. This service allows Meshtastic devices to bridge over the internet, providing global connectivity for remote networks.
For instructions on connecting to the public MQTT server, please refer to [Connect to the Default Public Server](/docs/configuration/module/mqtt/#connect-to-the-default-public-server).
### Restrictions on the Public MQTT Server
To maintain optimal performance and protect LoRa meshes, traffic restrictions are currently applied to the public MQTT server.
#### Zero-Hop Policy
Traffic from the public MQTT server does not fully propagate through local mesh networks. Directly connected nodes will receive the data, but due to the zero-hop policy, it will not spread further to other nodes within the local mesh network.
#### Optimized Traffic Filtering
Only specific portnums are prioritized for transmission over the public MQTT server when using the default PSK:
- NodeinfoApp
- TextMessageCompressedApp
- TextMessageApp
- PositionApp
- TelemetryApp
- MapReportApp
- RoutingApp
#### Location Precision Filtering
On the default PSK, the public server also limits the precision of location data to help protect user privacy. Only position packets containing imprecise location data (10-16 bits) are shared on the topic, ensuring that precise location details are not exposed. For more information on how location precision works, see the [Position Precision](https://meshtastic.org/docs/configuration/radio/channels/#position-precision) section.
This filtering focuses network resources on critical traffic, improving overall performance and reducing unnecessary data flow. Since these restrictions are applied at the network level, no firmware updates are required. As Meshtastic networks continue to grow, further traffic reduction measures may become necessary to manage network load and maintain reliable performance across all channels.
### Using Private Brokers
It is not recommended to use the default key (PSK) on a private broker. Doing so potentially allows security vulnerabilities and can flood the mesh with traffic, as private brokers do not enforce the zero-hop policy needed for public channels. Private brokers are intended for use with private channels, where custom PSKs provide secure, isolated communication.
## Software Integrations
Using or emitting packets directly in/from smart home control software such as Home Assistant or other consumers that can work with JSON messages.
When MQTT is enabled, the Meshtastic device simply uplinks and/or downlinks every raw protobuf MeshPacket that it sees to the MQTT broker, encapsulated in a [ServiceEnvelope protobuf](https://buf.build/meshtastic/protobufs/docs/main:meshtastic#meshtastic.ServiceEnvelope). In addition, some packet types are serialized or deserialized from/to JSON messages for easier use in consumers. All packets are sent to the broker, whether they originate from another device on the mesh, or the gateway node itself.
### MQTT [Topics](https://www.hivemq.com/blog/mqtt-essentials-part-5-mqtt-topics-best-practices)
If no specific [root topic](/docs/configuration/module/mqtt#root-topic) is configured, the default root topic will be `msh/REGION`.
For each channel where uplink and/or downlink is enabled, two topics might be used:
#### Protobufs topic
A gateway node will uplink and/or downlink raw ([protobuf](https://developers.google.com/protocol-buffers)) MeshPackets to the topic:
`msh/REGION/2/e/CHANNELNAME/USERID`, where `CHANNELNAME` is the name of the channel (firmware versions prior to 2.3.0 will publish to a topic with `/c/` in the place of `/e/`).
For example: `msh/US/2/e/LongFast/!abcd1234`
The payload is a raw protobuf, whose definitions for Meshtastic can be found [here](https://github.com/meshtastic/protobufs/blob/master/meshtastic). Reference guides for working with protobufs in several popular programming languages can be found [here](https://protobuf.dev/reference/). Looking at the MQTT traffic with a program like `mosquitto_sub` will tell you it's working, but you won't get much useful information out of it. For example:
```text
苓????"!
!937bed1cTanksTnk"D???05??=???aP`
ShortFast !937bed1c
```
If [encryption_enabled](/docs/configuration/module/mqtt#encryption-enabled) is set to true, the payload of the MeshPacket will remain encrypted with the key for the specified channel.
#### JSON topic
:::note
JSON is not supported on the nRF52 platform.
:::
If [JSON is enabled](/docs/configuration/module/mqtt#json-enabled), packets from the following [port numbers](/docs/development/firmware/portnum) are serialized to JSON: `TEXT_MESSAGE_APP`, `TELEMETRY_APP`, `NODEINFO_APP`, `POSITION_APP`, `WAYPOINT_APP`, `NEIGHBORINFO_APP`, `TRACEROUTE_APP`, `DETECTION_SENSOR_APP`, `PAXCOUNTER_APP` and `REMOTE_HARDWARE_APP`. These are then forwarded to the topic:
`msh/US/2/json/CHANNELNAME/USERID`.
An example of a received `NODEINFO_APP` message:
```json
{
"id": 452664778,
"channel": 0,
"from": 2130636288,
"payload": {
"hardware": 10,
"id": "!7efeee00",
"longname": "base0",
"shortname": "BA0"
},
"sender": "!7efeee00",
"timestamp": 1646832724,
"to": -1,
"type": "nodeinfo"
}
```
The meaning of these fields is as follows:
- "`id`" is the unique ID for this message.
- "`channel`" is the channel index this message was received on.
- "`from`" is the unique decimal-equivalent Node ID of the node on the mesh that sent this message. (The hexadecimal value `7efeee00` represented by an integer in decimal is `2130636288`).
- "`id`" inside the payload of a `NODEINFO_APP` message is the hexadecimal Node ID (sometimes called User ID) of the node that sent it.
- "`hardware`" is the [hardware model](https://github.com/meshtastic/protobufs/blob/master/meshtastic/mesh.proto#L215) of the node sending the `NODEINFO_APP` message.
- "`longname`" is the long name of the device that sent the `NODEINFO_APP` message.
- "`shortname`" is the short name of the device that sent the `NODEINFO_APP` message.
- "`sender`" is the hexadecimal Node ID of the gateway device, which is in this case the same node that sent the `NODEINFO_APP` message.
- "`timestamp`" is the Unix Epoch when the message was received, represented as an integer in decimal.
- "`to`" is the decimal-equivalent Node ID of the destination of the message. In this case, "-1" means it was a broadcast message (this is the decimal integer representation of `0xFFFFFFFF`).
- "`type`" is the type of the message, in this case it was a `NODEINFO_APP` message.
The `from` field can thus be used as a stable identifier for a specific node. Note that in firmware prior to 2.2.0, this is a signed value in JSON, while in firmware 2.2.0 and higher, the JSON values are unsigned.
If the message received contains valid JSON in the payload, the JSON is deserialized and added as a JSON object rather than a string containing the serialized JSON.
#### JSON downlink to instruct a node to send a message
You can also send a JSON message to the topic `msh/US/2/json/mqtt/` to instruct a gateway node to send a message to the mesh.
To make this work, ensure that your node has a Meshtastic channel configured called "mqtt". Enable Downlink. The PSK can be random and doesn't matter. This channel allows the node to listen to messages on the `msh/US/2/json/mqtt/` topic.
Reboot your device after creating this channel.
The JSON message should contain the following fields:
```json
{
"from": <decimal Node ID of MQTT node>,
"to": <decimal Node ID of recipient for a DM (optional)>,
"channel": <channel index (optional)>,
"type": "type",
"payload": {
"key":"value"
...
}
}
```
`from` and `payload` fields are required for a valid envelope (note that in firmware &lt;2.2.20 a field `sender` was required, but this is no longer the case). The `from` field should be equal to the decimal Node ID of the node that will transmit the message. If the Node ID (sometimes called User ID) is !7efeee00, then the decimal equivalent would be 2130636288. Optionally, you can specify a different channel than the primary channel by setting the `channel` field to a channel index (0-7). Furthermore, you can send a direct message by setting the `to` field to the decimal Node ID of the destination. If the `to` field is not set, the message will be broadcast to all nodes on the mesh.
Currently two types of messages are supported: `"sendtext"` and `"sendposition"`.
For the type `sendtext`, the `payload` should be a string containing the text to send. For the type `sendposition`, the payload should be an object with the fields `latitude_i`, `longitude_i`, `altitude` (optional) and `time` (optional).
### Basic Configuration
Check out [MQTT Settings](/docs/configuration/module/mqtt) for full information. For quick start instructions, read on.
- Connect your gateway node to wifi, by setting the `network.wifi_ssid`, `network.wifi_psk` and `network.wifi_enabled` preferences.
- Alternatively use the RAK4631 with Ethernet Module RAK13800, by setting `network.eth_mode` and `network.eth_enabled` (note that JSON is not supported on the nRF52 platform).
- Configure your broker settings: `mqtt.address`, `mqtt.username`, and `mqtt.password`. If all are left blank, the device will connect to the Meshtastic broker.
- Set `uplink_enabled` and `downlink_enabled` as appropriate for each channel. Most users will just have a single channel (at channel index 0). `meshtastic --ch-index 0 --ch-set uplink_enabled true`
`uplink_enabled` will tell the device to publish mesh packets to MQTT.
`downlink_enabled` will tell the device to subscribe to MQTT, and forward any packets from there onto the mesh.
### Gateway nodes
Any meshtastic node that has a direct connection to the internet (either via a helper app or installed WiFi/4G/satellite hardware) can function as a "Gateway node".
Gateway nodes (via code running in the phone) will contain two tables to whitelist particular traffic to either be delivered toward the internet, or down toward the mesh. Users that are developing custom apps will be able to customize these filters/subscriptions.
Since multiple gateway nodes might be connected to a single mesh, it is possible that duplicate messages will be published on any particular topic. Therefore, subscribers to these topics should
deduplicate if needed by using the packet ID of each message.
## Examples
- [Using mosquitto on a mac](/docs/software/integrations/mqtt/mosquitto.mdx)
- [Sending/receiving messages on mosquitto server using python](/docs/software/integrations/mqtt/python.mdx)
- [Using MQTT with Node-RED](/docs/software/integrations/mqtt/nodered.mdx)
- [Using MQTT with Home Assistant](/docs/software/integrations/mqtt/home-assistant.mdx)
- [Using MQTT with Adafruit IO](/docs/software/integrations/mqtt/adafruit-io.mdx)