meshtastic/docs/software/other/mqtt.mdx

---
id: mqtt
title: MQTT
sidebar_label: MQTT
---

## MQTT

Meshtastic devices with wifi hardware (ESP32) are able to connect to an MQTT broker to uplink and downlink mesh packets. This is useful for a number of purposes:

- Connecting your mesh to the official Meshtastic MQTT broker. This makes your devices appear on the world map, and provides a limited copy of your mesh traffic, translated into JSON.
- Using a custom MQTT broker to bridge several mesh networks together, via the internet (or just a local IP network)
- Using a custom MQTT broker and a translator program to decode the raw protobuf packets and translate them into a plain text form for use in other systems. eg plotting temperature readings in Grafana, or device positions in Traccar.
- Using or emitting packets directly in/from smart home control software such as Home Assistant or other consumers that can work with JSON messages.

When MQTT enabled, the Meshtastic device simply uplinks and/or downlinks every raw protobuf packet that it sees to the MQTT broker. In addition, some packet types are serialized or deserialized from/to JSON messages for easier use in consumers. All packets are sent to the broker, whether they originate from another device on the mesh, or the gateway node itself.

Packets may be encrypted. If you use the default meshtastic MQTT server, packets are always encrypted. If you use a custom MQTT broker (ie set `mqtt_server`), the `mqtt_encryption_enabled` setting applies, which by default is false.

IMPORTANT: When MQTT is turned on, you are potentially broadcasting your entire mesh traffic onto the public internet. This includes messages and position information.

### MQTT Topics

The device will uplink and downlink raw ([protobuf](https://developers.google.com/protocol-buffers)) packets to the `msh/` prefix:

`msh/1/c/ShortFast/!12345678` where

- `!12345678` is the address of the gateway device.
- `ShortFast` is the channel name.

The payload is a raw protobuf. Looking at the MQTT traffic with a program like `mosquitto_sub` will tell you it's working, but you won't get much useful information out of it. For example:

```
苓????"!
	!937bed1cTanksTnk"D???05??=???aP`
	ShortFast	!937bed1c
```

Packets from the following [port numbers](/docs/developers/Firmware/portnum) are serialized to JSON and then forwarded to the `msh/1/json/CHANNELID/DEVICEID` topic: `TEXT_MESSAGE_APP`, `ENVIRONMENTAL_MEASUREMENT_APP`, `NODEINFO_APP` and `POSITION_APP`.

An example of a received `NODEINFO_APP` message:

```json
{
  "id": 452664778,
  "channel": 0,
  "from": 2130636288,
  "payload": {
    "hardware": 10,
    "id": "!7efeee00",
    "longname": "base0",
    "shortname": "BA0"
  },
  "sender": "!7efeee00",
  "timestamp": 1646832724,
  "to": -1,
  "type": "nodeinfo"
}
```

If the message received contains valid JSON in the payload, the JSON is deserialized and added as a JSON object rather than a string containing the serialized JSON.

**Sent messages** will be checked if the MQTT payload contains a valid JSON-encoded envelope:

```json
{
   "sender": "SENDER",
   "payload": {
       "key":"value"
       ...
    }
}
```

`sender` and `payload` fields are required for a valid envelope. If a valid MQTT message is found, the message is sent over the radio as a message of type `TEXT_MESSAGE_APP` with the serialized `payload` value in the message payload.

### Basic Configuration

Check out [MQTT Settings](/docs/settings/moduleconfig/mqtt) for full information. For quick start instructions, read on.

- Connect your gateway node to wifi, by setting the `wifi_ssid` and `wifi_password` preferences.
- Configure your broker settings: `mqtt_server`, `mqtt_username`, and `mqtt_password`. If all are left blank, the device will connect to the Meshtastic broker.
- Set `uplink_enabled` and `downlink_enabled` as appropriate for each channel. Most users will just have a single channel (at channel index 0). `meshtastic --ch-index 0 --ch-set uplink_enabled true`

`uplink_enabled` will tell the device to publish mesh packets to MQTT.
`downlink_enabled` will tell the device to subscribe to MQTT, and forward any packets from there onto the mesh.

### Getting plain data out of the mesh

As of firmware 1.2.53, it is possible for the device to decrypt the protobufs before publishing to MQTT. To translate this into a plain format:

- Set up a gateway node to uplink packets to your MQTT broker:
  - `meshtastic --set wifi_ssid XXXX`
  - `meshtastic --set wifi_password XXXX`
  - `meshtastic --set mqtt_server 192.168.1.1`
  - `meshtastic --set mqtt_username XXXX`
  - `meshtastic --set mqtt_password XXXX`
  - `meshtastic --set mqtt_encryption_enabled false`
  - `meshtastic --ch-index 0 --ch-set uplink_enabled true`
- Grab the meshtastic-mqtt script from [here](https://github.com/joshpirihi/meshtastic-mqtt)
  - `git clone https://github.com/joshpirihi/meshtastic-mqtt && cd meshtastic-mqtt`
  - Edit `meshtastic_mqtt/meshtastic_mqtt.py` and enter your mqtt broker details
  - Install the script with `pip install .`
  - Run `meshtastic-mqtt`. It will print some debug output by default, and publish the plain values to the `meshtastic/` prefix.
  - View the plain data with `mosquitto_sub -h YOUR_MQTT_SERVER -t meshtastic/# -v`
  - You can then consume the data easily in other systems. For example, nodered->influx db->grafana.


## On device API

For information on the related on-device API see [here](/docs/developers/firmware/device-api).

## MQTT transport

Any gateway-device will contact the MQTT broker.

### Topics

The "mesh/crypt/CHANNELID/NODEID/PORTID" [topic](https://www.hivemq.com/blog/mqtt-essentials-part-5-mqtt-topics-best-practices) will be used for messages sent from/to a mesh.

Gateway nodes will forward any MeshPacket from a local mesh channel with uplink_enabled. The packet (encapsulated in a ServiceEnvelope) will remain encrypted with the key for the specified channel.

For any channels in the local node with downlink_enabled, the gateway node will forward packets from MQTT to the local mesh. It will do this by subscribing to mesh/crypt/CHANNELID/# and forwarding relevant packets.

If the channelid 'well known'/public it could be decrypted by a web service (if the web service was provided with the associated channel key), in which case it will be decrypted by a web service and appear at "mesh/clear/CHANNELID/NODEID/PORTID". Note: This is not in the initial deliverable.

#### Service Envelope

The payload published on mesh/... will always be wrapped in a [ServiceEnvelope protobuf](/docs/developers/protobufs/api#serviceenvelope).

ServiceEnvelope will include the message, and full information about arrival time, who forwarded it, source channel, source mesh id, etc...

#### NODEID

The unique ID for a node. A hex string that starts with an ! symbol.

#### USERID

A user ID string. This string is either a user ID if known or a nodeid to simply deliver the message to whoever the local user is of a particular device (i.e. person who might see the screen). FIXME, see what riot.im uses and perhaps use that convention? Or use the signal +phone number convention? Or the email addr?

#### CHANNELID

FIXME, figure out how channelids work

### Gateway nodes

Any meshtastic node that has a direct connection to the internet (either via a helper app or installed Wifi/4G/satellite hardware) can function as a "Gateway node".

Gateway nodes (via code running in the phone) will contain two tables to whitelist particular traffic to either be delivered toward the internet, or down toward the mesh. Users that are developing custom apps will be able to customize these filters/subscriptions.

Since multiple gateway nodes might be connected to a single mesh, it is possible that duplicate messages will be published on any particular topic. Therefore, subscribers to these topics should
deduplicate if needed by using the packet ID of each message.

### Optional web services

#### Public MQTT broker service

An existing public [MQTT broker](https://mosquitto.org) will be the default for this service, but clients can use any MQTT broker they choose.

FIXME - figure out how to avoid impersonation (because we are initially using a public MQTT server with no special security options). FIXME, include some ideas on this in the ServiceEnvelope documentation.

#### Riot.im messaging bridge

@Geeksville will run a riot.im bridge that talks to the public MQTT broker and sends/receives into the riot.im network.

There is apparently [already](https://github.com/derEisele/tuple) a riot.im [bridge](https://matrix.org/bridges) for MQTT. That will possibly need to be customized a bit. But by doing this, we should be able to let random riot.im users send/receive messages to/from any meshtastic device. (FIXME ponder security). See this [issue](https://github.com/meshtastic/Meshtastic-Android/issues/2#issuecomment-645660990) with discussion with the dev.

### Deprecated concepts

:::caution

All of the following concepts have been deprecated

:::

You can ignore these for now...

#### MESHID (deprecated)

Earlier drafts of this document included the concept of a MESHID. That concept has been removed for now, but might be useful in the future. The old idea is listed below:

A unique ID for this mesh. There will be some sort of key exchange process so that the mesh ID can not be impersonated by other meshes.

#### DESTCLASS (deprecated)

Earlier drafts of this document included the concept of a DESTCLASS. That concept has been removed for now, but might be useful in the future. The old idea is listed below:

The type of DESTID this message should be delivered to. A short one letter sequence:

| Symbol | Description                                                   |
| ------ | ------------------------------------------------------------- |
| R      | riot.im                                                       |
| L      | local mesh node ID or ^all                                    |
| A      | an application specific message, ID will be an APP ID         |
| S      | SMS gateway, DESTID is a phone number to reach via Twilio.com |
| E      | Emergency message, see bug #FIXME for more context            |

#### DESTID (deprecated)

Earlier drafts of this document included the concept of a DESTCLASS. That concept has been removed for now, but might be useful in the future. The old idea is listed below:

Can be...

- an internet username: kevinh@geeksville.com
- ^ALL for anyone
- An app ID (to allow apps out on the web to receive arbitrary binary data from nodes or simply other apps using meshtastic as a transport). They would connect to the MQTT broker and subscribe to their topic

### Mini tutorial on how to get up and running with mosquitto on a mac

1. install mqtt server

```
brew install mosquitto
```

2. start the mqtt server

```
brew services restart mosquitto
```

3. Do a quick test of server, start a subscriber on a topic:

Note: this will wait until you press control-c (publish a message, see below)

```
mosquitto_sub -t test/hello
```

4. In another window, publish a message to that topic:

```
mosquitto_pub -h localhost -q 0 -t test/hello -m 'yo!'
```

5. For Meshtastic to be able to access that server, two settings need to be changed in the
   `/usr/local/etc/mosquitto/mosquitto.conf` file:

```
listener 1883 0.0.0.0
allow_anonymous true
```

6. Restart the service:

```
brew services restart mosquitto
```

7. If you are using the mac firewall, you will need to go into: System Preferences > Security & Privacy > Firewall > Firewall Options and add it.

### Sending/receiving messages on mosquitto server using python

Here is an example publish message in python:

```
#!/usr/bin/env python3
import paho.mqtt.client as mqtt
from random import randrange, uniform
import time

client = mqtt.Client("some_client_id")
client.connect('localhost')

while True:
    randNumber = uniform(20.0, 21.0)
    client.publish("env/test/TEMPERATURE", randNumber)
    print("Just published " + str(randNumber) + " to topic TEMPERATURE")
    time.sleep(1)
```

Here is example subscribe in python:

```
#!/usr/bin/env python3

import paho.mqtt.client as paho

def on_message(mosq, obj, msg):
    print("%-20s %d %s" % (msg.topic, msg.qos, msg.payload))
    mosq.publish('pong', 'ack', 0)

def on_publish(mosq, obj, mid):
    pass

if __name__ == '__main__':
    client = paho.Client()
    client.on_message = on_message
    client.on_publish = on_publish

    client.connect("localhost", 1883, 60)

    client.subscribe("env/test/TEMPERATURE", 0)

    while client.loop() == 0:
        pass
```
### Tutorial on using MQTT with version 1.3.46+

Version 1.3.46+ is the first 1.3 release with MQTT and JSON fully functional on supported devices. Heltec JSON over MQTT is unlikely to work reliably.

One of the major changes is that the TOPICs are different from prior versions.
`msh/1/c/ChannelName/!12345678` and
`msh/1/json/ChannelName/!12345678`

are now
`msh/2/c/ChannelName/!12345678` and
`msh/2/json/ChannelName/!12345678`
Please see the documentation at the top of this topic for that.

Node-RED is a free cross-platform programming tool for wiring together hardware, APIs, and online services developed originally by IBM for IOT. It is widely used for home automation by many non-professional programmers and runs well on Pi's. Node-red has many plug-in modules written by the community. I will use this platform as a practical example on how to interface with the MQTT features of Meshtastic. Everything can be done from GUI's without using command line.

Step one: use http://client.meshtastic.org/ to connect to your device via usb and adjust these settings using the web GUI.
Enable and enter network ssid/psk. Settings-->Device Config-->Network; Save.
Set MQTT server address. Settings-->Module Config -->MQTT config; Verify Encryption Enabled is OFF. Turn JSON Output Enabled ON. Save.
Go to Channel Editor and set uplink and downling enabled to True. Save.

Step two: if you don't want to depend on JSON decoding on the device, you can decode the proto messages off-device. To do that you will need to get the .proto files from https://github.com/meshtastic/Meshtastic-protobufs. They function as a schema and are required for decoding in Node-RED. Save the files where the node-RED application can access them and note the file path of the "mqtt.proto" file.

Step three: install Node-RED plug-ins to your node-RED application for an embedded mqqt server and a protobuf decoder.
https://flows.nodered.org/node/node-red-contrib-aedes
https://flows.nodered.org/node/node-red-contrib-protobuf

Drag, drop, and wire the nodes like this. For this example, I ran node-RED on a Windows machine. Note that file paths might be specified differently on different platforms. MQTT server wild cards are usually the same. A "+" is a single level wildcard for a specific topic level. A "#" is a multiple level wildcard that can be used at the end of a topic filter. The debug messages shown are what happens when the inject button sends a json message with a topic designed to be picked up by the specified Meshtastic device and then having it resend the message by both radio and MQTT.

[<img src="/documents/mqtt/NodeRedTwo.jpg" style={{zoom:'50%'}} />](/documents/mqtt/NodeRedTwo.jpg)
[<img src="/documents/mqtt/NodeRedThree.jpg" style={{zoom:'50%'}} />](/documents/mqtt/NodeRedThree.jpg)
[<img src="/documents/mqtt/NR_nodes.jpg" style={{zoom:'50%'}} />](/documents/mqtt/NR_nodes.jpg)