if you use the command docker network ls on a new docker installation you will see three networks.

When you use the run command to run a container without using the network switch the container will use the default bridge network.

On the bridge network container apps can communicate using  the IP address of the container.

Containers on the bridge network are given an IP address from the address pool. This is  usually 172.x.0.0 range.

For applications on the main network to access applications in a container requires that the container must map/publish ports.

This is why you see  container run commands looking like this

docker run -it –name mos2 -p 1883:1883 -p 8853:8883

The Host network switch

You may see  run commands looking like this (mosquitto example):

docker run -it –name mos2 --network=host

You should notice that no port mapping is being used but applications on the main network can access the mosquitto broker on any ports that mosquitto has been configured to use.

This is very similar to starting mosquitto normally( without docker).

The IP address of the broker is the same as the IP address of the host.

However this option will expose all listening ports in the container and so it is not often used except for container Applications that need to listen to network broadcast messages.

Broadcast messages  can’t be forwarded using the bridge e.g a DHCP server –ref .

Home assistant on docker requires this option for device discovery to work.

Creating additional Networks

You can also create your own networks using the docker network create command. example

docker network create -d bridge my-net

Where my-net is the name of the network.

When you run a container you can connect it to the network using the network= switch. e.g

docker run --network=my-net -itd --name=mos eclipse-mosquitto:2

You can also connect  a running container to a network using the connect command and also specify an IP address the container will use on that network if desired. e.g.

docker network connect --ip 172.19.1.4 my-net mos

On  a network containers can communicate using the IP address or the container name.

So for example if you start two containers using the –-name switch set to c1 and c2 then from container c1 you can ping c2 using the command :

ping c2

and vice versa.

Example Docker Network

In the example docker network as shown in the diagram below we have 3 containers

• A mosquitto broker using port 1883
• A node-red Server using port 1880
• An influxdb database using port 8086

Requirements

1. Clients on the host network need to connect to the mosquitto broker and the node-red server.

2. The node-red server needs to connect to the influxdb database

Setup

1. We need to map ports for the mosquitto broker and node-red server but not the inflluxdb database.

2. The container for the influxdb database should have a fixed IP address which requires a user defined network.

3. Create a user defined network and place all 3 containers on it.

4. For completion give all containers a fixed IP address (optional).

Common Questions and Answers

Q- Do I need to create my own network?

A- No it is usually only done when running multiple containers that need to be isolated from each other or containers that require a fixed IP address.

Q-If I don’t map ports will my applications still work?

A- Yes but will only be accessible to containers on the same network and not to users on the host network.

Related Tutorials and Resources:

• Running The Mosquitto MQTT Broker In Docker – Beginners Guide

However MQTT wasn’t designed for file sending but for sending small data packets from remote sensors.

Having said that the actual maximum message size is very large (256MB).

However not all MQTT brokers will support the maximum packet size and all public brokers have message size restrictions often between 128KB and 256kB.

In addition clients will also have message size restrictions.

Therefore a common practise is to split large files into chunks for sending.

File Data and Meta Data

Generally we will want to send the actual file data along with the file name.

When reading a file from disk a text file can be converted to text on reading or read directly as bytes.

For image/video files then they will be read as bytes.

To avoid complications I prefer to read all files as binary data so that they can be place directly in the message payload as MQTT is a binary protocol.

However sending binary data makes it more difficult to add meta data to the file data.

For the file name which is most important bit of meta data the easiest approach is to include it as part of the topic.

MQTTv5 introduced some important additions which allows you to add the file meta data outside of the payload using User properties.

Indicating Start and end of File

To tell the receiver that the file transfer is starting we can use a start and end message.

These messages would not contain file data but would contain file meta data.

Verifying Successful transfer

There are a number of ways this can be done e.g.

• Use the file length
• Use a file hash
• length + file hash.

To avoid the the problem of buffering the entire file then the file length and digest is send in the end of file  message.

Broker/Sever Extensions

Some commercial brokers have developed server extensions aimed at making file transfer over MQTT easier. See hiveMQ and EMQX

Common Questions and Answers

Q1- MQTT wasn’t designed for file transfer so why use it?

A1- For convenience. HTTP wasn’t designed for email or file downloading but it used for both.

Q2 – Is MQTTv5 better than MQTT v3.1.1 for File transfer.

A2- MQTTv5 has some useful features that make file sending easier as data can be sent outside the message payload.

Q3 – Can I use QOS of 0?

A3 -Yes but many programs will use the puback to control message flow

Common Use Cases

• Uploading diagnostic logs or telemetry data from vehicles or sensors.
• Sending firmware updates (OTA) to IoT devices.
• Transferring images or video clips from security cameras to a server or to different locations.

Summary

Although MQTT was intended as a file transport protocol it can be used for sending files across a network.

Although it may be frowned upon by some it will probably become a common use case in the future due the ease of doing it and the convenience.

File Transfer Examples

• Send files using Python and MQTT

Birth and Death messages are special messages that help communicate the state of a device to other clients in a MQTT network.

The MQTT protocol has provision for death messages known as last will and testament but not birth messages.

Birth messages are part of the Sparkplug specification and are sent by the MQTT client and are normal MQTT messages with a specific format.

 Birth and Death Message Formats

Birth Messages are sent by a client when it connects (or reconnects) to an MQTT broker, signalling that it’s alive and ready to communicate.

Death Messages are published by the broker on behalf of the client when the client unexpectedly disconnects (e.g., due to a crash or power failure).

A last will and testament message requires:

• a Topic
• A message
• A QOS level (quality of service)
• A message Retain setting.

Birth messages follow the same format.

Example

Birth messages

Topic=topic_base/connection
Message=online
QOS=0
Retain=true

Death messages

Topic=topic_base/connection
Message=offline
QOS=0
Retain=true

Example Implementation Using Python

It can easily be added to existing code in the on_connect callback. Here is my code with the addition.

def on_connect(client, userdata, flags, rc):
 logging.debug("Connected flags"+str(flags)+"result code "\
 +str(rc)+"client1_id")
 if rc==0:
 client.connected_flag=True
client.publish(connection_topic,"online",0,retain=True)
 else:
 client.bad_connection_flag=True

We also need to Set the last will message.Payload is 0 and we use the retain flag.

client.will_set(connected_topic,"offline",0, retain=True) #set will

The last will will take care of an abnormal disconnect. For a normal disconnect we need to remember to update the status before we disconnect as follows:

client.publish(connection_topic,0,retain=True)
client.disconnect()

Note: The retain flag should be set in both messages to ensure new subscribers get the current state.

Example implementation including Python Code.

The checking active connections tutorial has an example and also suggestions on making the message more useful.

Related Tutorials

• Using The Mosquitto_pub and Mosquitto_sub MQTT Client Tools- Examples
• MQTT Retained Messages Explained
• MQTT Last Will and Testament Examples
• Understanding MQTT Topic Naming and Design Notes
• Introduction to MQTT +Sparkplug For IIOT

The HTTP protocol is generally used for initial  device set up.

However when it comes to sending device data and for subsequent device control HTTP is not usually the best protocol.

Generally when you look for a comparison between the protocols the main emphasis is on the fact that HTTP is a text based protocol and not as efficient as MQTT which is a binary based protocol.

However when dealing with only a few devices that send data infrequently then it is not really a problem.

IMO ,the main issues  are the fact that MQTT is a real time protocol and because of the publish and subscribe model is excellent for working with a large number of devices and real time data.

As an example imagine you have 100 parking space sensors. How would you get the status of each sensor?

With HTTP you would need to poll each sensor frequently and you would require 100 client connections from the control dashboard.

With MQTT you would only require one connection to the broker from the control dashboard, and to subscribe to a single topic to receive status changes in real time.

This is what ChatGPT said about use cases

Device IP Addresses

Another very important point that is not discussed is that when using HTTP for device control then the control dashboard needs to know the IP address of the device.

With many devices DHCP will be used and so keeping track of device IP addresses isn’t always easy.

See Finding New Device IP addresses

However the task will become increasingly more difficult as networks move to IPV6 as the methods used for IPv4 networks won’t work on IPv6.

Because MQTT doesn’t have a direct connection between devices then knowing the IP address of any device isn’t necessary.

Video

Related Tutorials and Resources:

• Understanding HTTP Basics
• Using HTTP APIs For IOT -Beginners Guide
• IPv6 Explained for Beginners
• How MQTT Works 

Overview

In this design example we take the case of a network of parking sensors. Sensors will be sending an occupied/unoccupied state message as well as supplementary information like battery level,signal level IP address etc.

Data is not confidential and not critical.

A control dashboard application will receive the data and process as necessary

Topic and Payload Structure

A starting structure is shown below. All messages will be sent as JSON encoded data.

parking/
├── sensors/
│ ├── zone1/
│ │ ├── sensor001/
│ │ │ ├── status (e.g., “occupied”, “vacant”)
│ │ │ ├── battery (e.g., “80%”)
│ │ │ ├── signal (e.g., “strong”, “weak”)
│ │ │ ├── connected (e.g., “online”, “offline”)
│ │ ├── sensor002/
│ │ ├── …
│ ├── zone2/
│ ├── …
├── commands/
│ ├── reset/
│ │ ├── all
│ │ ├── zone1
│ │ ├── sensor001
│ ├── update-firmware/
│ │ ├── all
│ │ ├── zone1
│ │ ├── sensor001
├── alerts/
│ ├── low-battery/
│ │ ├── sensor001
│ │ ├── sensor045
│ ├── malfunction/
│ │ ├── sensor120
│ │ ├── sensor234
│ ├── disconnected/
│ │ ├── sensor050
│ │ ├── sensor123
├── analytics/
│ ├── occupancy/
│ │ ├── zone1
│ │ ├── zone2
│ │ ├── total
│ ├── usage-patterns/
│ ├── average-duration/

Explanation:

sensors/zoneX/sensorYYY/ → Each sensor reports status, battery, and signal strength.

commands/ → Commands to reset, update firmware, or send other instructions.

alerts/ → Separate topics for low battery, malfunction, or other issues.

analytics/ → Aggregated data for parking usage and occupancy statistics.

connected status under each sensor to indicate if it’s online or offline.

alerts/disconnected/ for tracking sensors that have lost connection.

Possible Enhancements

Include a time stamp for each sensor attribute e.g. battery,signal etc

Example Topics and MQTT payloads

Topic: parking/sensors/zone1/sensor001/status

Payload: JSON

{
“value”: “occupied”,
“timestamp”: “2025-03-23T12:34:56Z”
}

Payload:

{
“value”: “online”,
“timestamp”: “2025-03-23T12:37:15Z”
}

Using Sparkplug For MQTT

The topic structure does resemble the Sparkplug topic structure. However adopting the Sparkplug topic and payload structure is a consideration so here is what chatGPT has to say:

Sparkplug is a better option than plain JSON when dealing with IIoT (Industrial IoT) applications that require structured payloads, state awareness, and efficient bandwidth usage. It provides data modeling, compression, and device state tracking out of the box.

However I would go for JSON data as it is much easier to work with than Google Protocol buffers.

QOS and Retained

Occupancy data can use a QOS of either 0 or 1. Generally 1 is the default choice but as this data is not really critical I would opt for a QOS of 0 unless the network was unreliable.

For occupancy data,will  and birth messages the retained flag should also be set.

Designing an ACL (Access Control List) for MQTT Sensors on Mosquitto Broker

The sensors will both send data and receive commands. It is important that sensors are restricted to sending data on their own topic and that only authorised devices are able to send commands.

An ACL (Access Control List) in Mosquitto controls which clients (sensors, gateways, admins) can publish and subscribe to specific topics.

Below is a structured ACL design for your parking lot sensor system, considering security and scalability.

Roles:

• Sensors (sensor_XXX) → Can only publish their own data.

• Gateways (gateway_XXX) → Can publish and subscribe to all sensors in their zone.

• Admin (admin) → Full access to all topics.

Notes: although the ACL uses the user name it is not necessary to have a passwords file unless you are also using user name and password authentication.

The following file was suggested by ChatGPT

# ===========================
# ACL for MQTT Parking System
# ===========================

# ----- Admin Full Access -----
user admin
topic readwrite parking/#

# ----- Gateway Access -----
# Each gateway can publish and subscribe to its zone's sensors.
user gateway_zone1
topic readwrite parking/sensors/zone1/#
topic readwrite parking/alerts/zone1/#
topic readwrite parking/commands/zone1/#

user gateway_zone2
topic readwrite parking/sensors/zone2/#
topic readwrite parking/alerts/zone2/#
topic readwrite parking/commands/zone2/#

# ----- Sensor Access -----
# Each sensor can only publish to its own topic and read commands for itself.

user sensor_001
topic write parking/sensors/zone1/sensor001/#
topic read parking/commands/zone1/sensor001/#
topic read parking/alerts/disconnected/sensor001

user sensor_002
topic write parking/sensors/zone1/sensor002/#
topic read parking/commands/zone1/sensor002/#
topic read parking/alerts/disconnected/sensor002

# (Repeat for all sensors...)

# ----- Public Read-Only Access -----
# If you want a dashboard or external system to read aggregated data.
user dashboard
topic read parking/analytics/#

# ===========================

However a better solution is to use the pattern option:

pattern write parking/sensors/zone1/%u/#
pattern read parking/commands/zone1/%u/#
pattern read parking/alerts/disconnected/%u

An alternative is to use the Pattern option with the sensor Id. This is my preferred approach if user name authentication isn’t going to be used.

pattern write parking/sensors/zone1/%c/#
pattern read parking/commands/zone1/%c/#
pattern read parking/alerts/disconnected/%c

mosquitto conf file addition.

allow_anonymous true
acl_file /etc/mosquitto/mosquitto.acl

Note: I have used the standard location for the acl file on Linux in the example above. However in practise, for testing, I always place the files in my home folder and the example files you can download will use my local path.

Authentication

As an extra level of security you can also use user name and password authentication.

This requires creating a passwords file and forcing authentication in the mosquitto.conf file.

allow_anonymous false
acl_file /etc/mosquitto/mosquitto.acl
password_file /etc/mosquitto/passwd

Although most texts will recommend a unique user name and password for each sensor such a scenario is difficult to administer.

If you are using user names as part of the acl restrictions then a unique user name for each sensor is necessary but a unique password isn’t.

If you use the sensor ID for ACL restrictions (recommended) then all sensors can use the same user name and password.

Because the data isn’t really confidential I would not implement user name authentication in this scenario.

Encryption (SSL or End to End)

In our current implantation we are only sending non sensitive data and so there is no need to encrypt the data.

If our messages contained personal data like vehicle registration plates, or billing data then we would need to use some form of encryption.

If you use SSL then you must implement it on each link.

Problem: If data is only protected with SSL/TLS (which secures data in transit but not at rest), the broker can still see and store unencrypted license plates and payment data. If the broker is compromised, attackers could steal sensitive information.

If data is to be stored then end to end Encryption is the preferred choice.

Key Benefits of E2EE

Even if the MQTT broker is hacked, data remains protected.

Prevents unauthorized users from accessing sensitive parking and payment data.

Ensures only intended recipients (backend, app) can decrypt messages.

Complies with privacy regulations (GDPR, PCI-DSS for payments).

Data Frequency

A 5 minute send cycle would be often enough as changes are always sent immediately.

Questions

Q1- What would be the occupancy status if the sensor was detected to be offline?

Q2- If anonymous access is allowed and there is no password file do you need to send a password with the user name?

Workbook and Scripts

I will be making test python scripts, ACL files and mosquitto conf files available soon .

Answers

A1- unknown

A2 – no –

The follow table shows how the anonymous access setting and the password file settings affect client access.

Note1: If a password file is specified then if the client sends a username/password then it must be valid otherwise an authentication error is returned.

If it doesn’t send one then none is required and a normal connection results.

Note 2: The client must send a User name and password, but it is not checked. If the client doesn’t send a user name/password then an authentication error code is generated.

Taken from Mosquitto Username and Password Authentication -Configuration and Testing

Questions?

Do you have any questions? Please use the comments form to ask

References:

• Mosquitto ACL -Configuring and Testing MQTT Topic Restrictions
• Configure a Mosquitto Bridge With SSL Encryption

I have developed a number of tools for monitoring MQTT brokers that use node-red.

The reason for using node-red is that it is easy to use, and comes with a dashboard.

However when it comes to monitoring multiple brokers it gets complicated.

My solution it to do the monitoring using Python and then use node-red to log and display the results.

Python MQTT Broker Checker

The broker check monitors to state of a list of MQTT brokers by periodically trying to connect to the broker.

The script makes use of threads so that a failed connections doesn’t cause a big delay.

If it can connect then it flags the broker as UP otherwise the broker is DOWN.

In addition it also logs the connection time interval as this is a measure of how responsive the broker is.

The broker list can be reloaded as an array of dictionary objects. An example is shown below:

{
“broker_name”:”192.168.1.23″,
“port”:1883,
“username”:”steve”,
“password”:”password”,
“SSL”:False
}

In addition you can add/remove brokers dynamically be sending commands using MQTT.

The tool reports the status of the broker on the topic

public/brokers/alerts

it receives commands on the topic

public/brokers/commands/[add/remove]

A command response is sent on the topic

public/brokers/responses

When sending commands the payload is a JSON  object like that shown below and also in the script.

Currently script defaults are

• Maximum threads 4
• Loop interval 30secs
• Max connection time 20secs

The results are sent to an MQTT report broker at the end of each cycle and they can be retrieved by a suitable monitor.

This is what the connection report looks like:

Here are some command examples screen shots using MQTT Explorer

Removing a broker from the Lists

Main Loop

This is a while loop which usually runs indefinitely (count=0). The code is show below:

The broker check is started in a separate thread.Once the maximum number of threads has been started then the script waits (while loop).

Code Download

Questions

If you have any questions require code explanation then let me know and I will assist.

If you have any suggestions,maybe you would like to see the data displayed on a simple web page without node-red. Or perhaps you want email alerts using python etc then please let me know and I will try to incorporate your suggestion into the project

Part 2

In Part 2 we will look at using node-red to display the broker status on a Dashboard.

All of the scripts currently available on this site used the 1.5 to 1.6 client version and so if you upgrade to the new version you will need to make changes to these scripts.

The changes revolve around the callback functions which have been changed to take into account the properties feature of MQTTv5.

Previously when using version 1.x with MQTTv5 you would need to add the properties object to some of the callbacks as scripts written for MQTTv3.1 would fail because they didn’t have the properties object.

As an illustration the callback function for on_connect

Old signature for MQTT v3.1 and v3.1.1 is:
on_connect(client, userdata, flags, rc)

signature for MQTT v5.0 client:
on_connect(client, userdata, flags, reasonCode, properties=None)

Notice the extra properties field.

However existing code written for v3.1.1 used the following callback signature:

connect(client, userdata, flags, rc)

This will still work with MQTTv3.1.1 but you will get an error with MQTTv5

To make the callback work with v3.1.1 and v5 connections use:

on_connect(client, userdata, flags, rc, properties=None)

Version 2 Changes

In version all callbacks that require the properties field have one.

This means that it works with both MQTTv3.1 and MQTTv5.

In addition a flag has been added so that old code written for MQTTv3.1 and version 1.x client will still work with the version2 client.

The flag is shown below:

mqtt.CallbackAPIVersion.VERSION1

and

mqtt.CallbackAPIVersion.VERSION2

This is set at client creation so instead of:

client = mqtt.Client(client_name)

you need to use:

client = mqtt.Client(mqtt.CallbackAPIVersion.VERSION1,client_name)

if you are still using old callbacks or

client = mqtt.Client(mqtt.CallbackAPIVersion.VERSION2,client_name)

for new callbacks.

In addition the reason code (rc) returned in callbacks was previously an integer, but is now an instance of ReasonCode.

Finally new properties have been added to the client class which might be an issue if you have sub classed or added additional properties.

I use a connected_flag in many of my scripts and there is no conflict with the new ones.

You can find all the details and example code on the migration page here

I did an article on python client changes for MQTTv5 a while ago and it is here.

Video Overview

Related tutorial and Resources:

• Paho Python MQTT Client – Understanding Callbacks
• Paho Python MQTT Client-Understanding The Loop
• Paho Python MQTT Client Objects

When you start mosquitto using a basic configuration file like the one shown below mosquitto will listen for connections on both  IPv4 and IPv6.

include_dir /etc/mosquitto/conf.d
log_type error
log_type notice
log_type information
log_type debug
log_type websockets
log_timestamp_format %Y-%m-%dT%H:%M:%S
connection_messages true
log_dest stdout

allow_anonymous true
listener 1883

This is shown in the screen shot below:

Just like with IPv4 you need to know the IPv6 address of the broker or the domain name if using DNS to connect to it.

With IPv4 the IP address is usually static and preferably fixed via DHCP.

Note DHCP can be used to provide static IP addresses.

With IPv6 the addresses can be automatically assigned or can be  static depending on your network configuration.

The screen shot below shows the IPv6 addresses automatically assigned on a raspberry Pi.

If I reboot the machine the IP addresses will be the same.

So all we need to do is use one of these addresses to connect to the broker.

The address starting with 2a00 (global) would be used by an external client, and  the address start with fe80 (link local) would be used on the local network with no routers.

The address starting with fdaa (unique local) would be used on the local network with or without routers.

Because of the length of these addresses and for ease of use making a DNS entry for these addresses would be the best option.

Below is a screen shot of my DNS entry:

You should not that I have an entry for IPv4 and IPv6.

Restricting IP Versions on The Broker

It is possible to restrict the broker to listening on a protocol version using the socket_domain option as shown below:

listener 1883
socket_domain ipv4

will restrict the broker to IPv4

and

listener 1883
socket_domain ipv6

will restrict the broker to IPv6

With no bind address specified for the listener the broker will accept connections on any of the 3 IPv6 addresses. In my case the addresses are:

fdaa:bbcc:ddee:0:d0a7:5928:ac3c:5b56
fe80::a3f2:69f7:ca08:f9f
2a00:23c6:d886:2901:a838:6c7d:851b:8a39

You can get the broker to listen on a particular address only using the following:

listener 1883 fdaa:bbcc:ddee:0:d0a7:5928:ac3c:5b56

Note: it also works with the global address 2a00 but failed to start using the link local address.(not sure why).

Connecting to the broker using a Client

I tried connecting using a Python client and the mosquitto_sub client and using the domain name.

In both cases it connected using IPv6  by default.

This was with the client on a Windows 10 and a raspberry pi machine.

Summary

The mosquitto broker has had support for both IPv4 and IPv6 for a long time and generally to use it there is no need for any extra broker configuration required.

If using IPv6 then Using a Local DNS server is recommended.

Related Tutorials and resources:

• IPv6 Explained for Beginners
• Understanding IPv4 Addressing and Address Classes
• Quick Guide to The Mosquitto.conf File With Examples
• Understanding DNS – Beginners Guide to DNS
• Understanding DHCP

Before you can use this client you need to install it into the Arduino Library.

Go to the Library manager and so a search for MQTT. You will find quite a few listings scroll down the select the ArduinoMQTT client.

The documentation for the library is here, but I did not find it particularly helpful.

There are also code examples on the Github page here.

About This Tutorial

In this tutorial we will look at using the ArduinoMQTT Client Library with code examples for Arduino Uno (Ethernet), ESP8266 and ESP32 boards.

We will start by going through the basic functions available and how and in what order they are used.

We will see how you connect to a broker,subscribe to topics and publish messages and finally how we receive incoming messages. Working Demo scripts are available for download at the end.

MQTT Client Basics

For any MQTT client we have 4 distinct stages

1. Connect
2. Subscribe
3. Publish
4. Disconnect

Steps 2 and 3 are optional.

Before we connect we need to create a client object.

This client object is tied to the underlying physical connection i.e. (Ethernet or Wi-Fi).

For Wi-FI you will see something similar to this near the top of the code

WiFiClient espClient;
MqttClient mqttClient(espClient);

All MQTT clients are required to have a unique client id .

and for Ethernet we have

EthernetClient ethClient;
MqttClient mqttClient(ethClient);

You need to set this before connecting along with any other connection parameters like clean session, username/password etc.
To set the client id use:

mqttClient.setId("esp32-1");

were esp32-1 is the client name.

Some Other commands are :

• MqttClient.setKeepAliveInterval(unsigned long interval)
• MqttClient.setCleanSession(bool cleanSession)
• void MqttClient.setUsernamePassword(const char* username, const char* password)

Once you have set the client options you can then connect to the broker using the mqttClient.connect(broker,port)  call as shown below:

const char broker[] = "192.168.1.33";
int        port     = 1883;
mqttClient.connect(broker,port);

The call returns true if successful and false if not, and so it is normal to put this call in a while and if loop the psuedo code is:

while not connected
try and connect

Actual code is below and uses the mqttClient.connected() function call to test the connection (0=connected,1=not connected).

  
void reconnect() {
while (mqttClient.connected()==0) {
      mqttClient.setId("esp32-1");
      Serial.print("Attempting to connect to the MQTT broker: ");
      Serial.println(broker);

  if (!mqttClient.connect(broker, port)) {
    Serial.print("MQTT connection failed! Error code = ");
    Serial.println(mqttClient.connectError());
         Serial.print("failed, rc=");
      Serial.println(" try again in 5 seconds");
      delay(5000);
   }
 else {
  
  Serial.println("Connected to the MQTT broker!");
  Serial.println();
 Serial.println("connected so subscribe");
 mqttClient.subscribe(topic);
    }
  }//end while
} //end function

In many online scripts this code is placed in the setup, but if you do this the client will not reconnect so I place it inside a function and call the function from the loop. You can see this in the code above.

Receiving Messages

In order to receive messages you will need to:

• subscribe to a topic or topics
• configure a callback to handle the received message

To subscribe to a topic or topics use mqttClient.subscribe(topic) function.

You should note that you can use wild cards when subscribing see Understanding MQTT topics for details.

The subscribe code is paled in the connect loop and you can see it in the code above– mqttClient.subscribe(topic).

The call back consists of a callback function and a set up. Sample call back function is shown below:

void onMqttMessage(int messageSize) {
  // we received a message, print out the topic and contents
  Serial.println("Received a message with topic '");
  Serial.print(mqttClient.messageTopic());
  Serial.print("', length ");
  Serial.print(messageSize);
  Serial.println(" bytes:");

  // use the Stream interface to print the contents


  while (mqttClient.available()) {
   Serial.print((char)mqttClient.read());
  }
  Serial.println();

}

In the setup you need to register this callback using

 mqttClient.onMessage(onMqttMessage); //callback

mqttClient.poll() Function

This function needs to be called on a regular basis and it processes the receive buffer.

Without this function you want see any incoming messages including control messages like puback,pingresp.

I did a tutorial on the loop in relation to python but it is applicable to all MQTT clients.

The arduino client is single threaded and so it must be placed in the loop.

In addition the loop should not  have any delays . Here is my loop example:

void loop() {
  // call poll() regularly to allow the library to send MQTT keep alives which
  // avoids being disconnected by the broker
  mqttClient.poll();
 if (mqttClient.connected()==0) {
    reconnect();
  }
  // to avoid having delays in loop, we'll use the strategy from BlinkWithoutDelay
  // see: File -> Examples -> 02.Digital -> BlinkWithoutDelay for more info
      unsigned long now = millis();
     if (now - lastMsg > 5000) {
    lastMsg = now;
    Serial.print("Publishing message: ");
    Serial.println(msg);
    publish_msg(topic, msg);
    //Serial.printf("RSSI: %d dBm\n", WiFi.RSSI());
    WiFi.printDiag(Serial);
 
     }
    
}

Notice we do not use delay(5000) to control the print as this would cause a delay.
Also notice that the connection is tested and if it has failed the reconnect function is called.

Publishing Messages

This is done with the calls to three functions
beginMessage(),print() and end. Actual code is shown below:

    mqttClient.beginMessage(topic);
    mqttClient.print(msg);
    mqttClient.endMessage();

The begin message function can also take the retain flag and QOS,
e.g to publish with a QOS or 1 and retain flag set use:

mqttClient.beginMessage(topic,true,1);

User names and Passwords

If the broker requires authentication you can set the username and password the you can set this before you connect using

mqttClient.setUsernamePassword(“username”, “password”);

Last Will Message

The last will message and topic need to be set before connecting and use the same format as the publish as shown in the code below.

   char willTopic[]        = "connected/esp-32";
  char willPayload[] = "not connected";
  bool willRetain = true;
  int willQos = 1;

  mqttClient.beginWill(willTopic, willRetain, willQos);
  mqttClient.print(willPayload);
  mqttClient.endWill();

Code Examples

The example code will subscribe to a topic and publish a message on the same  topic every 5 seconds.

It also sets a will message and uses authentication.

The code for all three boards -arduino, ESP32 and ESP8266 is virtually identical and only differs in the libraries.

At the top of the ESP code you will see:

#include <ArduinoMqttClient.h>
#include <ESP8266WiFi.h> //use this for ESP8266
//#include <WiFi.h> //use this and comment out above when using ESP32 boards
//#include "arduino_secrets.h"
//#include "arduino_secrets.h"
//#include "arduino_secrets.h"
//#include "arduino_secrets.h"
///////please enter your sensitive data in the Secret tab/arduino_secrets.h
char ssid[] = "Akehurst";    // your network SSID (name)
char pass[] = "akehurst122012";    // your network password (use for WPA, or use as key for WEP)

WiFiClient espClient;
MqttClient mqttClient(espClient);

for esp8266 we use:

#include <ESP8266WiFi.h>

and for ESP32 we use

#include <WiFi.h>

Arduino Ethernet

For the arduino board with Ethernet we have:

#include <ArduinoMqttClient.h>
#include <Ethernet.h>

// Update these with values suitable for your network.
byte mac[]    = {  0xDE, 0xED, 0xBA, 0xFE, 0xFE, 0xED };
IPAddress ip(192, 168, 1, 89); //address of client
EthernetClient ethClient;
MqttClient mqttClient(ethClient);

Full Demo code

Related Tutorials and Resources:

• Send and Receive Integers and Floats with Arduino over MQTT
• Arduino -Sending and Receiving JSON Data over MQTT
• Using the Arduino PubSub MQTT Client

We will not be using callbacks. If you use callbacks then second thread is started automatically to process the callbacks.

If you don’t declare callbacks then you need to manually check for received messages using the MQTTClient_receive function.

This function needs to be called often, and is often found in an infinite loop.

Basic Code Structure

In the previous tutorial MQTT Client Basics we looked at the general code structure:

The Basic code structure used is:

• Create a client object
• Set the options to connect to an MQTT server
• Set up callback functions if multi-threaded
• Subscribe to any topics.
• Publish  messages
• Handle any incoming messages
• Disconnect the client
• Free any memory being used by the client

Now will look at an example client and the code.-

The example  code simulates a simple sensor that publishes data at regular intervals (10 secs) on topic MQTT-OUT and can also receive data (commands) on the topic MQTT-IN.

At the top of the code we create the client and then initialise the connection options, and the publish message structures.
The delivery token is actually the msgid that is present on messages published with QOS of 1 or 2.

MQTTClient Client; 
MQTTClient_connectOptions conn_opts = MQTTClient_connectOptions_initializer;
MQTTClient_message pubmsg = MQTTClient_message_initializer; MQTTClient_deliveryToken token;

Because we are using a single thread we need to call MQTTClient_receive function at regular intervals.

This is a blocking call so we set the time out to be short (0.01) so as to not cause a delay in the loop.

We use the time function with a flag to determine when to call the publish function (every 10 seconds) . The code is below

while (1)
{

rc=MQTTClient_receive(Client,&topicName,&topiclen,&message,0.01);

if(topicName){
printf("Message received on topic %s is %.*s.\n", topicName, message->payloadlen, (char*)(message->payload));
}
seconds=time(NULL);
if(seconds%10==0 && pub_flag==0)
{
printf("publishing");
pub_flag=1; //don't want to publish too many messages
pub_msg();
}
if(seconds%10==1&& pub_flag==1)
{

pub_flag=0; //reset
;
}

So as to avoid having to pass parameters into the pub_msg function I declare most of the variables outside of the main function.

The code will publish messages and receive messages until stopped using CTRL+C and the complete code is shown below:

#include 
#include 
#include 
#include "MQTTClient.h"
#include 
#include 

#define ADDRESS     "tcp://localhost:1884"
#define CLIENTID    "ExampleClientPub"
#define TOPIC_IN    "MQTT-IN"
#define TOPIC_OUT    "MQTT-OUT"
#define PAYLOAD     "testing"
#define QOS         1
#define TIMEOUT     10000L
time_t seconds;

MQTTClient Client;
MQTTClient_connectOptions conn_opts = MQTTClient_connectOptions_initializer;
MQTTClient_message pubmsg = MQTTClient_message_initializer;
MQTTClient_deliveryToken token;

void pub_msg()
{
    pubmsg.payload=PAYLOAD;
    pubmsg.payloadlen = (int)strlen(pubmsg.payload);
    pubmsg.qos = QOS;
    pubmsg.retained = 0;
    int rc;

    MQTTClient_publishMessage(Client, TOPIC_OUT, &pubmsg, &token);
    printf("Waiting here for up to %d seconds for publication of %s\n"
            "on topic %s for client with ClientID: %s\n",
            (int)(TIMEOUT/1000), PAYLOAD, TOPIC_OUT, CLIENTID);
    rc = MQTTClient_waitForCompletion(Client, token, TIMEOUT);
    printf("Message with delivery token %d delivered\n", token);
}

int main(int argc, char* argv[])
{

    int rc;
    rc=MQTTClient_create(&Client, ADDRESS, CLIENTID,
        MQTTCLIENT_PERSISTENCE_NONE, NULL);

    char * topicName=NULL;
    int topiclen;
    MQTTClient_message *message=NULL;

    conn_opts.keepAliveInterval = 120;
    conn_opts.cleansession = 1;
    if ((rc = MQTTClient_connect(Client, &conn_opts)) != MQTTCLIENT_SUCCESS)
    {
        printf("Failed to connect, return code %d\n", rc);
        exit(EXIT_FAILURE);
    }
    rc=MQTTClient_subscribe(Client,TOPIC_IN,QOS);
    printf("Subscribed %d \n",rc);
int pub_flag=0;
while (1)
{
    
rc=MQTTClient_receive(Client,&topicName,&topiclen,&message,0.01);

if(topicName){
        printf("Message received on topic %s is %.*s.\n", topicName, message->payloadlen, (char*)(message->payload));
}
seconds=time(NULL);
//sleep(5);
if(seconds%10==0 && pub_flag==0)
{
	printf("publishing");
	pub_flag=1; //don't want to publish too many messages
pub_msg();
}
if(seconds%10==1&& pub_flag==1)
{

	pub_flag=0; //reset
;
}
}

    printf("end Sleeping \n");
    //sleep(100);
    MQTTClient_disconnect(Client, 10000);
    MQTTClient_destroy(&Client);
    return rc;
}

Related Tutorials and resources:

• Paho MQTT C Client Introduction
• Paho Python MQTT Client-Understanding The Loop