IoT(Internet Of Things)

 

 

 

What is MQTT ?

 

MQTT stands for Message Queuing Telemetry Transport. or MQ Telemetry Transport. It is extremly simple and lightweight messaging protocol based on publish / subscribe mechanism.

 

In case of current high bandwidth connection (like ordinary internet, WiFi, Cellular), we already have so wide varieties of protocols like http, ftp, sip etc. Most of these protocols is generating relatively large traffic even though it can be considered as almost nothing with recent broadband communication technology and it would not be a big issue even with IoT devices if the devices is based on celluar network (e.g, WCDMA or LTE). However, this kind of protocol can cause untolerable amount of traffic for the devices that is based on very low throughput and small sized packet system like 6LoWPAN. For this kind of low throughput/small packet based system, we would need very light , simple and small sized application layer protocol. MQTT is one of the most popular protocol that meets this requirement.

 

For learning any kind of protocol, my personal approach is almost always same. To understand the packet (or frame) structure and then following through a couple of example protol sequences line by line. Sometimes I practice decoding the captured hex stream by hand based on Packet Structure specification. I will do the same thing here.

 

Followings are the topics that will be covered in this post.

 

 

Protocol Overview

 

Protocol Sequence of MQTT is very simple as illustrated below. First, Client (IoT End Device) request connection to a Broker (a kind of server). If the connection accepted, the device (client) would send 'SUBSRIBE' message saying "I want to get this and this kind of data. So please send me the data whenever those data is upadated on your side (Broker)". If this request is accepted by Broker, the broker would send the data to the device whenever any new data comes into the database. For more details on profocol sequence, see 'Protocol Sequence' section. For more details on the structure and contents of each message, refer to Packet structure section.

 

 

 

Packet Structure

 

Even though the description and examples came from myself, the packet structure shown here is based on MQTT V3.1 Protocol Specification by International Business Machines Corporation (IBM) Eurotech. Format representation is modified a little bit for easier understanding for myself.

 

< Message Header : Fixed length >

 

Bits

7

6

5

4

3

2

1

0
Byte 1

Message Type

DUP Flag

QoS Level

RETAIN
Byte 2

Remaining Length

 

Message Type - 4 bits :

Mnemonic

Enumeration

Description
Reserved

0000 (0)

 Reserved
CONNECT

0001 (1)

 Client request to connect to Server
CONNACK

0010 (2)

 Connect Acknowledgement
PUBLISH

0011 (3)

 Publish message
PUBACK

0100 (4)

 Publish Acknowlegement
PUBREC

0101 (5)

 Publish Recieved (assured delivery part 1)
PUBREL

0110 (6)

Publish Release (assured delivery part 2)
PUBCOMP

0111 (7)

Publish Complete (assured delivery part 3)
SUBSCRIBE

1000 (8)

 Client Subscribe Request
SUBACK

1001 (9)

 Subscribe Acknowlegement
UNSUBSCRIBE

1010 (10)

 Client Unsubscribe Request
UNSUBACK

1011 (11)

 Unsubscribe Acknowlegement
PINGREQ

1100 (12)

 PING Request
PINGRESP

1101 (13)

 PING Response
DISCONNECT

1110 (14)

 Client is disconnecting
Reserved

1111 (15)

 Reserved

 

DUP Flag (Duplicate Flag) - 1 bit :

 

Value

Description

0

 This is not a duplicate message

1

 This is a duplicate message, meaning 'I am trying to send the same message again'

 

QoS Level - 2 bits :

 

Value

Description

00 (0)

 At most once. Fire and forget. (Just send and no request for Ack)

01 (1)

 At least once. (Require Ack from the reciever)

10 (2)

 Exactly once. (Assured delivered. No duplication allowed)

11 (3)

 Reserved

 

RETAIN - 1 bit : Applies only to PUBLISH message

 

Value

Meaning

0

  

1

 The Server should hold on to the message after it has beed delivered to the current subscribers.

 

 

< CONNECT >

 

Fixed Header :

Bits

7

6

5

4

3

2

1

0
Byte 1

0

0

0

1

X

X

X

X
Byte 2

Remaining Length

 

Message Header :

Bits

7

6

5

4

3

2

1

0
Byte 1

Protocol Name Length MSB
Byte 2

Protocol Name Length LSB
Byte 3

'M'
Byte 4

'Q'
Byte 5

'I'
Byte 6

's'
Byte 7

'd'
Byte 8

'p'
Byte 9

Protocol Version
Byte 10

Connect Flag
Byte 11

Keep Alive Timer MSB
Byte 12

Keep Alive Timer MSB
Byte 13 ...

Payload

 

Example :

 

0000  10 21 00 06 4d 51 49 73 64 70 03 02 00 3c 00 13   .!..MQIsdp...<..

0010  63 6c 69 65 6e 74 49 64 2d 75 56 78 53 6a 43 41   clientId-uVxSjCA

0020  4b 71 41                                          KqA

 

10 21 (hex) - 00010000 00100001 (bin) : Header

    0001 - Message Type - CONNECT

    0 - DUP Flag

    00 - QoS Level

    0 - RETAIN

     00100001 - Remaing Length : 33 Bytes

 

00 06 4d 51 49 73 64 70 03 02 00 3c 00 13 (hex) : Header of CONNECT message

     00 - Protocol Name Length MSB

     06 - Protocol Name Length LSB

     4d - 'M'

     51 - 'Q'

     49 - 'I'

     73 - 's'

     64 - 'd'

     70 - 'p'

     03 - Protocol Version

     02 - 00000010 - Connect Flag

       0            : User Name Flag

        0           : Password Flag

         0          : WILL Retain

          00        : WILL QoS

            0       : WILL Flag

              1     : Clean Session

     00 - Keep Alive MSB

     3c - Keep Alive LSB

 

 

< CONNECT ACK>

 

Fixed Header :

Bits

7

6

5

4

3

2

1

0
Byte 1

0

0

1

0

X

X

X

X
Byte 2

0

0

0

0

0

0

1

0

 

Message Header :

Bits

7

6

5

4

3

2

1

0
Byte 1

Topic Name Compression Response
Byte 2

Connect Return Code

 

Return Code :

Enumeration

Hex

Meaning
0

0x00

 Connection Accepted
1

0x01

 Connection Refused : Unacceptable Protocol Version
2

0x02

 Connection Refused : Identifier Rejected
3

0x03

 Connection Refused : Server Unavailable
4

0x04

 Connection Refused : Bad User Name or password
5

0x05

 Connection Refused : Not Authorized
6-255    

 

Example :

 

0000  20 02 00 00

 

20 02 (hex) - 00100000 00000010 (bin) : Header

    0010 - Message Type - CONNACK

    0 - DUP Flag

    00 - QoS Level

    0 - RETAIN

     00000010 - Remaing Length : 2 Bytes

 

00 00 (hex) : Header of CONNACK message

     00 - Topic Name Compression Response (Not used)

     00 - Return Code - Connection Accepted

 

 

< PUBLISH >

 

Fixed Header :

Bits

7

6

5

4

3

2

1

0
Byte 1

0

0

1

1

0

*

*

0
Byte 2

Remaining Length

 

Message Header and Payload

Bits

7

6

5

4

3

2

1

0
Byte 1

Topic Name Length MSB
Byte 2

Topic Name Length LSB
Byte 3

First Ascii Character of Topic Name
Byte 4

Topic Name Length LSB
Byte 5

First Ascii Character of Topic Name
Byte 6

 
Byte ...

 
Byte N-1

 
Byte N

Last Ascii Character of Topic Name
Byte N+1

Message ID MSB
Byte N+2

Message ID LSB
Byte ...

Message Data

 

Example :

 

0000  35 17 00 0b 74 65 73 74 74 6f 70 69 63 2f 32 00   5...testtopic/2.

0010  01 73 61 64 73 64 61 73 64                        .sadsdasd

 

35 17 (hex) - 00111001 00011011 (bin) : Header

    0011 - Message Type - PUBLISH

    1 - DUP Flag

    00 - QoS Level

    1 - RETAIN

    00011011 - Remaing Length : 23 Bytes

00 (hex) - Topic Name MSB

0b (hex) - Topic Name LSB : 11 bytes

00 (hex) - Topic Name Length MSB

0b (hex) - Topic Name Length LSB : 11 Bytes

74 65 73 74 74 6f 70 69 63 2f 32 (hex) - testtopic/2

00 (hex) - Message ID MSB

01 (hex) - Message ID LSB

 

 

< SUBSCRIBE >

 

Fixed Header

Bits

7

6

5

4

3

2

1

0
Byte 1

1

0

0

0

0

0

1

X
Byte 2

Remaining Length

 

Message Header and Payload

Bits

7

6

5

4

3

2

1

0
Byte 1

Message ID MSB
Byte 2

Message ID LSB
Byte 3

Topic Name Length MSB
Byte 4

Topic Name Length LSB
Byte 5

First Ascii Character of Topic Name
Byte 6

 
Byte 7

 
Byte 8

 
Byte 9

 
Byte ...

 
Byte N-1

 
Byte N

Last Ascii Character of Topic Name
Last Byte

X

X

X

X

X

X

*

*

 

Example :

 

0000  82 10 00 01 00 0b 74 65 73 74 74 6f 70 69 63 2f   ......testtopic/

0010  23 02                                             #.

 

82 10 (hex) - 10000010 00010000 (bin) : Header

    1000 - Message Type - SUBSCRIBE

    0 - DUP Flag

    01 - QoS Level

    0 - RETAIN

    00010000 - Remaing Length : 16 Bytes

00 (hex) - Message ID MSB

01 (hex) - Message ID LSB

00 (hex) - Topic Name Length MSB

0b (hex) - Topic Name Length LSB : 11 Bytes

74 65 73 74 74 6f 70 69 63 2f 23 (hex) - testtopic/#

02 (hex) - 00000010 (bin) - Requested QoS : 2

 

 

< SUBACK >

 

Fixed Header:

Bits

7

6

5

4

3

2

1

0
Byte 1

1

0

0

1

X

X

X

X
Byte 2

Remaining Length Header

 

Message Header and Payload:

Bits

7

6

5

4

3

2

1

0
Byte 1

Message ID MSB
Byte 2

Message ID LSB
Byte 3

Granted QoS
Byte 4

Granted QoS

 

Example :

 

0000  90 03 00 01 02

 

09 03 (hex) - 10010000 00000011 (bin) : Header

    1001 - Message Type - SUBACK

    0 - DUP Flag

    00 - QoS Level

    0 - RETAIN

     00000011 - Remaing Length : 3 Bytes

 

00 01 02 (hex) : Header of SUBACK message

     00 - Message ID MSB

     01 - Message ID LSB

     02 - Granted QoS : 2

 

 

Protocol Sequence

 

Following is one example protocol sequence that I captured from my trial with HiveMQ Websocket Client. You may get a little bit different variations if you try on your own, but overall procedure and message structure would be similar to the example shown here. Give it a try on your own and you will get used to it very quickly. MQTT is one of the simplest protocol you can see in wireless communication area and I think this would be the biggest reason why MQTT is adopted as one of the dominant protocols in IoT.

 

Step

Direction

Message

Memo

1

Client --> Broker

 CONNECT

ex > Connect with Client id = -uVxSjCAKqA

2

Client <-- Broker

 CONNACK Connection Acknowledged

3

Client --> Broker

 SUBSCRIBE ex > Subscribe testtopic/#

4

Client <-- Broker

 SUBACK Subscription Acknowledged

5

Client <-- Broker

 PUBLISH ex > Publish testtopic/2 value

6

Client <-- Broker

 PUBLISH ex > Publish testtopic/1 value

7

Client --> Broker

 PUBREC Publish Recieved

8

Client <-- Broker

 PUBREL Publish Release

9

Client --> Broker

 PUBREC Publish Recieved

10

Client --> Broker

 PUBREL Publish Release

11

Client --> Broker

 PUBCOM Publish Complete

12

Client --> Broker

 PUBCOM Publish Complete

13

Client --> Broker

 PUBLISH ex > Publish testtopic/1 value

14

Client <-- Broker

 PUBLISH ex > Publish testtopic/1 value

15

Client --> Broker

 PINGREQ Ping Request

16

Client <-- Broker

 PINGRESP Ping Response

17

Client --> Broker

 DISCONNECT  

 

 

Can I give it a quick try ?

 

Yes, Try with HiveMQ Websocket Client. This is what I used to capture the most of sample log for this page.

 

 

Bits

7

6

5

4

3

2

1

0
Byte 1                
Byte 2