Testing IoT

This page looks at test equipment for IoT from the radio access side. We start with what an R&D engineer needs from a test system, and we turn that into a wish list. Then we compare the wish list with the tools that were actually on the market in 2015 and 2016, for cellular IoT and for RF measurement.

What do you want as an R&D engineer ? and what can you sell in the development market ?

There would be a lot of demand on Software side including the data processing / analysis (e.g, Big Data/ Data Science), but I am not an expert on these area and nothing to talk about it. I don't think the readers of this page are not in this area either. I am an engineer in Radio Access Technology. So I would mainly comments on this area.

First let's think of what kind of radio technology will be used in whole IoT network. Followings are the list of technologies that are most commonly talked about as of now (Jun 2015). I don't think you will see completely new technologies even when IoT market gets matured. Definately you will see a lot of evolution within these technology, but you wouldn't see completely unheard-of technology. Especially as terminal devices, it is likely that ZigBee, 6LoPAN and WLAN will be the dominant technologies.

ZigBee
6LoPAN
WLAN
Bluetooth
LTN
Cellular Technology (5G, LTE, WCDMA etc)

Except LTN, everything is already there. Do we need any further research and development ?

Basic technology is already available, but there are long way to go to integrate these component technology into the IoT system we are dreaming of. Roughly there are several items to be improved

Lower Energy Consumption
Lower Cost
Improved Networking capability
Common/More Compact Protocol
Improved Security

These are also already implemented to some degree, but there should be a lot to be improved and I think major technical effort for next several years will be around these items.

Wish to Have

With things described above in mind, you may easily guess what kind of development tool/testing tool will be at the top of wishlist. The previous section named five items that still need work: lower energy, lower cost, networking, a common protocol and security. A good test tool should help you work on each of them, and the list below is written with that goal in mind.

I wish to have following development/testing tool if I were the developer in this area.

I want to have a test equipment that has following features :

i) ZigBee, 6LoPAN, WLAN, Bluetooth, LTN, Cellular Technologies all in one box

ii) Supports full stack from RF to IP

iii) Supports capability of testing multiple DUT simultaneously and allows end-to-end communication among DUTs

iv) Modular implementation of Core Network software so that the equipment radio stack can interface with external core network component

vi) Enough flexibility to change all the internal configuration so that we can try tweaking various protocol parameters

vii) Each technology combined together in modular base so that the developer can chose only one option to full options depending on their needs

viii) Powerful logging capability for all the technologies for easy troubleshooting and debugging

Each wish on this list answers one of those five items. Testing multiple DUTs with end-to-end communication checks the networking capability. An IoT system is many devices talking through one network, not one phone talking to one base station. The full stack from RF to IP and the powerful logging check the protocol, because most IoT failures appear as an interaction between layers. Flexible internal configuration lets you tune the parameters that decide energy consumption, such as the PSM and eDRX timers in cellular IoT. Finally, the modular design keeps the cost down, because a developer pays only for the technologies the product uses.

One box should cover many radios : An IoT product often combines several radio technologies, so a tester that covers only one of them sees only part of the system.
Multiple DUTs matter more than peak throughput : An IoT test is about many low rate devices sharing one network.
Logging and configurability are the debugging tools : They let you see and change the protocol behaviour that decides battery life.

Back to Reality

Am I asking for too much ? Let's compare the wish list with what the market offered when this page was written. The answer is different for the protocol side and for the RF side, so the discussion below covers them one after the other.

yes, I agree if I want to have this kind of development/testing tool right now. But I am pretty sure that we will see this kind of equipment in the market as industry grows. Recently, SDR (Software Defined Radio) has been drastically evolved and many of the equipment vendors come up with slot based (common BUS based) hardware and generic FPGA programming capability. It would be difficulty to implement 5G/LTE completely with SDR, but all the other technologies can be implemented by SDR or simple FPGA. So it would be possible to design a hardware with the hybrid of dedicated technology and SDR.

Even in LTE based IoT, you might already have seen very low cost and flexible/SDR based solutions in the market. Currently there are strong evolution towards IoT in LTE side. Followings are some LTE categories that is mainly designed for IoT/M2M/MTC.

I don't think Category 1 is efficient enough for IoT. It is just a kind of best effort solution as an initial movement (as of 2015). Category 1 would not restrict the system bandwidth to the narrowest band. They may apply Category 1 in any system bandwidth, but if the system BW goes wider there would be some difficulties to minimize the device cost and energy consumption. It is higly likely that Category 0 would go with 1.4 Mhz (even thought it might not be mandatory) and Half duplex in some case and Category 0 with PSM(Power Saving Mode), it can be taken as pretty mature IoT solution. If you go with Category M, you would have more optimized IoT solution both for device and test equipment not only in terms of operation but also in terms of the cost.

If the system Bandwith goes only with 1.4 Mhz or less, I think pure hardware cost for the test equipment can be only a few thousand USD. You can even use the off-the-shelf RF Modules for SDR(Software Defined Radio) with even lower cost. However, if you are in the field of device development, don't think of building your own test system on your own. Even though you may get the hardware at very low cost, implementing LTE protocol stack is not an easy task. Just purchasing the test equipment would be much, much, much cheaper. However, if you are in the field of designing/developing test equipment, you may be considering the developing super-low cost IoT/MTC/M2M only test solution like "low Hardware + SDR".

In terms of Cellular IoT(e.g, Cat 1, Cat 0, Cat M) protocol,

as of Jun 2016, I haven't seen any equipment that supports all the protocols. I know there are a couple of equipments that supports Cat 1, Cat 0, but I haven't seen any equipment that supports Cat M protocol as of now.
as of Sep 2016, Amarisoft announced that their SDR solution support both LTE M1 and LTE-NB(LTE-M2) and Anritsu MD8430 support LTE M1.

In terms of RF, you have so many different choices. If you are working mostly on software side and integration stage using the ready-made IoT modules, you may not be interested in very accurate RF measurement or high quality demodulation capability. You may say 'I just want to get quick/light/handy solution just to check if 'some' RF signal is going out of my device. In this case, you may use any kind of spectrum analyzer you can get. In some cases, you can even use 20 Dollor solution like RTL based device just to detect some signal(See [2]). Of course, if you need more accuracy or more dynamic range, you would need to use those spectrum analyzers from major vendors. The tricky thing is the case when you want to demodulate the signal from your IoT devices. Depending on the technologies you are using, you may be able to get some vector signal analyzer with specific demodulation capability for the technology you are using, however there would be many cases where you cannot find any equipment that support the demodulation capability for the technology that you are using.

Reference

[1] Testing & Certifying IOT Products (YouTube)

[2] Scanning LTE Signal with RTL

[3] The Challenges Of Wireless Testing In The IoT (YouTube)

[4] IoT Test & Compatibility Lab (YouTube)