It seems certain that we will employ the Massive MIMO as one of the core technology in 5G. However, it doesn't mean that this technology is already mature (complete). There are many things to be improved or resolved about this technology. This page would list up some of the area that are commonly listed as further study items.

How to arrange Antenna ? 

As you know, in Massive Antenna you would have huge number of antenna. Now you would have questions .. how should I arrange those antenna to achieve the best performance ?

Following illustration shows the various type of antenna arrangement that I have seen from various technical materials.

What would be the best arrangement ? Will there be any new method of arrangement ?

These are questions that should be answered from further research.

How to model the 3D Channel ?

If you arrange the antenna as (B), (C), (D), you would point the direction of the beam both in horizontal and in vertical direction. If you combine the two direction, you can point the beam to any direction in 3D space (at least almost of half of 3D sphere). It is good, but there is complication as well. Now you need to consider the channel factors in all of those direction and you would need mathematical models to take those 3D factors into account.

This kind of channel models are one of the area that require a lot of further study.

How to apply it for FDD Operation ?

I think this is the biggest drawback of Massive MIMO (at least as of now). In order to perform the best beamforming, you need to have accurate (detailed) information of channels that is continuously changing. In order to get this kind of information, you need to get the report from UE on downlink channel quality. To do this, you need to allocate a lot of resources for downlink reference signal which would cause serious waste of resource. In FDD, we don't have any good idea to get the channel information without using this kind of channel quality report based on reference signal.

However in TDD, we can use some alternative technology which may not require this kind of UE reporting. In TDD, we are using the same frequency band for both downlink and uplink. So if Network can estimate the uplink channel quality from UE transmission signal, you can use that information as downlink channel quality. Therefore, in TDD you can create pretty optimized beam without getting explicit channel quality report from UE.

Of course, the estimation derived from uplink signal may not be exactly same as downlink signal because timeslot for uplink and downlink is different. So the channel estimation for UL at a certain timeslot may not be exactly same as the downlink slot. However, this is the most commonly accepted and practiced idea as of now.

Due to this reason, most of Massive MIMO implementation are being done in TDD mode.

How to generate wide beam from a large array ?

One of the key idea behind Massive MIMO is to increase Antenna gain by constructively adding up multiple antenna output to a single beam and by this process the width of the resulting beam tends to get narrower. We can say this narrow beam is good in terms of energy density, but it also means the area covered by a beam would be very narrow. It means that the beamforming and directing should be very quick and accurate to properly focus on the target UE, but this is not always simple and easy especially when the UE is in fast moving condition.

So it would be necessary to widen the beam width without sacrificing too much of the performance of the massive MIMO.

How to Calibrate the Antenna System ?

Anybody who has experience of RF/mmWave design or testing would understand that the complexity and difficulties of design/testing would increase exponentially as you have more signal path. Even assuming that the design is properly done, you have to make it sure that all of the signal path and antenna are properly calibrated in order for the antenna system to work as intended. Calibrating those huge number of antenna path is definitely challenging task.

How to handle the complexity of Scheduling and Precoding ?

As you know, the biggest motivation of the Massive MIMO is to increase the directivity and gain for specified target devices. Another motivation (or requirement caused by beam forming) is to implement MU-MIMO (Multi-User MIMO). However, the scheduling and Precoding would get more complicated as more antenna is used and more user is targetted. How to handle this kind of situation would be a big question. Just to increase DSP power ? or to come up with a new/smart mathematical method to handle this without increasing DSP requirement too much ?