The publisher just completed its 24th 5G NR benchmark study. For this endeavor they collaborated with Accuver Americas and Spirent Communications to conduct an independent benchmark study of the Rakuten LTE, 5G, and 5G mmWave networks in Tokyo, Japan.
Highlights of the Report include the following:
Acknowledgements
This study was conducted in collaboration with Accuver Americas (XCAL-Solo and XCAP) and Spirent Communications (Umetrix Data). The publisher is responsible for the data collection and all analysis and commentary provided in this report.
Methodology
Testing took place in April in Tokyo, Japan over a period of three days. The publisher leveraged five smartphones, including two smartphones with DM logging capabilities. They conducted full buffer downlink and uplink data transfers, using HTTP and UDP protocols. Tests involved walk tests and stationary tests. They also did latency/jitter stress tests using a low bit rate UDP data transfer to the Umetrix Data server, which was also located in Japan.
Why it matters
The 5G portion of the Rakuten network leverages the Open RAN network architecture. From the perspective of this testing, the Open RAN architecture didn’t define the results we obtained, but the architecture did enable the inclusion of new vendors, which are not typically associated with today’s 5G networks. This study largely focuses on the performance of these respective components and their respective vendors.
Key Features Supported
The publisher found that many of the key features commonly found with today’s traditional LTE and 5G networks are supported in the Rakuten network. There was concurrent LTE and 5G data transfers in the downlink and uplink directions (PDCP combining), including with 5G mmWave. Likewise, there was 256QAM and 4×4 MIMO on both LTE and mid-band 5G. There wasn’t features, such as LTE carrier aggregation - the operator only has a single LTE band - while mmWave uplink was limited to a single 100 MHz channel, as are most networks in the world.
Room for Improvement
Although the network supported many key features, the publisher found opportunities for improvement. Schedule inefficiencies, in particular with the HTTP protocol, resulted in 5G spectral efficiency which lagged that of the operator’s LTE network and which was well below what was experienced in other networks tested. LTE performance, when operating in parallel with 5G (PDCP combining) also underperformed relative to LTE when there wasn’t a 5G radio bearer. 5G mmWave performance was consistent with the RF conditions, but the RF conditions where we found mmWave signals were never optimal.
Table of Contents
1.0 Executive Summary
2.0 Key Observations
3.0 Performance Results and Analysis
3.1 Downlink HTTP Walk Test
3.2 Uplink HTTP Walk Test
3.3 HTTP and UDP Comparative Analysis
3.3.1 Downlink
3.3.2 Uplink
3.3.3 Speedtest Versus HTTP
3.3.4 Simultaneous
3.3.5 mmWave (Band n257)
3.3.5.1 Downlink
3.3.5.2 Uplink
3.3.6 Downlink and Uplink Latency Stress Test Results
4.0 Test Methodology
5.0 Final Thoughts
Index of Figures & Tables
Figure 1. KDDI 5G Coverage
Figure 2. NTT DoCoMo 5G Coverage
Figure 3. Rakuten 5G Coverage
Figure 4. Downlink Walk Test
Figure 5. Downlink Walk Test Mapped to Rakuten Coverage Map
Figure 6. Band n77 RSRP Distribution
Figure 7. Band n77 SINR Distribution16
Figure 8. LTE and Band n77 PDSCH Throughput Time Series
Figure 9. LTE and Band n77 PDSCH RB Allocations Time Series
Figure 10. LTE and Band n77 Throughput Distribution and Average Values
Figure 11. LTE and Band n77 Modulation Scheme Distributions
Figure 12. LTE and Band n77 MIMO Rank Distributions
Figure 13. LTE and Band n77 Spectral Efficiency
Figure 14. LTE Throughput and RB Allocation Time Series
Figure 15. LTE and 5G Throughput Time Series
Figure 16. 5G Uplink MCS and PUSCH BLER Time Series
Figure 17. 5G RSRP and PUSCH BLER Time Series
Figure 18. LTE and 5G PUSCH RBs and BLER Time Series
Figure 19. LTE and 5G PDSCH Throughput Time Series
Figure 20. LTE and 5G PDSCH RBs Time Series
Figure 21. Key Stats
Figure 22. LTE and 5G PUSCH Throughput Time Series
Figure 23. LTE and 5G PUSCH RBS and Uplink MCS Time Series
Figure 24. LTE and 5G PDSCH Throughput and RBs Time Series
Figure 25. 5G PDSCH Throughput and BLER Time Series
Figure 26. LTE and 5G PDSCH and PUSCH Throughput Time Series - UDP
Figure 27. LTE and 5G PDSCH and PUSCH Throughput Time Series - HTTP
Figure 28. 5G mmWave Test Location
Figure 29. 5G mmWave Site
Figure 30. 5G mmWave RSRP and SINR
Figure 31. LTE and 5G mmWave PDSCH Throughput
Figure 32. LTE and 5G mmWave PDSCH Throughput - by carrier
Figure 33. LTE and 5G mmWave PDSCH RBs - by carrier
Figure 34. 5G mmWave MCS Values - by carrier
Figure 35. 5G mmWave RSRP and SINR
Figure 36. LTE and 5G mmWave PUSCH Throughput
Figure 37. LTE and 5G mmWave Uplink Average MCS Values
Figure 38. Coverage Walk Test
Figure 39. Coverage Walk Test Mapped to Rakuten Coverage Map
Figure 40. LTE Downlink Latency Results
Figure 41. 5G Downlink Latency Results
Figure 42. LTE Uplink Latency Results
Figure 43. 5G Uplink Latency Results
Figure 44. XCAL-Solo
Figure 45. Umetrix Data Architecture
Companies Mentioned
- Rakuten
