Rakuten Mobile and Rakuten Vissel Kobe Successfully Demonstrate a New Spectator Experience using 5G and VPS Technology

Tokyo, November 10, 2021 – Rakuten Mobile, Inc. and Rakuten Vissel Kobe, Inc. today announced the success of a trial of a new spectator experience utilizing Rakuten Mobile’s 5G mmWave network and VPS (Visual Positioning System) technology, held at Noevir Stadium Kobe on November 3, 2021^*1. The experience enabled spectators to view player and match statistics through an AR (augmented reality) overlay, while also integrating AR advertising and e-commerce functionality.

Pictures from the trial at Noevir Stadium Kobe

The trial demonstrated the viability of using VPS technology in a large area, such as a stadium, to pinpoint the precise location of a given device and enable AR content^*2 to be displayed anywhere within the area. VPS technology utilizes information from the camera of a user’s smartphone or smart glasses to identify the device’s orientation and precise location.

The high precision of VPS is achieved by sending images captured by the device’s camera at frequent intervals to the server, presenting a perfect use case for the high speed, high capacity and low latency nature of 5G, which was used for parts of the transmissions between the smartphone and the server during the trial.

The VPS technology utilized in this PoC was provided by Rakuten Mobile Partner Program^*3 member Immersal Ltd^*4, a technology company delivering spatial mapping and visual positioning tools based in Helsinki, Finland.

About this trial
　1. AR display of player and match statistics through VPS
　2. Integration of AR advertising and e-commerce functionality through VPS

1. AR display of player and match statistics through VPS
In this trial, VPS technology was utilized to display match information and official J.League player statistics (JSTATS)^*5 through AR on a dedicated app^*6 when pointing their smartphone towards the pitch, or by looking towards the pitch with a pair of smart glasses.

During the previous trial^*7 held in November 2020, spectators needed to scan an AR marker prior to using the app in order to determine their location. In this trial, VPS technology was utilized to pinpoint the precise locations of spectator devices from seats around the stadium without the need for an AR marker, allowing for an even more seamless AR spectator experience.

This trial also included the use of smart glasses in addition to smartphones, realizing an even more immersive and dynamic spectator experience.

Mockup of the AR display

2. Integration of AR advertising and e-commerce functionality through VPS
Using VPS technology to pinpoint each device’s precise location from seats around the stadium, this trial demonstrated the viability of displaying AR advertising that matches the structure of the stadium. Rakuten is currently considering the display of virtual advertising that combines signs and real advertising; the display of advertising over areas that would not otherwise be possible to install physically; as well as advertisements that change in response to the action on the field during a match and other measures that could further expand potential advertising opportunities.

This trial also demonstrated a new shopping experience through the use of Rakuten Ichiba e-commerce functionality. Spectators using the app could tap on any players running on the pitch shown through the screen of their smartphone, or select a specific player using their smart glasses controller, and intuitively shop for items such as the uniform worn by that player.

Rakuten will continue exploring O2O (Online to Offline) and OMO (Online Merges with Offline) possibilities such as the integration of the Rakuten ID system for an even more personalized advertising and shopping experience, and further integration into the Rakuten Ecosystem.

Example of the integration of AR advertisements and e-commerce functionality

Rakuten is also planning to provide the 3D point clouds used in the VPS system to members of the Rakuten Mobile Partner Program, aiming for wider adoption as a stadium solution, starting with Noevir Stadium Kobe.

3D point clouds and visual map of Noevir Stadium Kobe

Rakuten Mobile and Rakuten Vissel Kobe will continue exploring new ways for stadium visitors to enjoy the fan experience, including the creation of new stadium experiences that make use of the Rakuten Ecosystem and integration of services such as online shopping.

*1 This trial took place during the 2021 Meiji Yasuda J1 League 34^th Sec match between Vissel Kobe and Vegalta Sendai (Nov 3, kick-off 16:00).

*2 Design unit ‘Rakuten Design Lab’ provided the graphic design, motion graphics, and UI/UX art direction for the AR experience.

*3 Details of Rakuten Mobile’s collaborative program, the ‘Rakuten Mobile Partner Program’ can be found on the following page:
https://corp.mobile.rakuten.co.jp/innovation/partner/project/(Japanese)

*4 For more information about Immersal Ltd., please visit the following site: https://immersal.com/

*5 JSTATS refers to the various types of match data recognized by the J.League. JSTATS is a registered trademark of the Japan Professional Football League.

*6 The J.League club logos and emblems displayed in AR are provided by the Japan Professional Football League.

*7 Rakuten Mobile and Rakuten Vissel Kobe Collaborate on Trials of New 5G Match Viewing Experience
https://corp.mobile.rakuten.co.jp/english/news/press/2020/1203_01/