The current advances in the use of Virtual Reality technology in book publishing

With the advent of digital publishing, the publishing industry has witnessed a transformation in terms of its production, operation, and industrial structure. Publications can be categorized by their product forms into eight main categories: e-books, electronic journals, electronic newspapers, online games, digital animation, electronic music, online advertising, and databases.

Virtual Reality (VR) is a digital technology redefining how we perceive information and exploring users' experiences through immersive, interactive, and imaginative content^[1]. With the consumerization and popularity of VR technology, the publishing industry has also become vigilant. As the publishing industry moves into the meta-universe era, VR publishing is a significant breakthrough.

As significant publishing products, VR books provide a solution for improving reading comprehension and lifelong reading skills. The contents of e-books have been developed in VR with the spread of VR, such as HTC Vivepaper^[2]. Therefore, the aim of this paper is to summarize the current advances in VR book publishing to support the research and decision-making of researchers and publishing workers.

VR books are rare in practice. Most of them are limited to children's books related to themes of literature, cultural tourism, education, and books for special readers.

VR literary book design involves analyzing 3D art, VR scenes, and character design, and exploring immersive relationships between characters, scenes, animations, sounds, and text^[3]. VR books in literature are very popular and there are many representative works: (1) The StoryVR App is a VR reading application with science fiction content, which makes literary texts in VR more appealing by providing an immersive and interactive experience while listening to stories. It has obvious positive effects on improving readers' reading immersion, attention, and emotional engagement^[4]; (2) Yuelu Book Publishing Group in China published VR books of the four great classic Chinese novels in 2017 which are named Romance of the Three Kingdoms, Outlaws of the Marsh, A Dream of Red Mansions, and Journey to the West. It uses expert dubbing from multiple angles such as literature, history, and art to interpret the content of famous works. It allows readers to scan the 2D code in the book, and then visit the famous VR cultural scenes based on smartphones; (3) Literacy Land is designed by GlassVR in London to support reading literacy and takes students on a journey through some of the most famous literature. It provides four fantastic reading literacy zones for K12 students to explore, such as Classic Books, Early Years, Poets Corner and Shakespeare. It enables students to develop from reading fairy tales to the in-depth study of classical books and analysis of Shakespeare's words^[5].

VR educational books contain content related to education. The applications are typically designed with a specific training or educational objective in mind. There are two main groups of current application: (1) adult vocational training; and (2) high school and university education^[6]. These themes are commonly needle abstraction learning content that are difficult to understand or boring learning content that lacks reading interest. A useful or interesting VR book design could help to resolve these problems. There are several representative cases: (1) EU-funded MiReBooks (Mixed Reality Books) project, for higher education across European engineering education, which develops a series of AR/VR-enabled mining manuals as a new digital standard. Students are kept away from dangerous situations in mining education^[7]; (2) In education for soil and water conservation, Tsai et al. designed a VR book system prototype that allows students to view a picture book through a storyline. It can be experienced based on a smartphone APP with VR helmets^[8]; (3) Immersive electronic books (IE Books) for surgical training enable surgeons to explore previous surgical procedures in a 3D environment in the medical field^[9]; (4) In China, Hengxin Shambala Culture Co., Ltd. in conjunction with National Geographic China since 2020, has embarked on the development of a VR interactive book project for the youth version of the Geography Encyclopedia. VR technology is integrated into children's science books to achieve a compelling combination of static text, picture content, and VR panoramic travel images. This solves the dullness and mutual weight of science books, increasing the fun of reading, and enhancing children's curiosity and desire for exploration of the world^[10]; (5) Utopic Studio aims to increase children's motivation by developing a VR library platform with interactive and immersive VR games. Children's reading is transformed into an exciting learning experience. The company team assesses their reading skills using non-traditional and more sociocultural research approaches, which shows that VR games are an innovative way to improve children's reading abilities^[11].

VR cultural tourism books provide readers' with a trip in an immersive environment whilst reading. They are designed to let readers embody a historical event with a stronger sense of presence, which enhances and creates memorable travel experiences for readers. This may lead readers to seek more information about the event in which they participated after the experience, and could also increase readers' engagement in reading this type of book in the future^[12]. Guttentag pointed out that VR could support cultural tourism in six aspects: heritage preservation, education, accessibility, marketing, entertainment, planning, and management^[13]. VirtuaLand is one of the representative applications that deliver a VR interactive platform that allows readers to explore assets of interest as well as experience life in another era. It enhances readers' experiences by interacting with the living conditions of that period^[14]. Kiss & Fly as an example, produced by Studio 96 publishing house in New York City, brand-new company that combines print media with digital technology to provide a virtual vacation. It offers virtual holidays to nine different locations: France, Brazil, Iceland, Mexico, Italy, Jordan, Greece, India, and Israel. This book has moving images, hyperlinks to interactive forums or social media channels, codes to download playlists tailored for the reader's reading experience, and even links to tickets for exclusive and online events^[15].

A VR book for readers with print disabilities is a publishing method to enable reading with virtual content, to ensure perceptibility, comprehensibility, and operability of information, and to reduce the information gap, which exists between print disabilities and normal readers^[16]. For example, Weir et al. designed and developed a VR book for visually impaired readers that supports voice control of font size, change in the number of lines, font, light source, brightness, font color, and many other features^[17]. Daly studied the auxiliary effect of situational cognition on the reading comprehension level of readers with weak reading abilities using science fiction texts as the research object to test readers aged 8−18, concluding that VR has a positive effect on improving comprehension ability and reading interest^[18]. Kobi360 as an example, based on the Kobi mobile app, is an immersive VR game for children with reading difficulties like dyslexia and ADHD that turns such difficulties into an entertaining, child-friendly experience. It allows dyslexic children to train their balance, coordination, and reaction time and also helps their brains to distinguish letters faster. It is a free application that allows readers to scan the book content and import them to Kobi360 for use^[19].

An annotation system is commonly used to highlight challenging text elements in VR book design. The design of VR books needs to take into account both communication effect and economic investment so that not all VR books replace VR images for graphic content. In the case of a limited budget, the relevant content is annotated in a graphic and textual manner. The 3D stereoscopic image of the VR book is used when the knowledge elements of the books are difficult or more abstract (like mechanical motion and atomic structure). As a means of explaining complex or dangerous contents (such as atomic reactions, fire tests, and electric shock experiments), VR scenes are utilized to demonstrate these points^[20]. For instance, the VR enabled Old Tang Book was designed with an annotation system that provides the cultural elements from the museum, libraries, and cultural travel websites to illustrate the text elements (such as a person, organization, location, artifact and so on)^[21].

Multi-modal text resources are widely used for VR book design. It refers to the form in which content is read in a way that is transmitted as more than one resource but also includes handwriting, pictures, videos, animations, and audio, which is more effective in reading comprehension and communication^[22]. The resources used in VR book design include^[23]: (1) 2D static footage, such as photos, paintings, sketches, texts, ordinances, and so on; (2) 2D dynamic content, such as videos and animations; (3) 3D content, such as 3D static models (physical objects, environment), 3D dynamic models (animations, 3D videos); (4) sounds, such as background music, spatial music (3D sound depends on the reader' position and action), and interactive sounds. The representative case is Vivepaper, a reading magazine developed by HTC VIVE, which enables AR/VR functionality through the Vive front-facing camera, merging AR and VR technologies to give readers the ability to interact with real-world content in a virtual world, ensuring a virtual experience with an enhanced sense of reality. The magazine's content is designed to be light on interaction and heavy on content, using a combination of panoramic pictures, panoramic video, stereo audio, and 3D models to provide readers with immersive, engrossing, and easy-to-use reading content^[24].

All VR books transpose the 2D page design into a 3D space replicating the haptic affordance of paper books. Ideally, it may have interactive functions, such as annotation, link, navigation, virtual background, page-turning, etc. As early as 1993, Verlinden had begun to study the design, development, application, and evaluation of VR books, exploring the integration of media information such as text, pictures, audio, video, and animation into the 3D environment, supporting the interaction between authors/users and media information, and symbolically proposing the concept and functional elements of VR books^[25]. Card et al. proposed a 3D interactive visualization of a codex book (3Book) prototype in 2004. It is a system that ports the functions and performance of real books to a 3D environment and explores the VR book design and development process^[26]. Based on preliminary research, Card's group studied how to use 3D image technology to add realistic annotations to virtual books based on the 3Book prototype^[27]. Subsequently, they designed the page-turning function of 3Book in 2006, balancing critical factors such as visual realism, readability, interactivity, and the scalability of readers. The book simulated real book page-turning function and realized the seamless conversion of the book opening and closing^[28].

With regard to the presence of senses and readers' preference, multi-sensory immersive VR book content has a clear advantage over a typical VR system (which only offers vision and audio). In a location-matching task that requires both perceptual and cognitive work in VR, Jung et al. found the effects of multi-sensory stimuli, namely visuals, audio, two types of tactile (floor vibration and wind), and smell were more helpful in finishing the tasks^[29]. The Tate Sensorium project explored the scope for enhancing the experience of visual art through the addition of sound, taste, touch, and smell for museum-goers. It is shown that the senses are more vivid and more likely to retain memories of what they see^[30]. Multi-sensory immersive design is also helpful for readers with reading difficulties. Readers develop social, communicative, intellectual, and cultural skills through reading. While the reader's mental imagery is generally constructed primarily through visual experiences, the mental images of the visually impaired are a product of touch, taste, smell, and sound. For visually impaired readers a multi-sensory interactive picture book can combine multi-sensory interaction (touch, sound, and smell) to create a new reading experience for the visually impaired^[31].

At present, the development of VR hardware equipment is still in the exploratory stage. Affected by VR technology and hardware, VR consumption has not yet been popularized, and the VR readers' market is facing a long exploration period. It is mainly reflected in the fact that the developer should pursue the authenticity of the virtual world, which has high technical requirements for environmental levels such as resolution, followed by the readers requirements for network transmission technology and battery life, which requires the dual support of technology and capital^[32].

With the metaverse industry coming into the limelight, the VR industry is currently in the pre-explosion stage. There have been some achievements in the development of VR hardware around the world. However, even the most sophisticated VR hardware and content at present still have the fatal flaw of too much visual vertigo. Many VR users mentioned discomfort, nausea, and even vomiting after using VR products^[33,34]. This has become one of the biggest stumbling blocks in the development of VR, and solving the problem of vertigo has become an urgent requirement.

Current VR equipment has obvious flaws in its immersion implementation. This is evident in both the Oculus Rift and HTC Vive. This is because the reader loses control of the 'virtual hand' when the reader steps out of sensor range. This inevitably interrupts the reader's experience and forces them to readjust their position in the real world to enter the virtual world. It is therefore a difficult challenge for content developers to adapt to the limitations of the device and still create a quality immersive experience.

The VR terminal equipment does not match the consumption conditions of readers

The VR terminal equipment required to read VR books in an immersive and interactive manner is not popular. The common disadvantages of VR equipment are expensive, non-portability, complex operation, and limited effect^[35]. Mobile VR is cheaper, but its performance is weak and cannot meet the needs of VR high's definition. VR performance on PC is better, but the cost is higher. A more reader-friendly equipment price is needed for the popularization of VR books.

The aforementioned situation also leads to a vicious cycle of VR equipment consumption. Consumers are being misled by the low-end VR products currently available on the market. Consumers who are curious about VR are interested in low-end cheap products, and even if it is not a pleasant experience, throwing it away is not a problem. There are several consumers, in this case, who think that VR products are disposable and thuerefore won't buy high-end products with more features and higher-tech content.

There are multiple channels for selling VR books, but the expensive price limits the market. This aggravates the digital gaps between VR reading experience and content acquisition among different reader groups, particularly in rural and urban areas. In addition, the same VR book has limited reading time, resulting in insufficient utilization and a waste of book resources.

Readers' VR literacy is insufficient

Readers lack experience with VR equipment because of its low popularity. This limits the readers ability to play and understand VR, resulting in a sense of resistance to using VR products. This problem requires the upgrading and optimization of VR equipment and the reduction of pricing.

From a software design perspective, there is no globally accepted standard for VR/AR interaction design, so with each upcoming VR/AR application that emerges, readers must invest time and effort to learn how to use it. Developers need to create simple and intuitive UX and interactions in VR/AR for reader experience and long-term usage. One of the main reasons why immersive experiences are confusing is that readers lack the experience and habit of interacting with 3D media.

Low return on capital investment

Compared with traditional books, the publication cost of VR books is high, which is reflected in the expensive equipment required and the complexity of content development. The high pricing of VR books weakens readers' enthusiasm for buying them, resulting in insufficient investment in VR books and stagnation of VR book development.

Most current VR companies have chosen to start in the hardware space, neglecting content production to cut costs. Readers buying a VR headset can only watch a few videos or experience simple games, which cannot meet their basic needs. In addition, current VR product content is dominated by adult content and games, which will raise VR products' prices if vendors develop more VR applications. Consequently, VR companies will have less incentive to develop VR applications. This is because it will be more difficult to sell VR products. This results in a vicious circle.

Currently, the business model for VR book publishing has not yet been formed. Both the books and the content of the related books are scarce. It is critical for high-quality VR books to adhere to improved content design and development to attract more readers. Most of the VR books are related to kids' education and high school education, which has problems of homogenization and low content quality. The design of VR books should probe deep into the breadth and depth of content and be accessible to a variety of reader groups.