Microsoft engineer develops software with global implications
BEIJING - A software engineer in Beijing has revealed a breakthrough in technology that could have a profound effect on museums globally and which puts Beijing firmly on the map of cutting edge development.
Xu Yingqing, 50, a lead researcher with Microsoft Research Asia in Beijing, has created a detailed, interactive and multi-layered digital rendition of the Song Dynasty ink-on-silk painting "Life Along the Bian River at the Pure Brightness Festival".
The image of the priceless work of art created by Zeduan Zhang and rarely shown in public can now be seen and studied in unique detail in the Hall of Martial Valor at Beijing Palace Museum, also known as the Forbidden City.
"It provides a virtual three-dimensional walkthrough of the painting with sound," he said. "The level of detail for a work of art like this has never been created before. The technique can also be applied to other paintings and even vases and the Terracotta Warriors. It will enable artists and art historians to study works at a level never before accomplished."
The original masterpiece - one of China's top 10 most famous paintings - measures 528.7 centimeters in length by 24.8 cms in height. It created a special challenge to the software developer, who has been granted 16 patents in the United States with more pending, because in common with many Chinese works of art there is not a single fixed focal point of interest but several.
The year-long work was carried out as part of Microsoft's eHeritage project in conjunction with Peking University and the Palace Museum free of charge. It was a pilot project with a view to rolling out the technology to other museums, initially in the Asia Pacific region and then in Europe and the Americas. No cost analysis has been applied to it and no commercial decisions have yet been made about its future.
The scroll depicts more than 800 people engaged in activities ranging from playing stringed instruments to paying taxes, illustrating the various strata of the Northern Song Dynasty (960 to 1127).
Microsoft Research Asia established the eHeritage project to facilitate collaborative research with academia in the Asia-Pacific region on applying advanced technology to the digitization, preservation, and dissemination of cultural heritage. eHeritage has funded more than 10 research proposals over the past several years.
"Building on the existing technologies provided by Microsoft for eHeritage, today we are in a position to both showcase and preserve the glory of the world's heritage for future generations," said Shirley Liu, communications and marketing public relations manager for Microsoft Research Asia.
Xu, who is also co-director of the Microsoft Digital Cartoon and Animation Laboratory of the Beijing Film Academy, set himself the challenge of designing and implementing a platform that would guide visitors to learn about the ancient culture depicted in the painting.
The high-resolution image of 1.4 gigapixels (its closest rival commercially is 400 megapixels) is annotated with 700 dialogues from 51 groups of people depicted by the artist as well as wildlife sounds and an overlay of music. The voices and sounds are attached to objects in the painting and become louder as the viewer pans and zooms in on the different images on the 63-inch screen.
The sound scripts were compiled by researchers from Beijing Palace Museum who had majored in the history and ancient culture of the painting. Voices were recorded in Mandarin with the Kaifeng accent from Hunan to reflect the original setting.
"The system is the first of its kind to provide a 3D modeling of moving focus, a famous Chinese painting technique that is different from the fixed viewpoints in Western paintings," said Liu.
"When a user navigates on the multi-touch screen, the user interface estimates the 3D position of the user's viewpoint from his or her actions, consequently activating the embedded stories to synthesize a realistic stereo audio field according to the viewer's orientation and distance from the sound sources in the hidden space. All the sounds play an important role in guiding the viewer to visualize the complex content of the painting."