< Cangjie 仓颉 >

​

Humans and machines are in constant conversations. Humans start the dialogue by using programming languages that will be compiled to binary digits that machines can interpret. However, Intelligent machines today are not only observers of the world, but they also make their own decisions. If A.I imitates human beings to create a symbolic system to communicate based on their own understandings of the universe and start to actively interact with us, how will this recontextualize and redefine our coexistence in this intertwined reality?

​

This VR project provides an immersive exploration in semantic human-machine reality generated by an intelligent system in real-time through perceiving the real-world via a camera [located in the exhibition space]. Inspired by Cangjie, an ancient Chinese legendary historian (c.2650 BCE), invented Chinese characters based on the characteristics of everything on the earth, we trained a neural network that we call Cangjie, to learn the constructions and principles of all the Chinese characters. It perceives the surroundings and transforms it into a collage of unique symbols made of Chinese strokes. The symbols produced through the lens of Cangjie, tangled with the imagery captured by the camera are visualized algorithmically as abstract pixelated semiotics, continuously evolving and compositing an ever-changing poetic virtual reality. Cangjie is not only a conceptual response to the tension and fragility in the coexistence of humans and machines but also an artistic imagination of our future language in this artificial intelligent era.

​

​

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

/

​

​

This project consists of two major components:

​

/  Machine Learning & Intelligent System Design​

An intelligent system is built by training a neural network to learn constructions of all the Chinese characters. The real-world image captured by Cangjie will get passed through the convolutional Variational Autoencoder (VAE)  model’s encoding-decoding step to transforms real-world images into unique symbols constructed by Chinese strokes.

 

+ Experimental Visualization & Virtual Reality World Building ​

This virtual environment is created with the unique symbols Cangjie generated in real-time, the processed real-world imagery, and the intimate sounds of human whispering. Cangjie is observing and analyzing the surroundings via a camera, which leads to the transformation of those symbols and alters the appearance of the virtual world. The visualization turns 2-d machine imagery into a 3d virtual world by using the image-processing, shader programming, and world-building strategies.

​

/

​

User Interaction​

The duration of this Virtual Reality experience is 5 - 10 minutes. Each audience could participate in this immersive audio-visual experience by wearing a VR headset and using a controller to navigate the space. During the virtual experience, the audience could fly to different places with diverse sounds and visuals, interacting with the intelligent system [Cangjie]. Cangjie will look at the surroundings through its eye [ camera ], and generate the virtual environments in real-time. The audience will wander in the space, be curious about the meaning of the textures, symbols, revolved with the real-world imagery, engage in the interaction, and rethink the issue regarding the future conversation between human beings and intelligent machines.

​

​

​

​

​

​

​

​

+ Cangjie is generating the environment [ screenshot ] 

​

​