"Trained on over 9,000 Chinese characters, the AI system Cangjie by Weidi Zhang and Donghao Ren is creating a new language to converse with the spectator, creating an immersive data visualization spectacle in a multi modal installation. Cangjie's Poetry tackles the issue of language creation between human and machine in a sensitive, poetic, and fragile way. Cangjie´s Poetry is an exceptional and far-reaching work bringing data visualization and the use of AI in co-creation between being and apparatus."

- Prix Ars Electronica Honorary Mention

 

Cangjie's Poetry is the second episode of  Cangjie   *Compared to the last chapter, this new episode is beyond a mere visual celebration, but more importantly, a semantic human-machine experience that evokes feelings.

 
Cangjie's Poetry 

Humans and machines are in constant conversation. Intelligent machines today are not only observers of the world, but they also make creative decisions. If A.I imitates human beings to generate a symbolic system and actively communicate with us based on their own understandings of the universe, to what degrees their messages and meanings will recontextualize our coexistence? 

VIDEO DOCUMENTATION   [ 2 minutes ] 

Cangjie’s Poetry is an intelligent multimodal system designed as a conceptual response to the future semantic human-machine reality. Inspired by an ancient Chinese legendary historian, Cangjie (c.2650 BCE), who invented Chinese characters based on the characteristics of everything on the earth, we trained a neural network that we call Cangjie, to learn the constructions and principles of over 9000 Chinese characters. After successfully training, Cangjie can interpret images through the lens of Chinese characters and produce new symbols constructed by Chinese strokes. Meanwhile, we implement a pre-trained model to simultaneously generate localized descriptive sentences of images in natural language to create meanings for this symbolic system.

In the art installation, the Cangjie system captures surroundings using a camera and transforms the real world streaming into a cluster of ever-changing new symbols in real-time. Those novel symbols made of Chinese strokes tangled with the imagery captured by the camera in the installation are visualized algorithmically as an abstract pixelated landscape. The landscape dynamically moves, evolves, and writes poetry based on the image data of the live streaming captured by the camera. We project the visualization of the semantic landscape on the wall in the exhibition space as the first projection. In the meantime, Cangjie also generates descriptive sentences of surroundings based on its interpretation. Those sentences are designed as flowing poetry written in ink, assembled with real-time captured imagery fragments as the second projection in the space.

Canglie's Poetry transforms the machine's interpretation of surroundings into an interactive semantic experience. It constructs conceptual and futuristic communication between humans and machines. This work is not only a conceptual response to the tension and fragility in the coexistence of humans and machines but also a conceptual expression of a future language that reflects on ancient truths, a way to evoke enchantment in this era of artificial intelligence. Like the legendary historian Cangjie did nearly 5000 years ago, the Cangjie system continuously writes poetry with humans collaboratively as long as the real world exists.

cjp__101085.jpg

CREDIT

-

Art Direction

Visualization Design

Interaction and Sound Design

Intelligent System Design

Weidi Zhang

/

Intelligent System Design

Intelligent System implementation

Donghao Ren

EXHIBITIONS / AWARDS / CONFERENCES
-
Cangjie 2.0: Cangjie's Poetry
Red Dot Award 2022
Prix Ars Electronica 2022 [ Honorary Mention ]
SIGGRAPH Art Gallery 2021 [ Best of Show Award ]

JOYNT Art+Tech Festival 
 
Cangjie 1.0 
Japan Media Arts Festival 2020 / Juried Selection
Lumen Prize 2020 / shortlist with Honorable mention 
SIGGRAPH Asia Art Gallery 2020
IEEE VIS Art Program 2020
ChinaVis Art Gallery Invited Show 2020

PUBLICATION-

-

Weidi Zhang, Donghao Ren, and George Legrady. 2021. Cangjie's Poetry: An Interactive Art Experience of a Semantic Human-Machine Reality. Proc. ACM Comput. Graph. Interact. Tech. 4, 2, Article 19 (July 2021), 9 pages. DOI:https://doi.org/10.1145/3465619

INTERVIEW 

-

'CANGJIE'S POETRY - BEYOND A MERE VISUAL CELEBRATION'

SIGGRAPH 2021, Invited Interview

SPECIAL THANKS

-

Prof. George Legrady & Experimental Visualization Lab (UCSB)

Prof. Marko Peljhan, Prof. Curtis Roads

Media Arts and Technology Program UC Santa Barbara

 
_AFINAL_02550.jpg
cangjie2_logo.jpg
out.jpeg