LAVIN
2018
An exploratory journey inside a neural network in VR
+ Exhibit in SwissNex Gallery on Nov 9th, in San Francisco
+ Selected into ACM SIGGRAPH 2019 ART GALLERY
+ Exhibited in M.A.D.E End of Year Show, at UCSB
credits :
Virtual Reality Environment Worldbuilding & Creative Coding
Weidi Zhang
Intelligent System Design & Creative Coding
Rodger Luo
(Duration 5 - 10 minutes)
Virtual Reality Art Installation
Enter Through the Virtual Reality Headset
This VR project is a conceptual response to Ground Truth in the modern AI age. From a neural network (NN) trained to recognize thousands of objects to a NN can only generate binary outputs, each NN, like human beings, has its own understanding of the real world, even the inputs are the same. LAVIN provides an immersive responsive experience to visually explore one understanding of a NN in which the real world is mapping to 50 daily objects. LAVIN constantly analyzes the real world via a camera and outputs semantic interpretations, which navigate the audience in a virtual world consisting of all the fluid abstract structures that designed and animated based on the photogrammetry of daily objects that the NN can recognize.
这个VR项目是对当今人工智能时代中 真实 的定义的艺术概念性的回答。 神经网络可以通过被训练 来认识成千上万的物体 输出二进制的回答，每个神经网络其实和人类一样，对真实的世界有着自己的理解。LAVIN 通过人工智能神经网络训练把真实的世界归类重组成五十个物件并通过VR技术视觉化的呈现在VR虚拟世界。LAVIN通过一个相机采集周围真实世界的数据，然后利用人工智能技术分析归类，得出一个物件的词语作为结论。每次人工智能分析完周围的数据，佩戴头盔的体验者就会被实时带到虚拟世界里相关物件的旁边。每次分析需要5 - 15秒时间，于是沉浸者通过人工智能持续性的错误理解这个世界来完成一趟虚拟世界的旅行。
VIDEO DOCUMENTATION
*Screen Recording of A visitor's experience
The current AI technique allows a neural network to recognize over 20 thousand different categories of objects. Meanwhile, countless neural networks are trained for different applications that the outputs of each neural network is a unique projection of its own understanding of the real world. Regardless of the complexity of the projection, it shapes the “world value” of the neutral network it belongs. Therefore, an interesting question arises as to what ground truth is in the modern AI age, given the fact that the most complex neural network model cannot inclusively represent the real world. This VR project tries to address this question by providing an immersive responsive experience to evoke people’s awareness of regrading values and beliefs.
The project consists of two major components:
1) a neural network trained to recognized less than a hundred objects constantly analyzes the real world via a web camera and outputs semantic interpretations. The objects chosen to train the neural network were carefully curated that are daily objects but not commonly to be observed. Hence, the audience would sometimes see some absurd results coming out from the neural network. The real-time results are shown on a wall along with a data visualization of all the previous results.
2) a virtual reality environment consisting of all the fluid abstract structures which are created and animated based on the photogrammetry of the objects used to train the neural network. The materials of those objects are generated based on the exact positions of surface points on photographs. Objects are spatially organized in the virtual space based on their semantic relationships.
The semantic interpretations resulting from the neural network navigates the audience in the virtual reality by taking the person to the related objects. It creates an interaction that the viewer’s experience in the VR is shaped by the surrounding environment in real-time.
.jpg)
special thanks to
Professor George Legrady and the Experimental Visualization Lab, UCSB
New media artist Mengyu Chen for his MetaZMQ library which is a runtime metaprogramming interface to ZeroMQ implemented in Unity Engine for high-performance distributed messaging system.