Experiential Technologies

Technologies

Experiential software and hardware are tools that allows the user to create new objects. They are analogous to the use of word processors for the creation of text documents.  Here is a list of reviewed experiential hardware and software:

Virtual Reality

Microsoft Hololens:

A useful twist on the Virtual Reality platforms.  The Hololens is similar to Google Glass in some ways; it is a heads-up screen/goggle set that allows the user to see both a computer screen and everything that the person would see if she or he was not using the goggles.  However, it differs significantly in some powerful ways.  The goggles are a full computer system running Windows 10 and create holograms or digital objects with which the viewer can interact and can walk around.  The systems is smart enough for objects to place themselves in appropriate places in a room – Select a digital calendar to view and it was hang itself as a hologram on the wall in which you are sitting via a concept call "Mixed Reality" – Play a view game and a hologram of a tool (etc.) can appear in the player’s hand will move with the player as she or moves around the room.  Games can even incorporate structures in the physical room and integrate them into the game such a holographic creatures hiding behind furniture the physical. The system will be wonderful for gamers, but it will completely change how people interact, teach, and research in VR environments as well as a person’s physical space.

https://www.youtube.com/watch?v=C3rNIxMlKmI

Possible instructional examples:

CAD - Truly be able to look at and design your new computer assisted design (CAD) product from every angle

Biology - Identify plants and animals based on the perspective from which they were observed

Allows students to interact with models of human bodies without the need for cadavers. Hololens allows students to view models with working internal body systems. 

Examples: https://www.youtube.com/watch?v=SKpKlh1en0 

& https://www.youtube.com/watch?v=ykpQjO_0tig

Chemistry - The ability to more easily view the chirality of molecules

Geology - The ability to walk around topographic maps, strata models, volcanoes/lava chambers or look at seismic data set maps in a 3d perspective

Physics - Forensics- reconstruct crime scenes and view them from multiple perspectives so that some evidence is not missed and other items can be ruled out as evidence; Illustrating the process of a solar eclipse from any perspective

Story Telling - Utilizing pre-made holographic avatars or creatures to deliver stories

Gaming -  Immersive virtual gaming environments that utilize physical space

Engineering / Product Design - Moving in and out or through the layers of an object, say a car, and and see how a real person would interact with the object

Analysis -  Greater use 3D statistics, multi-dimensional space, and multivariate statistical analysis because the viewer can walk around the data visualization

VR Building: Molecules & Animation

Molecular visualizers are nothing new (examples: Rasmol, PyMol, VDA, VMD, Chimera) but there is a growing effort to create virtual reality molecular visualizers. This is actually something students have brought up at the Science circulation desk. This link is a video of Oliver Kreylos (Associate Researcher, UC Davis) demonstrating his teams Nanotech Construction Kit software: https://www.youtube.com/watch?v=2MN3FHrQUa4
Information on the software can be found at: http://idav.ucdavis.edu/~okreylos/ResDev/NanoTech/

Researchers from Lund University (Sweden) have created similar software (Molecular Rift). The key difference being that this system is entirely motion controlled using an Occulus Rift headset paired with a Kinect 2.0: http://pubs.acs.org/doi/full/10.1021/acs.jcim.5b00544#_i12

On a completely different track, here is a video of a Disney artist using the HTC Vive to animate in VR: https://www.youtube.com/watch?v=hEK-J3ZgCZA

Data Visualization 

R and RStudio -  Is an open-source statistics program widely used in the academic community. It is quite flexible and can drive even very complex analyses. With the addition of other graphics modules, it is proving itself to be one of the best, if not the best, data visualization tool(s).     

            Mathematica 

Wolfram-Alpha - Wolfram|Alpha (WA) is a "computational knowledge engine". The web interface is laid out like a search engine, but instead of pointing the user to other resources for answers, WA returns factual or computational information, including customizable and downloadable graphics. Complex data can also be uploaded or inputted in the web interface. In the last couple of years WA has developed algorithms to interpret natural language questions and expanded the knowledge base to include humanities and social sciences. WA also produces a number of reference apps, a mobile app, the original stand-alone Mathematica for Windows, Mac and Linux machines, a system modeler, a financial platform and a data science tool among others.

Tableau - because of ease of use & broad use cases (maps, graphs), this is becoming a popular tool.

Game Production

Unity - Unity is a proprietary multi-platform game development engine with an enormous user community. Unity Personal can be downloaded and used for free, with restrictions elucidated in the Software License Agreement, or licensed as Unity Pro (Educational pricing available). The major advantage to Unity Pro is the collaboration and cloud functionality. Unity can also be used for audio editing, remixing and recording and for limited format video editing. Check out Made with Unity to see what other people are making.

Quest - used by Tara Fickle (ENG) in her gaming course

SfM / Photo-Alignment

Photo-alignment / Structure-from-motion (SfM) is a technique that uses pictures taken from different angles to create a 3D image of an object or landscape.  It is similar to using a LIDAR laser or Kinect scanner/camera to create 3d images.  There are two major benefits to using SfM. This technique can used for creating images of small coffee-cup size objects up to scale models of mountains without the need for special equipment.  Additionally, almost any camera can be used to capture the data and even repositories of images (Flickr etc.) from multiple cameras or people can be pooled to create the 3D image.  The down side to this technique is that is does require lots of computer computations so fast computers with lots of RAM are preferred.  Less powerful computers can be used but the 3d task could tie up a computer for hours or days in these cases. Photoscan by Agisoft is one of the commercial grade software packages that is available is accessible at the UO Science Library.  Additionally, 123Dcatch by Autodesk is a free online service.  It is less powerful that PhotoScan, but can be used by anyone with a internet connection.  Finally, VSfM is an open source package that have good reviews. SfM is an important tool because it allows people to recreate and print existing objects and to create 3D viewable landscapes for GIS analysis.  Objects created with SfM can even be incorporated into systems like ArcScene / 3D Analyst or GoogleEarth.