Libraries and Archives Augmenting the World
This is a supplement to my chapter Libraries and Archives Augmenting the World in The Top Technologies Every Librarian Needs to Know, edited by Kenneth J. Varnum and published by LITA in May 2014. My thanks to the ALA for letting me make the chapter available.
Mine is one of two chapters about AR in the book: the other is “Augmented Reality (AR): Practical Applications within Libraries,” by Brigitte Bell and Terry Cottrell. I’ll include some of the AR applications they mention, but our chapters have very little overlap so you’ll want to read both (and the rest of the book!).
There are a lot of links in my chapter, and having them all here will save the time of the reader. I’ll also include some extra links that are useful but I couldn’t fit in, and some videos that show some current augmented reality (AR) technology in action right now. This isn’t a complete list of everything going on in AR, by any means, but for librarians and archivists, at least, it’s an introduction.
Correction
On page 29 I say, "Now take out your smartphone and install and run Layar. 1 Tap the screen to make it scan this page. You’ll see buttons and images floating a little above it. You can hear me say hello, see a picture of Saturn, read what @ala_lita has been tweeting recently, and more."
This won't work on page 29, because it's too text-heavy and not graphically rich enough to be understood by Layar. Instead, use Layar to scan page 34 (which has a Layar Creator screenshot on it).
My apologies for this. Learn from my mistake!
First, though: if you haven’t read Rainbows End (2006) by Vernor Vinge then find a copy right now and read it as soon as you can.
Then, to see what people are talking about right this minute, start with a Twitter list of people and groups working on or interested in augmented reality. And keep your eye on Bruce Sterling (on Tumblr too).
“The future is already here—it’s just not very evenly distributed.” — William Gibson
What is augmented reality?
I like Ronald Azuma’s 1997 definition in A Survey of Augmented Reality: “AR allows the user to see the real world, with virtual objects superimposed upon or composited with the real world. Therefore, AR supplements reality, rather than completely replacing it” (which virtual reality does).
He set out three characteristics that a system must have to be augmented reality:
- Combines real and virtual;
- Interactive in real time;
- Registered in 3-D.
(The third point means accurately placed in a 3D coordinate system: poor registration means virtual objects wiggle or jump around.)
See also:
- Milgram, Paul, and Fumio Kishino. 1994. A Taxonomy of Mixed Reality Visual Displays. IEICE Transactions on Information Systems E77-D (12).
- Mann, Steve. 2012. Augmediated Reality and ‘McVeillance’.
- Mann, Steve. 2013. My ‘Augmediated’ Life: What I’ve Learned from 35 Years of Wearing Computerized Eyewear.
Four methods of augmenting reality
Geolocation
Apps to try on your smartphone. Search for them in your application store. Run them, look for some kind of “geo” mode if you’re not in one to start, then just look around you.
- Layar.
- Junaio (made by Metaio).
- WIkitude.
- Mixare (free and open source (code is on GitHub, but not under active development).
Also Historypin, which is barely AR but archives especially should know about. See their page for libraries, archives and museums.
Providing points of interest (POIs):
- Layar API documentation.
- Wikitude POI documentation.
- Metaio developer documentation for Junaio channels.
- BuildAR for Junaio.
- Avoirdupois (general POI provider) and Laertes for Layar (Twitter-aware POI provider meant for conferences and events), both written by me.
Markers and object recognition
Fiducial markers are objects pre-defined and ready for an AR application to recognize and use as a point of reference. More broadly, any object can be recognized. This Layar video shows augmented and interactive print:
Apps to try on your smartphone:
- Layar (try scanning images they posted on Pinterest).
- Junaio.
- Wikitude.
- Augment.
- Word Lens translates words in real time as they appear in the camera view.
- Daqri’s Anatomy 4D lets the user look at bones, nerves, the circulatory or digestive systems and more, from all angles.
Development:
- Qualcomm’s Vuforia is the leading software development kit (SDK). It integrates with the Unity 3D game engine. Vuforia comes at no charge but is not free software and has a restrictive license.
- OpenCV is the best free computer vision system, but it would take more work. The code examples from Mastering OpenCV with Practical Computer Vision Projects include AR.
- Layar SDK and Layar Creator.
- Metaio SDK.
- Wikitude SDK and Wikitude Studio.
See also Tools for Development Augmented Reality, by Hugo Ortega of Only Augmented.
Glasses
There is a lot of activity in this area. Google Glass isn’t AR but is probably what most people will think of first.
I mention the Vuzix Smart Glasses M100 in the chapter, but Epson’s Moverio BT-200 glasses look more promising.
META.01 glasses from Meta (Steve Mann is chief scientist) will be out this year and look very exciting. Here’s an ad for Meta Pro, which is also coming out this year, but are much more expensive:
The physical environment
Watch out for castAR from Technical Illusions, which has projectors in glasses that display onto a retroreflective surface and turn it into a sort of holodeck. Here Jeri Ellsworth, one of the two inventors, introduces it:
Motion detection systems that could be used:
- Leap Motion
- Thalmic’s Myo
- Kinect
They could be integrated with projection systems such as Microsoft’s IllumiRoom or Bot & Dolly’s Box, which does projection mapping onto moving surfaces:
It’s possible to use simple markers to do AR on screen with the physical environment right now.
- ARToolKit is a free and open system for doing this; there are many variations online.
- SnapDragon AR (commercial, made by colleagues at York University) was used by Geoffrey Alan Rhodes in his talk AR on AR: Occupying Virtual Space.
AR in use
A few examples of augmented reality in use. Also look at ShelvAR, “the Augmented Reality tool for shelf-reading and inventory management in libraries.”
Bavarian State Library
The Bavarian State Library (Bayerische Staatsbibliothek, or BSB) made an iOS app about Ludwig II of Bavaria, Ludwig II — Walking the Footsteps of the Fairytale King. As well, the 3D BSB Browser uses gesture control and eye tracking to give the user an interactive 3D digitized historical book—without requiring any glasses. Klaus Ceynowa’s March 2013 talk Augmented Reality, Location Based Services and More: Innovative User Scenarios for Library Content shows both in action:
SCARLET
Special Collections Using Augmented Reality to Enhance Learning and Teaching, or SCARLET, is a project of several UK universities that is interesting for what it has done and also how it has been organized. The website explains, “SCARLET addresses one of the principal obstacles to the use of Special Collections in teaching and learning—the fact that students must consult rare books, manuscripts and archives within the controlled conditions of library study rooms. The material is isolated from the secondary, supporting materials and the growing mass of related digital assets. This is an alien experience for students familiar with an information-rich, connected wireless world, and is a barrier to their use of Special Collections.”
Museums
- The Museum of London’s Street Museum app highlights historical photos from their collection similar to Historypin.
- The Asian Art Museum in San Francisco had an iOS app made for its China’s Terracotta Warriors exhibit in early 2013: the main feature is that you can see an “animated life-sized 3D terracotta warrior.”
- The British Museum has done several Junaio-based AR projects aimed at children, and Shelley Mannion’s 2012 talk Beyond Cool: Making Mobile Augmented Reality Work for Museum Education gives a good overview. See also her talk at insideAR 2011:
Video searches for “museums augmented reality” (YouTube, Vimeo) turn up interesting demonstrations.
“Sight”
It’s all brought to life in the short movie “Sight” by Eran May-raz and Daniel Lazo. “Sight” is wonderfully done—and very disturbing.
More information
On Twitter I maintain a list of people and groups working on or interested in augmented reality (suggestions are welcome).
Web sites
A few to begin. All the companies posts news on their web sites too, of course.
- Augmented Reality Blog
- Augmented Reality News
- ARMedia Augmented Reality Blog
- Ontario Augmented Reality Network
- Uberveillance
Podcasts
Conferences
Conference sites may include videos of presentations, links to papers, etc.
- IEEE International Symposium on Mixed and Augmented Reality (ISMAR) is the major academic conference.
- IEEE Society on the Social Implications of Technology’s International Symposium on Technology and Society (ISTAS) is another. The ISTAS 2013 site has details about the latest.
- Augmented World Expo is “the world’s largest augmented reality trade show” (US).
- Augmented Reality Summit (UK).
- Metaio’s insideAR (Germany).
Standards
- The grassroots ARStandards group has notes and recordings of their meetings online; they also have mailing lists.
- Augmented Reality Markup Language (ARML) from the Open Geospatial Consortium (not under active development right now).
- See also the W3C Augmented Reality Community Group.
Works cited
- Azuma, Ronald T. 1997. A Survey of Augmented Reality. Presence: Teleoperators and Virtual Environments 6 (4): 355–385.
- Ceynowa, Klaus. 2013. Augmented Reality, Location Based Services and More: Innovative User Scenarios for Library Content.
- Hahn, Jim. 2012. Mobile Augmented Reality Applications for Library Services. New Library World 113 (9/10): 429–438. DOI: 10.1108/03074801211273902.
- Kipper, Gregory, and Joseph Rampolla. 2012. Augmented Reality: An Emerging Technologies Guide to AR. Waltham, MA: Syngress.
- Mann, Steve. 2012. Augmediated Reality and ‘McVeillance’. Steve Mann’s Blog.
- Mann, Steve. 2013. My ‘Augmediated’ Life: What I’ve Learned from 35 Years of Wearing Computerized Eyewear. IEEE Spectrum. March 1.
- Mannion, Shelley. 2012. Beyond Cool: Making Mobile Augmented Reality Work for Museum Education
- Milgram, Paul, and Fumio Kishino. 1994. A Taxonomy of Mixed Reality Visual Displays. IEICE Transactions on Information Systems E77-D (12).
- Rhodes, Geoffrey Alan. 2012. AR on AR: Occupying Virtual Space.
- Vinge, Vernor. 2006. Rainbows End. New York: Tor.