Virtual worlds increasingly augment the economic and social life of physical world communities. The sharpness of many virtual and physical world distinctions will be eroded going forward. In both spaces, issues of identity, trust and reputation, social roles, rules, and interaction remain at the forefront.
Issues and Technologies
Discussion of the Metaverse usually begins with massively multi-user virtual worlds (VWs), a fast-growing space that is already mixing physical and virtual social, economic, and to a limited extent, political systems via both asynchronous single-user and realtime multi-user modes. Of all our scenarios, Metaverse Roadmap participants talked most about virtual worlds. At the same time, VWs evoked the greatest uncertainty and disagreement.
A key component of the VW scenario is one’s avatar (or in multiplayer games, character), the user’s personification in the VW. As in the physical world, capabilities accessible in digital space are contingent on the limitations of the avatar. But in comparison to one’s physical persona, growth in the social, economic, and functional capabilities of one’s avatar can be far more rapid, and learning experiences can be greatly accelerated. By contrast with the general 3D web, MVR participants expected only a limited non-entertainment adoption (and by inference, social utility and intelligence) of avatars and VWs over the near-term, ten-year roadmap horizon.
MVR Survey Question 17. In 2016, what percentage of internet users in more developed countries (MDCs) will use an interactive 3D avatar at least once a week for any purpose other than games and entertainment, including socializing, communication, creativity, education, barter, commerce, exercise, etc?
Summit Survey: 50% of users
Website Survey: 52% of users
Electronic virtual worlds (first text based, later graphical) have existed since our first personal computers (e.g., MUD, Adventureland, and CBBS 1978). See History (Sec. 1) in the MVR Inputs. They are digital versions of narratives set in “other realities” since the beginning of civilization. In the earliest years, the quality of textual narratives, story, and emotional appeal drove adoption. Later, visual aspects became a leading differentiator. As graphical technology improved, it crossed a usability threshold, then broadband connectivity increased, and now software advances are giving new creative powers to the user. These developments have allowed social and economic potential to become major new differentiators. Many of today’s “2.5D” VW’s such as Playdo and Habbo Hotel attract millions of youth users. Their less-than-3D graphics can be overlooked due to their social benefits, simplicity, and speed of operation on today’s computers.
Screenshots from Peacemaker, a serious game that teaches diplomacy, development, and conflict resolution in a fictional Middle East.
There is a useful distinction between VW-based multiplayer games, such as Everquest or World of Warcraft, and VW-based social environments, such as Second Life and Sony's Home. Multiplayer games are goal-oriented, with social interaction used as a tool for task completion; such worlds are set in an internally-consistent fictional or fantasy-based realm. In most, entertainment is a primary goal. In so-called “serious games,” training and education are primary goals.
MVR Survey Question 6. Within the next five years, a leading global web company will launch, or buy and launch, a 3D virtual world where users are encouraged to engage in economic transactions and own as legal property products they create in the world.
Social VWs, by contrast, exhibit fewer overt goals and value structures, and offer more open-ended user freedoms, creation of objects, economic and social interaction, and interpersonal networks. In a few social VWs, such as the rapidly growing world of Second Life, the user retains some ownership rights to the objects, land, and other assets acquired in the world. The emergence of broader individual rights inside VWs, a move beyond historically restrictive EULAs (End-User License Agreements) was discussed by MVR participants as a new convergence between virtual and physical space. While inspiring, the vision (John Perry Barlow, 1996) of an emerging independent cyberspace, with its own political and economic rules and jurisdictions, like any sovereign nation, was not echoed by MVR participants, who talked of increasing physical world regulation over virtual space in the foreseeable future.
MVR Survey Question 2. In 2016, the most popular global 3D worlds (by user base) will allow the importing of user trust and reputation rating systems from a variety of other online environments.
Summit Survey N=30, Mean=3.87
Website Survey N=134, Mean=3.94
In practice, the game vs. social world distinction is often blurred, as goal-directed games always emerge inside social VWs, and as social experiences broaden inside the more popular game worlds. The distinction may be further eroded by interoperability, as VW “syndication” emerges in coming years. Having more user freedom to move avatars, interfaces, and assets between worlds—subject to the need to maintain story integrity in game-based worlds—was a common desire of MVR participants. But to move beyond today’s “Walled Gardens,” not only new standards and syndicates, but better systems for user identity, trust, and reputation will be needed, to ensure player accountability to the unique rules of each world.
MVR Survey Question 21. In 2016, of the top 100 global 3D-enhanced online environments how many belong in each of the following interoperability categories?
Both VW’s and “mirror worlds” (virtual spaces that model physical space) offer object creation tools. But in multiplayer VWs, object creation is constrained by the setting and game rules. In mirror worlds, creation is constrained by the need to reflect reality. Only in social VWs is the creation process truly open-ended, and many are becoming open-source as well. At present, Second Life (SL) offers the most powerful object-creation toolset in a virtual world. With some effort, SL objects can be converted to professional 3D programs (Maya, 3DS Max, Solidworks, etc.) for animation, blueprint roughs, and even computer-aided design and production of simple physical-world objects. SL’s next generation of server upgrades will also support spatial audio streams of its inhabitants, which will provide an attractive and useful new dimension to the VW. These are promising developments, though many challenges remain.
A panel discussion inside Second Life. Avatars can watch video, hear audio of the speakers, and text chat with each other privately during the event.
How many social VWs will themselves be open source in the longer term future, and how many will flourish with a limited use of open source on top of proprietary platforms was another topic of debate. Most MVR participants expected significant growth in open source VWs, but with the majority of commerce staying in proprietary worlds.
What is life like in this scenario?
MVR Survey Question 8. In 2016, what percentage of global 3D virtual world and game commerce will occur in worlds that are operated under each of the following business models?
The virtual worlds scenario imagines broad future participation in virtual space commons. Many new forms of association will emerge that are presently cost-prohibitive in physical space, and VWs may outcompete physical space for many traditional social, economic, and political functions. In the 20 year scenario, they may become primary tools (with video and text secondary) for learning many aspects of history, for acquiring new skills, for job assessment, and for many of our most cost-effective and productive forms of collaboration.
In the stronger version of this scenario, VWs capture most, if not all, current forms of digital interaction, from entertainment to work to education to shopping to dating, even email and operating systems, though the 3D aspects may remain minimally used in the latter contexts. Youth raised in such conditions might live increasingly Spartan lives in the physical world, and rich, exotic lives in virtual space—lives they perceive as more empowering, creative and "real" than their physical existence, in the ways that count most.
New identities, new social experiences.
Aided by VW interoperability, an individual may easily access a far broader set of experiences in digital settings than she or he could in the physical world, as well as a vastly larger social network. At the same time, the emerging Participatory Web is providing tools and platforms that empower the user to tag, blog, comment, modify, augment, select from, rank, and talk back to the contributions of other users and the world community. Tomorrow’s 3D Participatory Web technologies will greatly enrich our virtual spaces. See Current Conditions (Sec. 2) of the MVR Inputs for more on the Participatory Web.
Many of today’s “netizens” use 2D personal web pages and home pages in MySpace, and in Korea, 2.5D “minihompy” in Cyworld as their preferred interface to the world. Will tomorrow’s “Metaversans” require potential contacts (those seeking emails, profile info, or live contact) to teleport to the VW address of one of their beautiful virtual homes, with exteriors that display their public interests and values to the world?
Virtual house in Sims 2. When will our favorite virtual objects be available from physical world retailers?
In a more limited version of the scenario, VWs become popular for a few social and professional interactions, and as an interface in certain social contexts, but end up filling a circumscribed role similar to that of present-day televisions, home game consoles, or personal computers. Much of what people do today in the physical world continues with little input from virtual worlds. This limited scenario came primarily from non-technologists, who thought cultural conservatism and economic barriers would be major roadblocks to the stronger vision. The limited VW scenario would be most likely to be dominated by large traditional media companies rather than the pluralistic ecosystem we might expect in the stronger VW scenario.
New thresholds in dynamic photorealism of computer graphics, driven by the entertainment industry, will clearly drive incremental adoption. For example, the ability of webcams to dynamically map the facial expressions of computer users onto their virtual world avatars was considered a probable near-term VW development. The ultimate expression of the VW scenario would include simulation of proprioception (body position), touch, scent and even taste, a form of immersive virtual reality. Yet few participants considered such science fictional advances likely for mass use even in a twenty-year speculation horizon.
A key enabler for the utility of avatars as representatives, screeners, assistants, etc. would be a Conversational Interface (CI), Inputs 8Ac, a dialog platform sophisticated enough to support web queries and responses (text or voice) using seven or more word “sentences,” approximating simple human conversation. Today humanity uses an average of three words in our web searches, and we used an average of 1.3 words in searches on a much smaller and simpler web in 1998. Most participants expected the CI to emerge some time after 2012, and a substantial portion expected it after 2017 or never. Empowering avatars with primitive conversational intelligence would allow us to use them as simple secretaries, agents, and customer support. Individuals could query your “digital twin” 24/7 to learn your public persona and current status, and a CI would promote universal access to and use of the 2D and 3D web, even for nonliterate youth in emerging nations.
MVR Survey Question 19. For users in the U.S., when will the average query length used in leading search applications grow to seven words (voice or text)?
On the social side, perhaps the most obvious persistent trend will be identity experimentation, self-revelation and role play in VWs, and the creative variation of social norms around gender, ethnicity, social class, etiquette, and group values and goals. We see this in today’s pioneering social VWs like Second Life, and social networks like MySpace. As the virtual worlds scenario unfolds, we can expect an explosion in the number of people engaged in such activities, and the ensuing social change to bring both positive and disruptive effects.
Mirror worlds are informationally-enhanced virtual models or “reflections” of the physical world. Their construction involves sophisticated virtual mapping, modeling, and annotation tools, geospatial and other sensors, and location-aware and other lifelogging (history recording) technologies.
Google Earth home screen, North America
Issues and Technologies
Unlike virtual worlds, which involve alternate realities that may be similar to Earth’s or wildly different, mirror worlds model the world around us. The best-known example of a mirror world (MW) is presently Google Earth, a free, web-based, open-standards digital map of Earth. Yet Google Earth is just one of a large class of mirror worlds, which are also known as geographic information systems (GIS). GIS systems capture, store, analyze and manage data and associated attributes that are spatially referenced to the Earth.
The first digital mirror worlds were government-built public resources (eg., the Canadian GIS, 1967). The next were expensive proprietary pre-Internet systems, funded by business and institutional customers (eg., ESRI’s ArcGIS). Such systems remain very popular today, and have some free components. With the advent of the free Google Earth in 2005, a powerful open-standards MW came to the web. Google, Microsoft and others also offer for-pay MWs with additional GIS features not available in the free versions.
Initially, MW maps were based on cartographic surveys, with informational overlays. Later maps were updated with satellite and aircraft imagery, and now some (Google Earth, military systems) are being augmented by ground-based imagery, often produced by cars mounted with scanning cameras, driving around cities to add ground-level images to the building models in our urban mirror worlds.
Amazon’s Block View, street-level GIS images
Amazon’s BlockView (2004-2006), was an early effort at ground-level urban images to supplement online Yellow Pages. While street pictures alone didn’t increase adoption of online Yellow Pages, they make more sense as part of a free open-standards MW with multiple uses (shopping, tourism, navigation, business, research, etc.) especially if the provider (Google, Yahoo, Microsoft, etc.) can sell location-based advertising to accompany MW use. In this regard, new picture-based MWs, like Street View in Google Maps, deliver compelling visual information. The addition of live camera feeds to popular destinations will further increase the stickiness of such environments, and these platforms may be the most direct path toward the Virtual Town Square scenario (see below).
3D building model imported from SketchUp to Google Earth.
With the acquisition and free release of SketchUp, an intuitive 3D modeling program, Google has made it easy for users to add 3D building data (example above) to any of the GIS overlays (tourist attractions, real estate maps, roads, businesses, etc.) in Google Earth. Such tools will help the current generation of web users to more easily create virtual objects. By and large however, individual humans will not build tomorrow’s 3D mirror worlds, though they will most certainly annotate them.
Look instead to automated 3D city model construction software, which converts digital pictures, video, laser, and embedded environmental inputs into 3D models via rapid drive-thru acquisition. This technology is presently in use for small 3D spaces (eg, RealViz, 3rd Tech) and is nearing city-level utility. MVR participants considered such automated 3D hardware and software likely to be a major contributor to our mirror worlds in the near-term roadmap timeframe.
First-gen 3D buildings in Osaka, Japan on Google Earth.
Digital Earth systems add a precise spatial context to physical world information, a context that is either missing or very poor in other media formats. MW maps serve not only as representations, but also as interfaces for access to other networks and devices.
Open-standards and public platforms, like Google Earth, may become the dominant mirror worlds within the near-term roadmap horizon, but proprietary and private versions are also likely to see continued growth in the corporate and institutional sectors, to protect strategies and to seek competitive advantages.
Firms with GIS, sensor or virtual world strategies and experience are potential first-movers in the MW scenario. Standards will be important here, and open source has some potential to shake things up.
What is life like in this scenario?
Some futurists have proclaimed that virtual worlds, the Internet, global outsourcing and telepresence are heralding the “end of geography.” Such ideas were paralleled with advent of the airplane and telegraph a century earlier, which led to predictions of a “borderless world.” There is certainly a limited truth to these perspectives, yet mirror worlds will also make borders, cities, and spatial positioning even more interesting, productive, and important.
In coming years, the proliferation of location- and context-aware sensors will create smart urban and rural environments, and the quality of our mirror world simulations, augmented reality interfaces and object and user lifelogs (history recording systems) will steadily improve. Future classes of RFID and other sensors will allow the emergence of “local positioning systems” (aka location-based systems) that enable us to locate everything we care about in our environment (e.g, tools in the house, children in the neighborhood, friends on the planet) on a realtime MW map.
Google Maps on the
Mirror worlds for the home will be a significant new market. Security, property insurance, moving and storage, rental and barter, interior decorating, construction, and home automation are just a few of many industries that will be significantly affected. The informational power of these tools will create new challenges for crime prevention and privacy protection.
As GPS migrates to the car and cellphone, MW navigation and community search functionality will achieve mass adoption. Location-based search on the cellphone is a near-term development that will be a major new source of social value and provider revenue. Google and other companies are presently sourcing low-price GPS-equipped cellphones, to be offered even in emerging nations. Google’s mobile browser, incorporating Google Maps, will return search queries (eg, “coffee shop” or “supermarket”) filtered based on dynamic user location, with location-based ads a click away.
GIS integration with virtual world object creation (SketchUp, Second Life, etc) will also advance, and mirror world mashups with web-based digital photo sites (Flickr, etc.) are already in use. As internet television software and bandwidth develop (Joost, etc.), MW video integration will be next.
A Mirror World and Flickr photo sharing mashup.
The mirror world interface is a compelling educational, organizational, and commercial tool for understanding and managing global events such as climate and geopolitics. Transnational institutions, NGOs, and others with global focus are likely to be early users as MW functionality improves. Digital Earth systems also offer a unique way to transition between global and local context.
At the regional and city level, the mirror world interface is very useful for navigation, education, commerce, and business analytics, including logistics, marketing, and finance. Once GPS-localization and videoconferencing can both be done within the MW, it seems a compelling platform for socializing and entertainment as well. See Inputs 8Bj for more on recent videoconferencing trends.
Imagine the following Virtual Town Square (VTS) scenario (Inputs 9Bb), in Anytown, USA, circa 2012. You are contemplating your evening entertainment options, so you teleport to various local VTS’s, accurate but flashier mirror world models of your town’s most popular social locations, to efficiently review your options. In each, you can browse 2D screens for movies, entertainment, etc., and quickly see which of your friends are at what venues, based on public reporting by their GPS-equipped phones.
You can also see their 3D avatars and talk to them firsthand, by voice or text, to see what’s going on, or unobtrusively read their public calendars to see if they’d like others to join them. You can talk with the avatars of others visiting the space, who are either browsing along with you or who are downtown in person. Once the digerati of one city find personal value in a social VTS mirror world, others may be built in rapid succession. As with websites, there would be a good economic case for businesses to keep them up to date with the most recent information, live feeds (webcams, etc.) and advert video of the local activity.
Personal use of mirror world tools arises from the value of visual location-based information: directions, local weather, traffic, business conditions, subtle environmental concerns (such as pollution or pollen levels), and the like. Media companies may end up preferring mirror world environments as easy ways to control how their content is displayed. The mirror worlds scenario is one in which a great deal of power comes from the technology's ability to "make the invisible visible"—that is, to reveal processes and flows (e.g. which local restaurants near your present location served the best-rated food last month, according to your preferred social groups) that would otherwise be too subtle or complex to recognize.
A virtual town square in the social VW Second Life.
In the longer-term time horizon, given a sufficiently robust model of the real world, complete with abundant live data sources and preferences and values maps of the inhabitants, mirror worlds will eventually come to offer a powerful method of testing plans through data mining and simulation. Business, environmental, and political strategists may use a mirror world system to check the plausibility of plans against a physical or virtual community’s publicly expressed preferences and values. For more on this concept, see the Valuecosm, Inputs 9Cp.
Such a high-reflectivity model of Earth’s visible and intangible aspects is outlined by David Gelernter in Mirror Worlds. Gelernter is optimistic that our coming data-rich geographic simulations can give us not only tree-level insight but also forest-level “topsight” into complex global systems, many of which are presently obscure.
Mirror Worlds by David Gelernter, 1992
If the leading mirror world tech trend is towards increased data inputs (proliferating global sensors) and complexity and accuracy in our sims, the leading MW social trend may be efforts of the powerful to control access to the most useful new information. Mirror worlds are democratizing and pluralizing only to the extent that everyone has access to and can annotate them. If that access is restricted, they can easily become instruments of state or corporate control. As long as this is seen as a socially-undesirable outcome, much political effort will go into finding ways to maintain and equalize access. The rising power of the individual to use technology in socially destructive ways will be one problem made worse by MW access. In wise societies, this problem will be countered by the rising social transparency and accountability that mirror worlds, augmented reality, lifelogs and related technologies provide. Different cultures will make different choices with respect to MW prevalence and access, but in general, they hold great promise to be a positive-sum social force, and to protect both civil liberties and social values and identity.
In augmented reality, Metaverse technologies enhance the external physical world for the individual, through the use of location-aware systems and interfaces that process and layer networked information on top of our everyday perception of the world.
Artist’s idea of augmented reality heads-up display (HUD)
Issues and Technologies
Historically, the augmented reality (AR) concept is based on the emergence of mirror world maps and global positioning networks, including the U.S. GPS and its European competitor, Galileo (due in 2011), as well as cellular phone localizers relying in part on triangulating cell towers. As GPS has become increasingly commonplace, new services have emerged to take advantage of this geographic information, from location tagging and logistics monitoring to location-based games and context-aware advertising. Such services are fairly rudimentary today, but will improve greatly in granularity, accuracy and usability.
Augmented reality depends on the further development of intelligent materials and the "smart environment"—networked computational intelligence embedded in physical objects and spaces. As described in Adam Greenfield's Everyware, this vision of the so-called "Internet of things" moves well beyond today’s primitive classes of RFID (radio frequency identification) tags. Concepts such as the "spimes" described by Bruce Sterling (individually-identified objects that can be tracked through both time and space over their lifetime) or Julian Bleecker's "blogjects" (objects that keep a running public record of their condition and use) offer examples of the ways in which materials, goods and the physical environment play a part in the augmented reality world.
Physical hyperlinks, Inputs 2Ao, are a recent major AR advance. PHs are machine-readable identifiers (1D and 2D barcode, RFID tag, image, sound, fingerprint) that can be resolved by a cell phone camera. A high-capacity (4,300 character) square 2D barcode called the QR (“Quick Response”) code is now proliferating in Japan, with QR code readers preinstalled on all new 3G cellphones. They are appearing on business cards (eliminating data entry), magazine pages (for discount coupons), packaged goods (for nutrition information), airport kiosks (for paperless airline travel), even billboards (for movie trailers). Once recognized by the camera, a single click dials a number, starts an email, or takes the user to an internet site. Future applications are limited only by the imagination.
A few of the many QR code applications in Japan.
Another important aspect of the AR scenario is the interface, the ways and choices users have to access virtual information overlaid on the physical world. One type of interface is a heads-up display (HUD), providing context-significant information through a mobile viewscreen (window, eyeglasses, cell phone screen, etc.). Microvision uses a tiny laser that paints a virtual image on a flipdown screen, or even directly on the user’s retina.
Microvision’s Nomad AR device
More conventional visual interfaces, such as mobile phones and the navigation screen in cars, are bound to be the most common AR interface for the near-term. Nevertheless there is room for innovation, as in wearable phones whose lightweight visual display covers the back of the hand and wrist (see the Carpal PC, Inputs 9Ab). Mobile wearable screens are to some degree virtual or mirror worlds, as they command all of the user’s attention, at least for a glance. But they are also AR, as context-sensitive information is overlaid on them as they move through the physical world.
Another promising AR approach is an audio interface, with voice- or context-driven information delivered via earpiece (e.g., the ability to ask your search engine anything, and have an answer whispered into your ear, contextualized to your physical location). Wearable audio AR may require a more robust Conversational Interface before it reaches mass adoption however.
What might emerge in the near-term is location-based cellular radio (LBCR), Inputs 9Bd. Today, 3.5 and 4G wireless platforms can already stream internet audio to the car radio and cell phone. Add GPS and a mirror world directory system and you can deliver location-based streaming radio to the mobile user. Many mobile users would like car and cell phone radio channels that give them 1) ultralocal news, politics, weather, and traffic, 2) reviews and business-published info on local restaurants, shops, and entertainment events, as they are approaching them, and 3) educational and historical information for local landmarks. LBCR might be a multi-billion dollar industry by 2016. Or it may be a white elephant platform still waiting for user adoption.
Steve Jobs demos the iPhone.
MVR Survey Question 12. In 2016, what percentage of global mobile device users (cell phone, PDA, etc.) will have always-on broadband internet accessibility from their devices?
Summit Survey: 81% of users
Website Survey: 79% of users
Another potential near-term AR platform is the Display Table/Game Table, Inputs 10Af, a kitchen, dining room, or workroom table with a touchscreen display surface, networked vertical wall display(s), and individual AR displays for each user/player. Once affordable, such a device will facilitate new videoconferencing, collaboration, entertainment, and social experiences beyond the living room and the standalone computer.
What is life like in this scenario?
The augmented reality scenario offers a world in which every item within view has a potential information shadow, a history and presence accessible via standard interfaces. Most items that can change state (be turned on or off, change appearance, etc.) can be controlled via wireless networking, and many objects that today would be "dumb" matter will, in the augmented reality scenario, be interactive and to a degree, controllable. To the AR generation, such properties will be like electricity to children of the 20th century: essentially universal, expected, and conspicuous only in their absence.
Whoever delivers the first useful and scalable AR operating system and standards, perhaps via the cell phone platform, may become a central player in this future. As virtual data proliferate, information overload will be a common problem. The best of these will regulate human use of the system, respecting natural work, rest, and recreation cycles. In the near-term, AR devices may employ today’s collaborative filters, which self-organize to advance one’s interests and values. This will empower user annotation and the expression of individual opinion: the Participatory Web. Smart tag-based networks will allow individuals to advise friends on which restaurants, shops or services are worth visiting, and which should be avoided. Time-based processes (such as appointments or deliveries) can be followed with a small widget in one's visual interface, unobtrusive but always available.
||Microvision’s wearable AR display concept.
In the longer-term future, different people may have very different experiences of the same physical location. In extreme cases, one could use AR to hide images (such as signs, video displays, even other people) considered distracting or offensive. In a new form of self-obsession, isolation, and addiction, some might choose see only “Potemkin Villages,” an information façade catering to their pre-existing biases and desires, and obscuring unpleasant reality. Media services, religious groups, software companies, and many other players are likely to compete in the filter market, and economic and political pluralism should help ensure these systems empower rather than control the individual.
In lifelogging, augmentation technologies record and report the intimate states and life histories of objects and users, in support of object- and self-memory, observation, communication, and behavior modeling. Object Lifelogs (“spimes,” "blogjects," etc.) maintain a narrative of use, environment and condition for physical objects. User Lifelogs, ("life-caching," “documented lives,” etc.) allow people to make similar recordings of their own lives. Object lifelogs overlap with the AR scenario, and both rely on AR information networks and ubiquitous sensors.
Issues and Technologies
Lifelogging is the capture, storage and distribution of everyday experiences and information for objects and people. This practice can serve as a way of providing useful historical or current status information, sharing unusual moments with others, for art and self-expression, and increasingly, as a kind of "backup memory," guaranteeing that what a person sees and hears will remain available for later examination, as desired—what Microsoft founder Bill Gates called a “documented life” in The Road Ahead, 1995.
Lifelogging emerges from accelerating technological trends in connectivity, bandwidth, storage capacity, sensor accuracy, miniaturization, and affordability.
TrackStick, a $200 GPS lifelog the size of a pack of gum
Object lifelogging is presently seeing a wide range of incremental advances. GPS lifelogs like TrackStick which interfaces to Google Earth, are concealable in cars and objects, and are making inroads in law enforcement. In the consumer market, Toyota Japan offers in-car cameras, networked to the Toyota Security Center, for auto theft prevention and recovery. Some consumer in-car cameras now record outside the windows, to detect the license plates of cars that do damage to the vehicle.
As inexpensive car video lifelogs become widely available, able to relay the last several minutes of their exterior visual footage to any phone number or email address at the touch of a button, accidents, red-light violations, unsafe driving, and other infractions may be reported by concerned citizens with great frequency. Traffic laws and norms will see adjustment as a result.
Nike+ and iPod: Personal trainer, global running community
Nike and Apple have formed a partnership to turn shoes into lifelogs and personal trainers, using the iPod and the web. Tens of thousands of runners upload their running statistics daily to the Nike Plus community. As their cost drops steadily in coming years, many new object lifelog opportunities will emerge. Would you pay an extra $10 for a computer screen that logs a memory of its recent visual states in case of a crash? What would you pay for lifelogs on your car, keys and wallet? For a wallet that notified you if the credit cards weren’t quickly replaced within it? Used appropriately, object history and smartness can improve our awareness, security, and productivity.
Nokia lifeblog organizes cell photos into an organic timeline
that can be annotated, mobile blogged, and shared.
User lifelogs are also in broad development. Perhaps the most obvious examples of early user lifelogs are the current generation’s widespread use of digital and cell phone cameras to document and share life experiences online. Leading phone makers like Nokia and websites like Flickr have platforms to facilitate photo taking, annotating, sharing and mobile blogging.
Justin.TV runs a
24/7 video ”lifecast”
Justin.tv, streamed to the web by Justin Kan via a small wearable headcam and four Verizon EV-DO cell modems ($240/month in bandwidth costs) is just the latest example of early “lifecasting” activities. Such systems may be affordable for youth and specialty use in the near-term roadmap horizon, and new “life sharing” opportunities will emerge.
MVR Survey Question 14. In 2016, what percentage of the U.S. population ages 13-30 will allow their trusted group to view 3D images of what they are doing in realtime (through wearable cameras) at least once a month, and to be able to give feedback or advice?
Summit Survey: 29% of population
Website Survey: 36% of population
Child security from abduction is a presently a significant public concern, and another potential for near-term user lifelogs. Affordable localizer devices that can be concealed by implant in the body are likely decades away, and may face high hurdles to public acceptance. What is feasible in the next ten years, however, are 3.5 and 4G security cellphones, worn at the belt or like a necklace, on which the camera, by its red light, is obviously on, recording, and wirelessly transmitting its image to a remote network. Such a security lifelog, with automatic recording and manual reviewing features, would capture criminal acts on video, even if the phone were immediately destroyed. In the same way that CCTV cameras in car parks lower car theft rates globally, such a device, worn by very young children and security conscious adults in public, might have some protective effect for their wearers. Recording in school to prevent bullying, etc. would be another potential use, and an obvious personal vs. state’s rights issue to be adjudicated in coming years.
concept from Wearcam.org
Lifelogging technologies offer two primary functions: first, they serve as a kind of "TiVo" for one's life, recording the sights and sounds one encounters throughout the day; second, they enable collaborative sharing and aggregation of life experiences. Both functions are potentially socially-disruptive, even as they offer capabilities of immediate value to users.
The primary technological hurdle for the mature lifelogging scenario isn't the hardware, but the software: how does one tag, index, search, and summarize the terabytes of rich media archives of one's own life? Several technology companies (Microsoft’s My Life Bits, etc.) are hard at work on this problem.
Beyond the near-term youth market, truly powerful user lifelogs seem unlikely to emerge until we have intelligent autocaptioning and autosummarizing systems, and a functional Conversational Interface (post 2016?), allowing voice-driven search on a wearable system through one’s archive of past experiences (e.g., “show me that conversation last Summer when I was discussing abc with xyz.”).
MVR Survey Question 15. In 2016, what percentage of the U.S. population ages 13-30 will use 'lifelogging' systems during significant portions of their lives?
Summit Survey: 24% of population
Website Survey: 32% of population
What is life like in this scenario?
Life in the lifelogging scenario has the potential to be simultaneously empowering and demoralizing, in the sense that the older generations may have some difficulty adjusting and a nostalgia for simpler, earlier times.
For lifelogging adopters, retention of past experiences will become functionally perfect, but recall and analysis of those experiences will only be as good as the web-based indexing and search software, which will constantly improve itself over the lifespan of the user. Even with minimal analytical capabilities, such systems would be of great value to experimental youth, to technology-inclined elderly (expect early uptake in Japan), to business people, to civil servants, and many others.
A perfect memory isn't necessarily an ideal, at least by current social standards. Human relationships are aided by the consensual misremembering of slights, allowing the sting of insults and personal offenses to fade over time. With easy access to records of past wrongs, “I forgot,” will be much less frequent, and some will find it impossible to "let bygones be bygones." On the positive side, new social accuracy will provide opportunities for individuals to more frequently admit their mistakes, and after some ego adjustment, help them be more tolerant and open to a change of mind and behavior. We see such learning on some (not all) blogs today, which are accurate text-based lifelogs of past arguments in social space.
David Brin makes this point well in The Transparent Society, 1999, an introduction to the social changes we can expect in tomorrow’s highly virtualized and publicly transparent society. Individuals in a democracy ultimately become nicer when their actions are available to social and self-observation, though not without a struggle. Behavior change is never an easy process.
Add network capability to this technology and life gets especially interesting. Unlike virtual worlds, lifelogging won’t allow you to walk in another person's shoes, but it does allow you to look at the world through another person's eyes. Or multiple people's eyes: memories tagged for a particular time and place can call up similar recordings from others at the scene, giving an individual access to multiple perspectives on an event.
Potential applications of such capacity are legion: more accuracy in law enforcement, better education, training, counseling, self and social awareness, conflict resolution, etc. It is also a powerful example of sousveillance ("watching from below") to balance the surveillance (“watching from above”) ability of the modern state. Today, the WITNESS project's online portal offers global human rights activists a place to send images and video documenting abuses by powerful actors; lifelogging technologies would make that possible for everyone.
Witness.org: Lifelogging circa 2007
Systems advanced enough to recognize objects, symbols and individual faces, visual AI tasks that many experts expect to be accurate enough for general use in ten to twenty years, will offer powerful new abilities not just to society but also to individuals. At a minimum, the software would be able to call up earlier interactions for quick review, or at the very least a name and context. If systems can be readily networked, the lifelogging gear could call up references from trusted friends and relatives, giving any one person access to the collective social memory of her personal network.
This has obvious implications for reputation networks. Inevitably,
once it's possible to access networked memory references about someone
or something, users will wish to share their opinions about their
own experiences with the subject. As long as the reputation network
focuses on products and services, the group ratings will differ
little from today's collaborative product recommendation systems.
As the network begins to apply to other people in noncommercial
dimensions, however, questions will arise about liability for spreading
harmful misinformation. New legal frameworks will likely ensue.
A leading technological trend over this time period will be the increasing ability of lifelogging systems to make meaningful connections between disparate "memories," both individual and collective. In its fullest expression, such technology may become not simply a backup memory, but a backup sub-conscious, offering powerful cognitive augmentation and advice by past example. Viewed from the biggest picture, when coupled with ongoing work on the development of artificial general intelligence, lifelogging becomes one of several valuable pathways to a greater integration of human and machine “minds.”
We should conclude our final scenario with a major observation: the technologies in three of our four scenarios, mirror worlds, augmented reality, and lifelogging, will all strongly increase public transparency—and with user consent, private transparency as well—in coming years.
How far might we take this transparency trend? Will we come to
regard the present, an era where people can go out in public without
biometrics or electronic signatures that uniquely identify them
to the network as a “Wild West” of crime and lawlessness?
The far future is hard to visualize, but we can imagine many socially
attractive near- and longer-term transparency steps along the way.
Networked localizable weapons: A future lifelog development?
How likely is it, for example, that once they are sufficiently
inexpensive and miniaturized, we will see laws mandating networks
and lifelogs (eg., GPS-on-a-chip, feeding into a gun’s “flight
recorder”) to be installed on all our new small arms, weapons,
explosives, and other mass lethal technologies (Inputs
8Aj)? Would societies that pioneer such networked and localizable
weapons (NLWs) find they turn offensive technologies into defensive
social assets, and are NLWs a particularly high-probability future
for liberal democracies?
It seems reasonable to expect that the leading long-term social
trend in lifelogging will be grappling with the impact of greatly
elevated transparency, including all the inevitable attempts to
hack, game or otherwise manipulate these systems. Security, privacy,
fraud prevention, and the protection of civil liberties for users
and those recorded will be ongoing concerns. As Justin.tv’s
website header says, “We’re going to need more lawyers.”
We can also expect significantly different responses to these technologies
country by country, particularly in their early years. To generalize,
we might look to security-conscious and innovative countries like
Singapore, Israel, South Africa, the U.S., and Korea for early innovation,
to Asia for technical leadership, and perhaps to Europe and the
U.S. for legal innovations that define their use consistent with
growing personal liberties and social responsibilities.
Citation: Smart, J.M., Cascio, J. and Paffendorf, J., Metaverse
Roadmap Overview, 2007.
Commons Attribution-Noncommercial-Share Alike 3.0 License.