Trend 3: Bridging Digital and Physical Experiences
As users transition across devices, environments, and activities, designers must diversify interaction and minimize the gaps within user experience.
Meredith Davis
Trend 3: Bridging Digital and Physical Experiences
As users transition across devices, environments, and activities, designers must diversify interaction and minimize the gaps within user experience.
Meredith Davis

Trend 3: Bridging Digital and Physical Experiences

Download the PDF version with competencies.

People transition across devices, environments, and activities in continuous communication and service activities. Users expect technology to provide seamless, unified experiences, even when moving among messages and services from different sources. New technological platforms amplify experiences in the physical world but also create gaps among devices and diversify the interactive behaviors required of users.

Consumers toggle between the digital and the physical. A study of more than 2,000 consumers by Salesforce found that 80 percent researched products online, even when they fully intended to make their purchases in brick-and-mortar stores. Consumers check in for a flight or doctor’s appointment, order groceries, summon a ride (perhaps in a driverless car), and book a stay in a stranger’s house from the convenience of a smartphone anywhere at any time.

Today’s digital experiences, therefore, are embedded within larger activities through which people interact with their social and physical world. Understood through this expanded framework, experiences mediated by design include what users do before and after their interactions with technological screens. It is in the gaps between online and offline environments, between devices, and between related service providers that users often lose support, unnecessarily repeat actions, or miss information important to completing a task. These moments interrupt the continuity of the brand, product, or service experience. Such circumstances will only accelerate as technology takes over more everyday activities. The challenges for designers, therefore, are to anticipate experience gaps for diverse users and to overcome them as simply and seamlessly as possible.

It is also crucial for designers to develop integrated systems—ecologies of products and services that cross different devices and service providers (see also Trend: Aggregation and Curation). The variety in these relationships makes them especially challenging, requiring new strategies and collaborations. Platform design, the set of components and protocols through which companies and organizations deliver services to customers via applications, supports rapid response to changes in the service ecology by the organization or third-party collaborators.

Evidence of the trend in practice

Click-and-collect — In click-and-collect experiences, consumers purchase items online for pick up in physical locations. Nearly one-third of all consumers used click-and-collect during the last holiday season. At the same time, 69 percent of those consumers also bought something else when they went to the store to pick up online purchases, and 36 percent bought something in an adjacent store. This is good news for brick-and-mortar retailers who fear losing sales to online services.

However, there can be areas of friction in click-and-collect systems that break continuity in the buying and brand experiences. Online inventory information can be inaccurate, images misleading, store parking unavailable, and hours of operation inconvenient. These shortfalls present design opportunities. Curbside is an app through which consumers order online at participating stores. Email notifies them that orders are ready (usually within 40 minutes) and employees bring purchases to the curb, eliminating parking and store navigation. Curbside customers visit participating stores 2.5 times more frequently than other shoppers.

Online giant Amazon uses lockers in its recently acquired Whole Foods stores to deliver customer orders of any products, avoiding the risk of porch theft for working people who are not at home during the day to receive packages. This service allows busy consumers to combine package pickup with grocery shopping.

Project name: Amazon Locker
© Amazon
Amazon Locker allows customers to shop online and pick up their package at their convenience, thereby avoiding the risk of theft when a package is left on the porch.

In-store experiences — The Google Shopper Marketing Council indicates that one in three smartphone shoppers would rather use their phones than ask a salesperson for information. Companies, therefore, pay increasing attention to the in-store experience as an extension of online research. Home improvement store Lowes, for example, offers an app through which customers find product information and confirm that the product is in stock. The app describes the aisle and shelf location of the product in a nearby store, as well as a map of the quickest in-store route for picking up all items. The route really matters when customers are pushing heavy carts loaded with building materials. Lowes employees use the same app in serving customers who prefer face-to-face interactions, rather than passing them off to an employee in another department. The company is testing an augmented reality in-store navigation system that directs customers’ real-time movement in finding products.

Augmented, virtual, and mixed reality — Practical applications of augmented reality impose computer-generated text and images on real-world/real-time settings for viewing on smartphones and other mobile devices (see also Trend: Making Sense in the Data Economy). Tourism applications add information, objects, and simulations to the experience of historic and archaeological sites. Cities use augmented reality in pedestrian wayfinding, showing access to public transportation and points of interest. Sportscasts impose graphics and statistics on live events, enhancing the fan experience. Architects add buildings to the landscape to help clients envision projects. In these examples, the digital adds to rather than leads to the physical experience.

IKEA has long been technologically sensitive, incorporating wireless charging into its furniture and creating lighting that responds to any smart home system. IKEA Place is a smartphone app that virtually places any of 2,000 IKEA products in the home. An advancement over the company’s 2012 augmented reality app, the recent version (developed in partnership with Apple) allows users to move the object around the room. Scanning the home floor first ensures that the item appears proportional in size to other objects in the room, solving the problem of fit. At 98 percent accuracy, the fidelity of these features builds buyer confidence that physical reality will match what they imagine, whether ordering online or purchasing in the store.

ikea-place-app

Educational applications of augmented reality overcome some of the gaps between learning at home and in school. Using the HP Reveal app (formerly Aurasma), students can scan a page of their homework assignments and the system will launch a video of their teacher helping them through the problem all the way to the solution. Scanning an image on a student’s classroom desk prompts pre-recorded words of encouragement from a parent. For hearing-impaired students, online videos that use sign language augment vocabulary flashcards. These and other HP Reveal features can help maintain a support system for students as they traverse the gaps between the two physical environments.

Although content for virtual reality technology is currently limited, there are applications that go beyond entertainment. NC State University Graphic Design Professor Dr. Derek Ham, in collaboration with the National Civil Rights Museum, created “I Am a Man,” a VR experience that places the viewer in the 1960s. Viewers “walk” with peaceful demonstrators on a sidewalk along a Memphis street patrolled by army tanks, and arrive at the Lorraine Motel as shots ring out, assassinating Reverend Martin Luther King. Ham’s work was supported by Oculus Launch Pad, a scholarship for content creators to help VR reach a wider audience.

Mixed or hybrid reality exists along a continuum between the completely real and the completely virtual. Unlike augmented reality that overlays virtual objects on the real world, mixed reality anchors them in the real environment, allowing users to interact with them. In other cases, mixed reality begins with the virtual world and overlaps real objects. Microsoft’s HoloLens aligns real-time, three-dimensional data with a job site by overlaying holographic models on the physical environment. Architectural clients can walk through a SketchUp model in HoloLens, in order to gain a better understanding of the implications of design decisions.

Core concepts and principles

Experiences, not objects — Shelley Evenson, managing director for strategy at Fjord, described the user experience cycle (see also Trend: Accountability for Anticipating Design Outcomes): The user enters the experience through a successful first impression that attracts attention and captures the imagination; Orientation follows, providing a preview of what is possible. Starbucks customers, for example, learn how to order coffee by watching and listening to counter interactions with others ahead of them in line. This is by design. A compelling interaction not only delivers something of value in ways that are delightful and informative but also representative of the organization providing the product or service. The language of Starbucks baristas—grande and venti rather than medium and large, for example—reinforces the special nature of the product. This brand language and behavior is embedded in the system, so when Starbucks customers come back or go to another Starbucks, they have an equivalent experience. Over time, consistency in the quality of the product and service experience retains customers. And eventually, customers actively advocate for the brand. 

experience cycle chart

While the typical Starbucks experience takes place entirely inside a physical environment carefully managed through design, the challenge in most of today’s digital-to-physical experiences is that they play out over both time and space. The system must maintain consistent success without dropping support for users as they move from one stage of the experience cycle to the next. And in most cases, digital-to-physical services must accommodate much greater variation in the motives and behaviors of users than those of the average coffee drinker.

It is important for college design students to pay attention to the full cycle and to understand that designing for experience is really about people’s goal-oriented behavior, not about objects and spaces. This means research and methods that focus on people’s activities are paramount in designing support for satisfying experiences.

Touch points — Touch points are any type of contact or interaction between a goal-oriented user and a company or organization. The user or the organization initiates these interactions in a variety of digital and physical channels. Effective touch point experiences allow the user to acquire information, interact with products, engage in services or activities, and gain feedback that affirms or redirects their behavior along the path. Touch points are frequently decision-making moments through which the user advances toward the satisfaction of a goal. With respect to the experience cycle, touch points should deliver the right experience at the right time.

Designers typically represent the touch points of an experience through a user-journey map, which captures discrete user decisions and interactions from thinking about information, products, and services to the actions necessary to complete a task. These maps are different from concept maps that identify the components of a system or site maps that show the organization of online content. Journey maps reflect particular users’ perspectives, often defined through personas or extreme users whose paths are likely to vary. While it is tempting to show a smooth journey from start to finish, the best maps also show points of user confusion and the consequences of error. In mapping digital-to-physical experiences, it is especially important to think about gaps where users lose support or experience friction in the transition from one environment to the next.

tegtmeyer-autism-conept-map

Leverage points are places where a small amount of effort produces a positive change in the overall function of the system or experience of the user. For example, providing feedback that confirms a step in the process has been completed—email saying that an order has been received—makes the user more accurate and confident in proceeding to the next step of the experience. In identifying every touch point, user-journey maps allow designers to visualize the cascading consequences of making design changes at any single point in the system.

Touch points are also critical to a brand strategy (see also Trend: Resilient Organizations). They identify places for introducing the organization and its purposes; confirming that interactions are with the company or organization, despite moving from one communication channel to another; and maintaining continuity of the brand experience. Increasingly, these touch points are conversational; the interactions between users and the system are two-way, and involve natural language rather than point-and-click responses. Empathy and feedback are built into the development of intelligent systems, which learn from individuals over time.

Resources

Aguera y Arcas, B. (2010). Augmented Reality Maps. TED Talk.

Atzmon, L. and Boradkar, P. (2017). Encountering Things: Design and the Theories of Things. London: Bloomsbury Publishing.

Deloitte Insights. (2018). “Digital Reality: The Focus Shifts from Technology to Opportunity.”

Dubberly, H. (2017). “Connecting Things: Broadening Design to Include Systems, Platforms, and Product-service Ecologies.” Encountering Things: Design and the Theories of Things. London: Bloomsbury Publishing.

Dubberly, H. and Evenson, S. (2010). “Designing for Service: Creating an Experience Advantage.”

Fjord. (2017). “What is Data + Design?

Forlizzi, J. (2008). “The Product Ecology: Understanding Social Product Use and Supporting Design Culture.” International Journal of Design, Volume 2, Number 1.

Google. (2018). Google’s Material Design.

Harris, D. (2015). “How Brands Can Bridge the Divide between Digital and Physical.” Creative Bloq.

LinkedIn. (2018). “2018 Workplace Learning Report: The Rise and Repsonsibility of Talent Development in the New Labor Market.”

Microsoft. (2018). “What is mixed reality?

It is in the gaps between online and offline environments, between devices, and between related service providers that users often lose support.
The digital adds to rather than leads to the physical experience.
Designing for experience is really about people’s goal-oriented behavior, not about objects and spaces.

Tags experience design service design technology data science research