Archie Miller of CarMax on Implementing Lean UX in Large Organizations

Introduction

In 2014, CarMax set off on a journey to bring in Lean UX design principles to its organization. The company, known for revolutionizing the used car buying process in the U.S., is now focused on redesigning its web and mobile apps to seamlessly integrate the digital and store experience for its customers. While Lean UX is often associated with small companies that already have lean organizational structures, there are distinct challenges when it comes to large companies. Archie Miller, Team Manager of CarMax’s Interaction Design Group, talks about some of those challenges, and how his team dealt with them, through the lens of a single project to validate the company’s online process for scheduling a test drive. This article is based on Archie’s presentation at the AxureWorld conference.

The Big Picture

With 16 million visits per month, CarMax.com lets car buyers browse thousands of vehicles, put a seven-day hold on vehicles that catch their eye, transfer cars to their local CarMax store, and inquire about financing, among other things. CarMax is focused on evolving its online capabilities as consumers do more of their shopping and research through web and mobile apps. As part of this larger effort, my team needed to evaluate and validate which existing functionalities worked well, and which could be improved.

We adopted a customer-centric approach for our digital storefronts. CarMax already had disrupted the industry more than 20 years ago by turning a universally dreaded life-event, buying a used car, into one that’s straightforward and simple. We’ve even designed ways to make the experience fun. When buyers are ready to take their cars home, our salespeople bring out the vehicle with a big bow and offer to take a picture of our customers with their new purchase; many share those photos with their friends on social media.

We wanted to redesign CarMax’s digital experience to reflect the company’s customer-centric approach of simplicity, transparency and innovation. In addition, we needed to retool our site to stay ahead of the shift towards mobile and escalating consumer expectations for their online experiences. To do this, CarMax reorganized their digital teams to operate more like a lean startup product organization, which included creating an Interaction Design team. This is about the time I joined CarMax. What attracted me was the company’s dedication to change—even though it continues to be successful. A lot of companies wait until success eludes them; then it’s too late.

Lean UX vs. Enterprise

While CarMax’s online and offline values are similar, the paths for each turned out to be very different. Success at the brick-and-mortar level required strict adherence to regimen. Similar to many large-scale retailers, our company used procedures precisely documented for all aspects of its operations. Fine-tuned over 25 years, these procedures ensured a uniformly high-quality customer service experience across our 155 stores and 22,000 associates nationwide.

My team had adopted a Lean UX approach laid out in the book by Jeff Gothelf and Josh Seiden. This method is challenging to pull off in an enterprise for a few reasons:

• Lean UX emphasizes skills over roles interacting within several small “one or two pizza” teams. Large companies are often more accustomed to larger teams with less flexible organizational structures.
• Related to the preference for skills over roles, Lean UX calls for plenty of collaboration among team members and stakeholders to foster a shared understanding. In large organizations, documentation and project task dependencies can impact communication and create silos.
• Designing lean means fast iterations and the ability to focus on the speed of learning thus mitigating risk earlier. Enterprises often place value on speed to market by focusing teams primarily on deliverables and output.
• For large organizations, accountability is tracked through KPIs. In Lean UX, an outcome that delivers value for customers becomes a leading indicator of success—those outcomes then drive the desired business impact for the enterprise.

In the case of CarMax, we had support from leadership down, and this places us in a much better place than other large companies where teams try to sell Lean UX from the bottom up.

The Lean Team

Our core product team consists of a product manager, lead developer, and UX designer. Each of the three has a voice, and there is constant collaboration and dialogue within the core team. When there’s a disagreement, the product manager acts as the tie-breaker.

The Task

Among our core online offerings is the ability to schedule a test drive. Customers who selected the option are shown a pop-up module that asks them to select a time and date for their test drive and enter their name, phone number, email address, password, and zip code. If the customer is brand new to CarMax, the site walks them through a registration process.

But if the email address they enter is already in CarMax’s system, our site returns an error message, saying there is already an account associated with that email address and suggesting that the customer sign in. That means, for existing or return customers, the scheduling process is interrupted. They must back up, click a tiny “Sign in” link and re-enter their email, password and phone number. We wanted to validate this process and see if it could be improved.

The First Hypothesis

Our first hypothesis was that if we removed the requirement to sign in or create an account, it would reduce friction for customers and make it easier to schedule test drives and increase increase leads to stores.

To test this, we recreated the scheduling process in Axure, using screengrabs and a handful of basic widgets such as text fields, drop menus and buttons to make a “retrotype” that mimicked our existing flow (as opposed to a prototype for products that don’t yet exist). The retrotype differed from the real thing in one key aspect—it replaced the account sign-in requirement with a simple checkmark Captcha that we’d need in place to foil bots. We used Ethnio to recruit online testers. We connected with our testers using a screen-sharing application and by phone and watched as they walked through our retrotype on Axure Share.

What we learned surprised us. Some of our testers expected to either create an account or sign in to their existing accounts. When there wasn’t a prompt to do so, they questioned if they had done the scheduling correctly. Had it really gone through? Would CarMax recognize who had scheduled the test drive? Secondly, our testers universally hated the Captcha, even though it required far fewer steps than the sign-in process.

In other words, our assumptions were wrong. The problem wasn’t the requirement to sign in or create an account; it was the way existing customers were asked to do so. And that sent us back to the drawing board.

The Second Hypothesis

We realized from our first round of testing that customers actually expected to associate their car purchasing activities with their accounts so they can also track which exact cars they’ve reserved, test drove and researched. This is especially important within a used car buying experience, because, unlike new vehicles, two cars of the same year, make and model can differ.

So we returned to the existing flow and looked to see how that could be improved while preserving the sign-in and account creation process. As mentioned earlier, if a customer enters an email address that is already in CarMax’s system, our site returns an error message, saying there is already an account associated with that email address and suggesting that the customer click a “Sign in” link, and re-enter their email, password and phone number.

We adjusted the flow so that both the sign-in and account creation was more obviously integrated with the scheduling. In Axure, we were able to reuse the assets from the first round to quickly create a test prototype that incorporated this new flow.

Outcome

In less than three days, we were able to validate sign-in / account creation within a core function of our website as having value to both for us as a business and for our customers. Our lean approach meant that we were able to accelerate our learning. The point was not to get done quickly and fail fast. It was to get done fast, so we could learn fast.

This “quick win” demonstrated the value of running lean experiments that ultimately lead to design decisions based on evidence rather than assumptions. In this particular test, our goal was to rev up our learning velocity and deliver the right solution for generating leads for our stores. We learned that account creation and sign-in was not the issue as we hypothesized. It was all about setting the expectation that an account was needed.

Lessons Learned

Lean UX in a large organization is challenging, but possible. Our initiative had the fortune of having the support of company leadership, including our CEO. But on a day-to-day basis, we still need to prove out the model to everyone else at the company whose help we greatly rely on. This is especially challenging when parts of the existing culture, which has yielded so much success, encounters new Lean UX principles.

The most common challenge for us is re-orienting expectations from output to outcome. In the past, we measured success by hitting a project deadline fast and under budget. Now we must to convince stakeholders to trust a new set of metrics. The velocity of learning should be more important than velocity to deliver—it’s no good to deliver the wrong solution quickly. Of course, accelerated learning results in accelerated delivery, but what gets delivered may be different.

From this and similar experiments, we learned the following:

Start small and prepare to compromise. When you initiate Lean UX practices to an enterprise, you might encounter “corporate antibodies.” These antibodies sense change and attempt to normalize the new process based on the more familiar process. Sometimes, you have to accept a compromise, but that’s part of the challenge—recognizing when Lean UX can help and when it can’t. Rather than completely redesigning the test drive scheduling process from scratch, for example, we started small, with a minor adjustment, to demonstrate the value of quick learning.

Build teams from within. In this exercise, our product manager was relatively new to Lean UX methods. This experiment initiated him into the process and won him over. He was able to see first-hand the value of thinking by doing.

Create orbits of communication. In a webinar on Lean UX for Enterprise, Josh Seiden, co-author of “Lean UX,” advocates the use of an “orbital system” as a way to loop in participants in an enterprise without bogging down the core team. He suggests holding regular “retrospectives” for people that your project needs down the line to succeed—compliance, customer service, and so on. At CarMax, we do this through a monthly Open House where we invite colleagues to see what we’ve been doing and share what we’ve learned. We usually just print screen shots and tape them to our wall so we can point to them to tell our story. Then we talk about what’s next. The visibility not only helps the team to externalize their thinking, it eases others into the process.

Prepare for obstacles. Open houses also help us prepare for potential obstacles. It’s held for stakeholders who don’t live with the product team. That usually includes executives who tend to see everything through a lens focused on strategy. That perspective can result in “grenades” being tossed. But, armed with information from an open house, they begin to appreciate that we can move quickly—and minimize risk—with little more than tape, sticky notes, and Axure prototypes. And the immediate feedback lets us address issues early, before they become showstoppers.

Be transparent, but also teach. Our Lean UX team creates many prototypes and documents, and we share them all using Axure’s Team Projects feature or through Axure Share. This creates a level of transparency that helps inform our colleagues about our process. The problem is that within many enterprises, documents tend to have disproportionate weight. Managers see a prototype, and they assume it’s a product in development when, in fact, it’s just an idea. Most prototypes in a Lean UX process are created to satisfy a learning goal. Understand that goal, and the prototype is better understood. We’ve started naming our prototypes so senior leaders can better understand what we’re doing, but not get overly excited if it’s not a real product. We try to use vocabulary that reflects what the prototype does. A Learning Prototype is for reaching a learning goal. A Vision Prototype externalizes an idea. A Spec Prototype has passed testing and is ready to be built. We don’t want anyone to confuse those types. Part of our job as a Lean UX team within a larger enterprise is also to educate the people around us and be transparent so that our methodologies are more likely to become part of the organizational culture.

If you’re a user experience designer and would like to be part of our team, please visit our CarMax jobs page to learn more about our projects and browse our list of openings.

About Archie Miller

With over 18 years of product design experience, Archie has worked with a host of internationally prominent clients such as American Express Co., General Electric Co., Owens Corning, and Wachovia Corp. (now Wells Fargo & Co.). Most recently, Archie has led UX for online product development for teams at Crutchfield Corp., and Snagajob Inc. He is currently the Team Manager for the Interaction Design Group at CarMax.

Little known fact: Archie was on the development team that created Amtrak’s first website in 1997.