Transcript

Jen

This is The Web Ahead, a weekly conversation about changing technologies and the future of the web. I'm your host, Jen Simmons, and this is episode number 106.

I first want to say thank you so much to today's sponsor, Media Temple. We'll talk more about them later in the show. I also want to thank Pantheon for powering The Web Ahead website. You can check out The Web Ahead website at thewebahead.net. You can check out Pantheon at pantheon.io and learn more about their website management platform. They run Drupal websites — which is what The Web Ahead is — or WordPress websites or... I don't know, maybe they have something else cooking somewhere. Also thanks to CacheFly for delivering the audio file that you are listening to right now, through their fastest, most reliable CDN in the business. They're at cachefly.com.

As you might know, if you're a longtime listener, I've been doing a lot of traveling lately and speaking at many, many conferences. Which I love doing. But one of the benefits and one of the reasons I love doing it is I get to meet the other speakers and see them and suddenly discover people I've never heard of before. I feel like I'm always scouting for the next person to have come on The Web Ahead.

I was at An Event Apart in Washington, DC two weeks ago. Gerry McGovern was there, too. Hi Gerry.

Gerry

[00:01:35] Hello Jen! Thanks for having me.

Jen

You got on stage and — I don't remember what the description was of your talk — but I probably was like, "That sounds interesting. Gerry sounds like a great guy. I want to watch his talk." I sat down and I was doing email or something. Then you started just kicked ass. You were so great. [Laughs] You were just amazing on stage. I don't know when I have laughed so hard at a tech conference event presentation.

You were talking about stuff that was incredibly useful and very interesting to me, through a lens that I had not ever heard of before. And I could feel it resonating with the audience. People were just feeling like, "Wow. This man has brought me something that's actually going to be very useful and helpful when I go back to my..." A lot of the folks in DC work in very bureaucratic, large organizations. Sometimes it can get so frustrating and hard in organizations like that. It just felt like the whole room was like, "This will be helpful. We need this."

So we're going to talk about that stuff today. I still haven't even said what it is, and I'm two minutes in to introducing you. You call this top task management.

Gerry

Yeah.

Jen

It's overall strategy stuff, content stuff, user experience stuff, design stuff. Explain what top task management is.

Gerry

[00:03:08] It's about figuring out what matters most to the customer, then measuring it. Finding a way to measure and manage it. To come up with scenarios around those top tasks and test them with real people and come out of that with metrics that say something like, "We've got a 60% success rate," or "It's taking four minutes to complete this, it should only take a minute."

It begins with establishing the stuff that really matters, and equally, the stuff that doesn't matter. It's often the stuff of low-level importance — what I call the tiny tasks — that takes up most of our week in the job. We're trying to deal with these tiny tasks, which are often political. I often say when a tiny task goes to sleep at night, it dreams of being a top task. There's all this competition within the organization, either to get into the application or get onto the homepage. Often the fiercest competition comes from the least important stuff, from the customer's point of view.

It's a philosophy of managing outcomes. Getting into the world of the customer, finding out what really matters to them, and coming up with a model of measurement that tells you how well what really matters to them is performing.

Jen

I guess this happens more often in larger organizations. The more people who are involved with a web project or website, the harder and harder it gets to make decisions about what's important. It does seem like there's a trend in the industry to use data to help with some of those fights, with those meetings. Where one person says, "I think this" and another person says, "I think this is important." You spend the next four months debating which one is more important. Maybe what gets decided is, this person outranks that person, or this person is that person's boss, or four people think this is important and two people think that is important, so we're going to go with the four people. But that doesn't mean that any of the right decisions are being made.

Gerry

[00:05:22] Exactly. That is the classic problem. People have used the examples of the hippos and seagulls. The highest paid person in the room has the loudest voice. The other analogy is of this seagull, the manager who swoops and poops and flies on and leaves you in the mess. "No, no, no, add that feature." Classical political animals where the most powerful voice gets heard most.

I think if we are to do better work, we need a way of using data in a productive way. But I think it's the data. The customer is the spearpoint of the change. You need to use appropriate customer data. So you can say, "That's interesting. But when we tested, that had a 40% success rate and this had an 80% success rate." We use data of behavior, of what customers actually did.

Not always; some organizations are marred in politics, and no matter what you do, you're not going to get through. But many organizations will listen if you have clear data on actual customer behavior. We need to get that sort of data of what customers actually did when they were trying to complete these really important tasks.

Jen

So how do you do that? There's so many different ways that people sit down. User experience designers are most attuned to this. From the customer's point of view, what is it that they came to your website for? Why do they want to do what they're doing? And we write these things down, and we have these debates, and we put them on post-it notes and order them on the wall.

So there's lots of tools that probably a lot of listeners know of. They're maybe already using them. What is it that you're talking about when you're identifying tasks? What's the methodology that you like to use on this?

Gerry

[00:07:41] The basic methodology uses a lot of what you described in the initial process. The first phase is task identification. There's a big gathering exercise. You're going to go out there and look at search; the annual top 50 or 100 searches. You're going to look at most visited pages. You're going to look at level 1 and level 2of your structure, your feature on your app. You're going to look at competitor websites. You're going to look at previous versions. You're going to look at Google. Google Adwords, I often find that's a good source for how are people searching? Not just on your website, but in general about his problem. You look at social media and communities.

A couple of years ago we did Microsoft Visual Studio development technologies. We found a huge range of tasks on the communities. There were loads of independent communities of developers using these Microsoft technologies. They were saying, "How do you do this?" and "I have problems with this" and "I want to do this."

You draw a very broad net. That's what a lot of good UI teams do. But then, to a degree, it falls down. It gets loads of post-its. But how do you rank them?

I discovered a crazy method, but it actually works. Often, we end up with about 60 or 80 or 100 tasks. IN Cisco, there would be download software configuration pricing, stuff like that. Defining the work of the customer. Broadly, you say, "These 60 or 80 tasks are the world of buying a car, looking after your health," or whatever it might be.

Then I discovered this method. You give the entire list to your target audience — the audience that you really want. Your customer. You ask them to vote. You say to them, "Just choose five from this randomized list." All research studies say, "This can't work. You can't give people a list of 80 tasks and they'll judge logically." It was accidental, how I discovered how this actually worked. It overloads the brain and makes people choose what absolutely matters to them, rather than interesting stuff. They can only choose five. Then they have to vote. They have to give 5 to the most important, then 4, 3, 2, 1.

We've done this over 400 times. Lets say you have 100 tasks. The top five tasks will get about as much of the vote as the bottom 50. You get this "long neck" and "long tail". You get this absolute clarity. This is the essence of what the customer actually wants. You get this ranking of importance. You would have a task that would be at the top.

I can't tell you about this exactly, because it's likely confidential, but we did do it for the European Union last year. We had over 100,000 voters. It was the biggest one we had ever done — 24 languages, 28 countries. The top task got 90,000 votes. The bottom task got about 2,000 votes.

So you get this clarity of importance. We also create a copy of the survey and give it to the team. We give it internally to the stakeholders. We say, "What do you think are the five most important tasks to the customer?" Then we can statistically analyze it: "You think these are really important, but they're not at all." You've got data back. You need over 400 voters. At the European Union, we had 110,000. It was the biggest by far. But if you get about 400 voters, we could have stopped at 500 voters in the European Union and we still would have had the same top tasks.

You get about 500 people to vote. You need a minimum of 200. Really, 400-500. You will get statistically reliable data. We did it for Cisco five years ago. We did it again five years later. Because tasks remain constant over time, we had very little change. It will give you a very clear picture of, "This is the league of important." From the most important to the least important to the customer. That gives you a real foundation for focus and going forward.

Jen

So you're talking about identifying a list of tasks and doing some sort of survey where you throw this giant list at people and say, "Pick five and put them in order." It's crazy, you said 110,000 people surveyed and 90,000 agreed, basically, that this one tasks was their most important?

Gerry

[00:13:05] 110,000 people voted. But they had 15 votes. They could choose five tasks and they get 5, 4, 3, 2, 1. In total, they had 15 votes. There were 1.6 million votes cast in that survey. It was a massive one. But the top two tasks had around 90,000 each. There were about 80 tasks. The number one task had 90,000. The 80th task had 2,000. There was that difference of huge chasm between the top and bottom tasks. The top five tasks got as much of the vote as the bottom 40 or 50.

Often, the top task gets as much of the vote as the bottom 20 tasks. These patterns repeat themselves. Whether we do it in Brazil or Portugal or Iceland or Norway. If there's an environment of behavior — whether dealing with your health or buying a car — there tends to be certain dominant tasks. In health, it's check symptoms, treatment... these really dominant-type core behavior patterns, the defining characteristic of that particular challenge or issue that you have.

I think crucial to design is understanding these top tasks. Then making sure they work. Once you've identified these crucial tasks, the next question is, "How are they performing?" These are things that are extraordinarily important to the customer. How well are they performing? How easy is it for the customer to do them? How long is it taking for the customer?

Step one is to clearly identify what is crucial within this environment. That's the task identification. Step two is what we call the task performance indicator. Which is, how are they performing?"

Jen

In a way, it sounds like it frees up the team to focus on those top 10 or 20 — whatever the things that dominate people's behavior. Why does someone come to your website? They're going to come because of these top 20 tasks. So how's it going? Everyone wants to check symptoms, but that link is actually buried pretty far down on the page because there's all of this information about this fundraiser that they just had for the hospital. Who knows what. The stuff that's important to the business models of the company, but not important to the customer.

Gerry

[00:16:09] Exactly.

One example I gave was the homepage for the Norwegian Cancer Society. They're a nonprofit. They need to raise funding. Logically and naturally, a lot of their homepage and pages on their site were calls to action around, "Please give us money. Please give us donations" etc. When we did the top task analysis, we asked Norwegian people, "What really matters to you when you're dealing with the Norwegian Cancer Society?"

The top tasks were symptoms of cancer, treatment of cancer, where to go. Core questions. At the very bottom of the list was, basically, all about giving money. Anything to do with tax credits or donations or becoming subscribers. Nobody was voting for them. They were quite disappointed because they were saying, "They do want to find out about symptoms. But how are we going to raise money if we don't have all of these calls to action?"

They did something very radical. They absolutely focused on the needs of the customer. They created a new website that not only focused on the symptoms, but it called out the top cancers. It happened there were three or four cancers that were way bigger than the other ones. When you clicked on those, you immediately got the symptoms and treatments. They got rid of all of the calls to actions around donations from most of the website.

The interesting thing is that, year on year, their donations have basically doubled. On the web, we're dealing with intelligent adults who want to be treated with respect and want their problems solved. If you solve their problems, if you make it easy for them to do what they've come to you to do, they tend to think better of you.

People coming to the Norwegian Cancer Society very quickly found symptoms — either for themselves or a loved one — and said, "Wow, this is a great society. I should support this society." Think of it the other way. You come, you're worried about a symptom, and the society is screaming at you, "Give us money before we tell you anything!" [Laughs]

Psychologically, it doesn't work. Everybody understands that as a consumer. Then they go into the organization at 9AM in the morning, and it's a Jekyll and Hyde behavior pattern. They become a totally different type of person when they sit down to design that website. All of the organization-centric needs come pouring out. "No, no, no, we've got to do all of these things."

We need a mechanism that holds that Mr Hyde, organization-centric, "I want the customer to do all of these things for me first, before I'll allow them to do the things they came to do." That My Hyde doesn't work very well on the web these days.

Jen

Yeah. It feels intensely risky. You go into work and say, "I'm a new web designer. I just came to this nonprofit, this health organization. I think we should get rid of all of these calls to action to give money. I'm going to tell everybody that we should get rid of them. I'm going to make a new design for the next version of the website, and I'm going to have that stuff be there, but be very muted. There's just one button that says Donate and that's it."

But what if it goes wrong? What if the donations are cut in half? What if your bosses have to start laying people off because they don't have enough money? You think, "Of course we have to keep all of them. They must be successful. That must be why they're there." It seems like the survey is a good way to start being more objective and less based in fear.

Gerry

[00:20:30] You're absolutely right. One thing I've noticed over the years is that the people within organizations that are the customer champions, who really want to help and are the voice of the customer... it's a career-limiting move. Not only is the fear, but the ego. You create this fear and you hurt the ego. "What do you mean they don't want the video from the CEO saying how wonderful it is that we're visiting the website?" You're sticking a pin in the bubble of the ego.

I've found many customer champions are either sidelined or alienated, and it's very often a career-limiting move. Which is a tragic situation, because these are the change agents. These are the people who are trying to save the organization for the future, in this new world of this impatient, skeptical, cynical-type of citizen.

Going forward, we cannot go up against fear and ego with our opinion. We must bring data.

The next step is, let's say you've done your top task and you see its treatment, or whatever it is. You say, "OK, let's not change anything for the moment. But we do recognize that symptoms are extremely important to our customers." Everyone says, "Yeah, we recognize that. But we've got to do all of these other things." OK, great. Let's just see how well symptoms are working.

The next stage is, now you've got a consensus. These are the top tasks. But you still have all of these people saying, "Yeah, but these are still important. I know they didn't get a big vote, but they're still important." The next thing to drive the change is to say, "Let's forget about all of these other ones for the moment. Let's look at not changing the website, but look at the current performance of these critical things." That can be a huge surprise.

With Cisco, the top task was downloading software. Very unsexy. The top tasks are usually not set, they're not exciting. "Download software! We've had that for 25 years." They're often the old, kludgy, basics mechanics of the world. So we went out and tested. What we do is more a time and motion study. It's like usability, except a step forward. Traditional usability is more about identifying something: "That's a real problem." So you go back and say, "We need to fix that." If you test with about eight people and carefully frame the test, you can identify core underlying problems.

We've found over the years is that it's not enough to say, "That's a real problem." It doesn't work as a management metric. We found that if you go out and test between 15-20 people in a time and motion study. Come up with a carefully framed task.

Cisco was downloading software. We would come up with a task like, "Download the latest firmware for the RV042 router." We'd come up with very specific examples of the major areas of software download within Cisco. We carefully selected 15-20 of their customers who are mainly typically engineers who are downloading this type of software. We'd give them the task in a remote setting, because we find remote is less disruptive. We want to get as close to the natural behavior pattern as is possible. Literally, then shut up. We paste it into GoToMeeting or WebEx or whatever. Get them to start on a blank tab or wherever they want. Then just shut up. When you shut up and let them just do it, patterns emerge. Out of those patterns, you can come back and say, "We've got a 60% success rate on this task. It's taking five times longer than it should." You begin to get metrics. If you don't fix that problem, and you test it again in six months, you've still got a 60% success rate. It's still taking five times as long.

This begins to create a management model. It's not just saying, "There's a problem there." It's saying, "That's a 40% failure rate. We've got 300,000 people downloading every month." You're bringing to another level of a metric that is repeatable and a number. Managers love numbers. We've got to replace the old numbers of, how many visitors, all of those old, terrible numbers that actually encourage worse practices. Time on page, hits, how idiots track success. What I call the cult of volume. We've got to replace the cult of volume — which is measuring inputs and crude metrics — to these outcome-based metrics that are founded in the world of the customer.

Jen

I go to epson.com every two or three years, every time I upgrade my computer's operating system and download new scanner software. Doesn't Epson want me to be on their website longer, spend more time there and click on more pages? "Look, this person was on the website for five minutes. That's great. That's better than if she was on the website for two minutes. Look, she clicked on 17 pages while she was here, instead of two pages. That's great. We want people to spend more time on our site and be here for longer."

Gerry

[00:27:11] It's so crazy, isn't it?

Back in the 90s, I remember seeing something happen. We were working in companies and we heard, "The CEO is going to give a talk and he's going to mention the web!" We want to say something positive. So we all said, "We'd better give the CEO something positive to say about the web. We looked around and looked through web trends and we saw this humongous number. This absolutely humongous number. We said, "Wow, that's a huge number. What is it?" We looked across and it said "hits". So we said, "Yeah! Tell him hits!" Because that was the biggest number we could find. [Jen laughs]

It was the biggest number we could find. We were desperate to impress. We found the biggest number we could find. It was like am arms race from there. How many hits do you have? How many pages do you have?

When I was in DC, I visited the US Department of Health. They told me they had 200,000 pages. They deleted 150,000 of them. And nobody noticed.

Jen

That boggles my mind.

Gerry

[00:28:44] But you know what, Jen? In four years, they'll be back with another 200,000 pages. Because the culture — and they're trying to change it — but the culture is, "I'm important. Therefore, I must publish. I must create pages. I must create sub-sites." It really hasn't changed. At a senior level, we still measure our success in the amount we produce. The amount of content we produce. The amount of pages we produce. The amount of features we produce. We're dying to get hits and visitors and search engine optimization.

I went up to someone and said, "That product is out of date. You don't sell that product anymore." They said, "No, but we can't take it off the website." "Why not?" "Because if we take it off the website, our SEO will drop." That's counter-productive. Fill the website with all of this crap because it might appear in Google.

At the Norwegian Cancer Society, they went from 5,000 to 1,000 pages. Their visitors went up by 200-300%. Much less pages, better managed, better focused, really focusing on what people need. You'll actually get more visitors. More of those crude metrics, but they'll mean something. Because you're focused on the right thing: helping people figure out their symptoms. When you focus there, all of the other metrics will look after themselves. Time on page, number of page views. It's so crude and counter-productive. We've got to get away from it.

Jen

It's really interesting to think about. For many of us who were part of the web in the 90s, there was this explosive rate of growth. In 1993, hardly anyone was on the web. Around '94-'95 — or really '95, '96, '97, '98 — all of the people who were "computer people" — the nerds, the people who liked to use their computer — those people all got online. Then, slowly, more and more and more, until everyday people were there. I'm sure that growth graph was staggered for different countries. Other countries hit at different places in the world at different times.

There was a period where you could say, "I put a video online and I had 150,000 people watch it." It was like, "WHAT?! WOW! That's so many people!" Now 150,000 is like, "Eh."

To have look at something and say, "Here's it's 150,000 and here it was 400,000 and here it was 2 million and here it was 20 million," says something about the growth of the web. And it says something about the success of your project. That's a crazy, amazing, exciting thing to have seen.

But you're right. If we keep thinking that way about every one of our projects, in this day and age, where in some places in the world, the number of people coming online is fairly stabilized. We're not seeing the rocket growth we were seeing 10 years ago. In some ways, our own understanding of metrics needs to change because the nature of the web and the speed at which people are getting online has changed.

Gerry

You're right. It's not that these metrics are not important. That the number of visitors isn't important. But we need a different context for understanding them.

Jen

They're not the most important, probably.

Gerry

[00:32:38] No. But if you pursue them and go after volume, you start doing a lot of bad things that will end up hurting your volume.

I've found in many situations, we end up deleting up to 80-90% of the website's content. Then all sorts of good things start happening. It's counter-intuitive. The core metrics are, your site is there to solve problems and help people to do things. For the vast majority of websites, that's the reason they actually exist. People use the website for a reason. To book a flight or figure out a problem.

Look at health. Look at ebola. You solve ebola. The ebola pages are going to drop. Is that a bad thing? [Jen laughs] Do we need another pandemic so the World Health Organization web team will start feeling good about that? [Jen laughs] "Nobody loves us. Last month we had 15 million visitors, now we only have two million visitors. What are we going to do? We need a new ebola." [Jen laughs] You're a support website. Bring out even crappier products? You're going to have lots of visitors.

A lot of this stuff is insidious. It encourages bloat and negative thinking. We need to think positively. If volume is appropriate, volume will come. But we shouldn't be pursuing volume as the objective in and of itself. Unless we're some media website and we're desperate. Do you see what they do? They break their articles into two pages or do stupid photo galleries in the hunger to get more page views. It's often a race to the bottom.

Jen

Yeah, yeah. Ok, let me jump in with our sponsor today. Then I want to go back to talking about the way you're envisioning this user testing.

So, I like this practice that you've invented, that you've been doing for years. You could do this over IRC or Skype. It sounds like you don't need fancy technology. You just need a way to communicate. Do you have the other person screenshare?

Gerry

Yes.

Jen

So you can watch their screen?

Gerry

[00:39:15] Yeah. Skype is improving in screensharing. It's not expensive. What you really need is to develop the ability to observe patterns. You're watching people.

As I said, you get about 15-20 and you get a number coming out of that. If you don't change the download software environment and test 15-20 people in three or four months, you'll get the same 60% failure rate or whatever.

Jen

You want to do it very scientifically, right? You want to say, "Let's come up with a very specific question. I'm going to introduce it in exactly the same way. I'm going to say this sentence, and I'm going to set them all up in exactly the same way. They're all going to start from a blank tab. Or they're all going to start from the homepage. Or they're all going to start from wherever." Like a scientist.

Gerry

[00:40:17] Totally. That's crucial. This is not a casual, "let's come up with a task." You have to spend a lot of effort framing the task in a very specific way, with very clear outcomes. The more precise it is, the more clearly singular the answer is.

We had the OECD a couple of years ago. Which is a big organization looking at how countries perform. How does Germany compare with France on unemployment rates? One of the top tasks was comparing a country's statistical data, because statisticians and policy makers were coming to the website. A task example we came up with was, "Did more people die of heart attacks in Canada than in France in 2004?" There's a very clear answer to that. Everybody has to start from exactly the same place. You want to get people in the same environment as much as possible. No talk allowed. You're trying to get their natural behavior.

You can use WebEx or GoToMeeting or any good screensharing service. You need to see their screen and you need to hear them. That's all. You record it. Then you look for the patterns. Obviously, the basic pattern is they say "yes" or "no". You have a clear answer. They have to give you an answer.

Jen

Oh, right. Then you can run a stopwatch. You can say, "This is exactly when they started and this is exactly when they accomplished that task."

Gerry

[00:42:09] Exactly. We have little tricks of saying, "Can you please tell me when you're starting the task?" Even when people tell you they're starting, you still have to watch for the cursor movement. So you begin to learn from when people normally actually are starting the task. Then they have to tell you the answer. "Yes, this is the software I download," or "This is the configuration guide," or "The answer is 27." There has to be a clear, specific answer.

We have to agree with the client on a target time. Knowing that it took five minutes... well, if it was a very complex task, that could be ok. If it was a really simple task, it should have only taken 30 seconds. You have to agree on a target time.

Jen

Do you do that ahead of time? Do you have a meeting and discuss it? How do you do that?

Gerry

[00:43:08] It's a combination of things. Let's say you're changing your password. You're doing a troubleshooting-type task and you want to change your password. We look at a number of factors. We look at, what's the current environment like? How do your peers do it? What's the best practice? Then we wait until all of the testing is over. How fast was the fastest person? You get a number of inputs, then you come to an agreement. You say, 45 or 30 seconds. It takes quite a bit of discussion to get the target time, and quite a bit of analytics to get an agreement.

Then you have a base: 90% of people are taking five minutes and the target time is 30 seconds. That's the key, Jen. We say to the team, "When we started testing download software in Cisco, it was taking 15 steps on average and about 300 seconds." We had a way of managing the team. The team used to be responsible for uploading software. They'd say, "I've uploaded the latest firmware. Job done." Now they're measured on the engineer's ability to download it.

The questions the manager was asking are, "How do we get it from 300 seconds down to 45 seconds? How do we get it from 15 steps down to 4 or 5 steps?" Today, for much of the software in Cisco, it has been reduced from 15 steps to 4 steps. From 300 seconds to about 45 seconds. But the reason that happened was we were managing the outcome. The team became focused on the customer, not on the input. Not on uploading the software. They became focused on the customer downloading the software. That's the bridge you need to cross. Where your metrics are focused on the outcomes of your customers, rather than the inputs of your organization.

Jen

What seems so brilliant about this is that is gets pretty obvious. In this example, I can imagine the meeting where they said, "Wow, it takes 15 steps and 300 seconds? Ok, I have ideas of how we can get that down. We can do this one thing and that one thing and that saved 20 seconds and that saved 10 seconds and that saved three seconds and that added four seconds. Ok, let's not do that." Right? You can find your way. Then your designers have ideas about what to do. Your developers have ideas about what to do. Your business folks have ideas about what to do. And good ideas. Instead of it being this crazy town, where everybody has other ideas and some C-level executive, like you said, swoops in, poops on the situation and says, "We need to da-da-da-da-da." And you're like, "I know that's going to make it worse." But you can't articulate what is going to be worse.

Gerry

If they're really important, you can say, "We'll make that change..."

Jen

"... and we'll measure it.

Gerry

[00:46:27] Exactly. What you just said, Jen, that's the new language. Of course, you'll do stuff, and it won't quite work. Sometimes, you'll solve one problem, and create another one. But you're constantly iterating and you've got your metrics.

There are three things we measure. Two I've mentioned: success rate and time on task. The third thing we measure is what we call the disaster rate. That's where people think they have the right answer but they have the wrong answer. At the end of every task, we also ask people about their confidence. "How confident are you in this answer?" If they say, "Yeah... but I have very little confidence." Particularly in a critical area, like checking symptoms or downloading software, and they say, "This is the version I download." That could have a major implication if they tried to install that in the system. Not every question has a disaster, but for some questions, if you get it wrong.

Once we were doing testing for Microsoft embedded in an operating system. It was something like, "Can you clone this part of the operating system?" The answer was no. It was so important that they put it in a big note in a big box on the page. All of the engineers found the right page and every one of them said the answer was yes. Because they didn't read the box. We find that in a lot of situations. Often the more you try and get the attention of somebody, all of the techniques are used to get attention.

We did a training task recently in a large website. This company had launched a brand new training course. It was really important, strategically. They put it in a huge banner on the training homepage. They had thousands of courses. They said they put two banners: a massive banner that took up about 1/3 of the page, and another big banner in the right hand column.

We tested with people. We said, "Find this course and find the sign up page for Course X." Course X was screaming at them. You know these nice executives in suits, holding laptops in their hands, saying, "Join Course X because it will improve your career," blahdy blah. All of that hero shot-y stuff. Not one person in all of tests clicked on either of the banners. We could show that to the senior executives. "Look at this stuff. It's not working."

You can take in the seagull's or hippo's opinion and say, "But look! That actually dropped the success rate by 5 percentage points."

Jen

Right. Or if you have the ability to A/B test, then you can A/B test stuff. Or if someone comes in and insists that you launch something, launch it and measure it afterwards. Then you can have stats that show very clearly these things that are important to the customer are worse, so we have to come up with a different strategy.

Gerry

[00:50:11] Absolutely. A/B testing and stuff like that is a way. What we test is the entire journey, to give a model of configuring a product. Checking to see when your bin is collected, if it's a local council. But A/B or other types of testing could be there to say, "There's all these problems that we discovered, so let's attack them one at a time. Then we can do A/B or other types of testing to attack certain points of the task."

You don't have to test the entire task. You can really zero in on a particular page and just A/B test it. You can do these discrete elements of testing with different techniques because you're only doing the overall task testing every six months or whatever longer period. But then you have these more precision-based testing techniques, like A/B testing, to look at just step three of that process, and see, can we simplify it?

Jen

Our industry has been talking a lot about performance, especially developers. How do you get page load speed down? Maybe we should put all your CSS in one file instead of multiple files. Or maybe break it out; some of it only loads on the pages where it's needed and loads inline immediately. The CSS that's most important. Just over and over. All of these techniques and ideas. How do we get the images to be smaller? Let's invent a whole new HTML element so we can have faster images. Let's look at mobile. Let's look at advertisements.

All of that focus is on getting webpages loaded for users much faster. Super important. A lot of us spend a big part of our time online in something like Twitter or maybe Facebook or maybe a website with lots of links. You click on the link and load that one article and read it and close the page and click on another link and look at that one page and read it. When that page loads slowly or is jumping around, it's really annoying.

But what you're describing is, the times when we're going to a website to do something. Like, I'm going to go on a big camping trip this fall. I want to research backpacks and shoes and figure out that sleeping bag I should buy. I'm going to be floating around the web trying to figure that out. Look, here's a website by a company who's hired you to come in and help with consulting. How is their website going to help me with that task or not help me with that task.

We don't measure that performance. The performance of the person trying to download software. The performance of the person trying to look up their cancer symptoms. You said it took 15 steps? Fifteen pages needed to load on this software website, making them load 20% faster isn't really going to speed up the fact that they had to load 15 of them. Only loading four pages is actually a much faster way to speed up that experience.

Gerry

[00:53:32] Exactly. As you said, fast loading pages, absolutely critical. It's foundational in the process. But if you load a page quickly that's really confusing, then it takes 35 seconds. If you bring a page down from three seconds to one second, wonderful. But if it takes 40 seconds to figure out which link I should click...

We need to measure the performance of the human. The performance of the page is an influence in the performance of the human. But it's the human who is on the page. It's the customer. It's the person trying to sign up for a training course, etc. The page load, absolutely critical. But it tends to be a relatively small. When we say it takes an average of three minutes, 45 seconds for this task to be completed, I would probably say a maximum of 20 seconds is page loading. Maybe 20-40 seconds. The three minutes is trying to figure out confusing menus and links or going to unnecessary pages. The loading of the page, while critical, is a relatively small percentage of the overall quantity of time that a person tends to take to complete their task efficiently.

Jen

It fascinating to think about. If a person is surfing around, clicking on things in the menu, trying to find the right thing and they can't find the right thing and they're trying to find the right thing. If the pages that they're going to are the wrong pages. Loading in 1/3 of a second. That's a far superior experience than if those pages are taking 2.5 or 3.5 seconds to load.

But the fact that they have to click and click and click and click on a whole bunch of pages is what's really hard. If instead, they just clicked and clicked and were already there, those pages could take two seconds to load. It would be better if they took a third of a millisecond. I would rather click on two pages that each take two seconds to load, than to click on 40 pages that take 1/3 of a second to load.

Gerry

[00:56:04] Absolutely. What we have found is that it's actually the menus and links that are the time wasters. They're the performance drags. Year in, year out. Confusing menus and links. Badly named. You go into a tech website and you see a link for "knowledge base". Is the rest "idiot dump"? You see a link for "resources" or "tools".

When we were testing the download software, one of the links that they were supposed to click on was called "tools and utilities". The engineers were saying, "I'm trying to download the firmware but where do I go next?" They were supposed to click on "tools and utilities".

These things get put up quickly without thought. Then they're just left there. Because we never measure the performance of the human. We're measuring the performance of the server and all of these other technical things. We need to measure the performance of the person who's actually using it. That's the most important performance.

There's a concept of perceived speed. Sometimes you can be too quick. People don't have the visual cues. That's quite rare. But there's a concept of perceived speed; I feel like I'm moving through the process quite quickly. That's more of a psychological design type of thing that you can manage.

But the performance of the person... that's customer experience.

Jen

I think you're absolutely right. Maybe when the site was originally designed, a lot of care went into thinking about what the words were going to be in the menus. Although, maybe not. Sometimes people are designing and are so focused on whether they want to use a 12 or 16 column grid or whether this box is going to take up four or six columns. "We'll just type some words temporarily into the menu bar. And we'll put a bunch of lorem ipsum in for content. Then later, of course, it's clear we're going to switch out the lorem ipsum."

But maybe no one goes back and says, "Oh, right! I only put 90 seconds of thought into which word to use," or "I only thought about that for one hour. I didn't have any conversations. We didn't have a meeting about that. That was supposed to be temporary. But we haven't come up with anything better." Or maybe it was thought through well at the beginning of the website, but then someone adds a section. Somebody else adds a section and they didn't notice that these words were nouns and they added a verb. Or vice versa.

Why did you call it "tools and resources"? Well, because this other thing was called "ideas". So it sounded similar. You're right, that ends up being a huge part of the actual experience of the customer.

Gerry

[00:59:33] We did a huge IT company intranet. They have two big divisions at this company. The products division and services division. The services division is growing like mad and delivering a lot of the profitability. There's a lot of competition between the services division and products division. They don't like each other.

So, on the intranet, they said, "We have to have a product section. We have to have a services section. Because we're politically powerful." So we started testing tasks. Where would you get a spare part for this machine? We asked the services people, because the top tasks were around services as well as products. We said, "Give us a typical services task connected with these top tasks." We went to the products people and said, "Give us some typical product type of tasks." When we tested them with the people that worked for the company, "Where would you find the spare part?" The services people said, "Everyone's going to click on services." But actually 30% of people clicked on products. Again and again when we were testing, we found there was a confusion — a normal confusion — in the company about what is a product and what is a service. Many people thought a service was a product. Because they sold services. There wasn't clarity in the organization among the staff about what a product and what a service was.

When they saw the data come back, they said, "Wow. That's interesting." In the next iteration they said, "We'd better come together. Our sales reps don't understand the difference." Even the service engineers were clicking on products. In the next iteration of the architecture, it was "products and services." We said, let's subdivide ourselves off, if necessary, at a lower level. But we can't divide ourselves off at a higher level. All we do is create absolute confusion if we have a separate link called "products" and a separate link called "services."

That sort of crucial stuff. People said to me, "You will never get those two groups to operate together." But when we brought the data back to them, they said, "Ok."

I know I harp a lot on menus and links. But those navigational paths are so critical. The words that you choose can have a massive impact on success or failure rates. As you say, Jen, they rarely get the attention they deserve.

Jen

Do you see them frequently mirror the structure of the organization?

Gerry

[01:02:43] Totally. Another company added a link called "resources." What's "resources" for? "We've got a video team and they liked to call themselves 'resources.' I know it doesn't work, but that's the structure." If you go into most intranets or technology websites, it's called a "knowledge base" because it's documentable or something like that. The navigational architecture is named based on the internal content management systems that they have.

It's not about troubleshooting. They have to come up with names. They've got all of these silo-based data or information. It's either architected based on the type of technology they have internally — because they've got all of these separated silos based on documents — or SharePoints that have been separated and don't interact.

Look at Dell. You go to Dell and they ask you, "Are you a home user? Are you a government user? Are you a business user?" [Jen laughs] We don't give a shit. For years, they know — and everybody knows — you want to buy a laptop. You want to buy a desktop. When people see these things, it makes them suspicious.

Jen

Yeah. Am I a home user or a business user? I am running a business but I'm doing it at my home. What do you mean "business"? What if it's a small business? Do you mean enterprise? I don't know what you mean. [Laughs]

Gerry

That's it as well. But not just that. You're thinking, "Did they get a better deal?"

Jen

Yeah, I'm thinking, "What's the difference between the two?" Are there different computers in one section and not the other? Which one should I say I am?

Gerry

[01:04:50] Exactly. It's not good. If you look at the good, you start with, "I want to buy an iPad." You start with the thing. You figure out the rest later.

The reason that exists is because that's the Dell business units. They couldn't agree how to share revenue if somebody just came in through "laptop." So they said, "We'll force them."

Many times, as you get older as an organization, you become arthritic. You have all of these fiefdoms and their silos. They say, "No. We must have business. We must have government. We must have home. Because these are our competing business units. They don't even like each other. We're going to force the customer through paths that reflect how we are organized."

That used to work, 25 years ago. Used to work maybe 30 years ago. Sorry, doesn't work today with the new customer. The empowered customer. The customer who can circumvent you and go straight to Google and start searching on the Google search results pages or go to Cnet or go wherever they want to go. They want to buy a laptop.

If you're testing, you can show the cost of the organizational arthritis.

Jen

What kind of successes have you seen with people using this methodology?

Gerry

[01:06:33] A lot of our work has been with Cisco over the years. A lot areas in their support websites have gone up by an average of 20 percentage points in many of the key tasks. Particularly in the support areas. But not only in support; in many other areas.

A lot of the sites with a radical simplification of the environment, like I said, with the Norwegian Cancer Society. We work with a lot of councils now, particularly in the UK, where there's a common architecture beginning to emerge around bin connection, leisure facilities. As we did more and more of these analyses, we said, "All of the councils are basically the same." When people live in a city of 100,000 people, it's the same basics things they need to do. You begin to see common architectures. We see these not only appearing in the UK, but appearing in Holland and other council and municipality marketplaces. We see it happening in a lot of spaces. We begin to see common architectures, where there's common types of tasks that are beginning to emerge.

IBM has used it in a number of their product areas to prioritize what really matters to customers in buying their products. The same with Microsoft. The same with NetApp. They use it to make decisions about areas that they're going to focus on and areas they're not going to focus on going forward.

Jen

How do you think this applies to folks who are working on much smaller projects? They're working at a nonprofit that's got five employees. Or working on a client website that's under $100,000. We've been talking about some huge corporations.

Gerry

[01:08:48] I wrote an article on it for A List Apart that has a lot of comments on it. Most of my work has been in large organizations. But a number of people came back and said, "I have a small web agency and I've been using this for years and it does work." It can work.

If you've got a very small environment, the core purpose of it is when you've got a lot of competing ideas. You might have a small website but you still have a lot of competing ideas of what should be prioritized. We've found, if you can get 100 people to vote, you begin to get useful data coming to you. You need to get at least 100 people. Even a small website. What is the constrainer of it? You could have a small website, but it's very complex. It could be more complex than a bigger organization, because there's a lot of demand. But you could equally have a small website. It's a no-brainer what your top task is. If there's no argumentation about what your top task is, then it's less useful. If there's a lot of arguments about, "No, this is important. This is important." You need some way to cut through that argumentation. That's what the method is for. It tends to be more useful in more complicated, more political environments.

If you're worried and you're not sure, as a small business, why people are coming to your website, go off and brainstorm with your team of two or three. Look at the search logs or whatever. You might get 30 or 40 things. It might even be 70 or 80. Ask 100 people of your customers to vote. It will still give you clarity.

Jen

Cool. Anything else you want people to know?

Gerry

[01:10:55] One of the other interesting things that we've found is, there's a little bit of fear at the beginning. When people say, "My god, I wrote the content for that page. If they don't understand it, what am I going to do? I created this." People are afraid. "I'd rather not find out," some people almost say. They're afraid of that first set of test results. But once you get through that fear factor, it's actually very liberating. You say, "Ah! Look, we changed the link and look at the success rate."

Oftentimes, we exist in a vacuum of the work that we do. We create a set of graphics, we create an architecture, we create content. But we never actually get feedback on whether it worked. I've had teams break into spontaneous applause, saying, "Well done guys!" It's very satisfying after a period. You're managing an outcome of the customer. You can show, "Look, these patients can now check their symptoms much faster, much more accurately than they used to. That content you rewrote really made a difference."

That sort of feedback is very professionally satisfying. To know that you did something that actually did make a difference. A lot of the times, we do things and we're not quite sure what difference it made.

If you manage from the world of your customer, you figure out what it is that they really want to do, and you measure examples of those over time. This is the thing: over time. It's not a one-of process. Every six months, we retest "download the latest firmware for the RV042 router." You keep testing the same thing and you can see improvement or disimprovement over time. It's very satisfying to make changes and see improvement.

Jen

It is so weird that we sit at our desks, hovered over these glass rectangles. We click on a bunch of stuff. Then we send something out into the world. When we look back at our glass rectangles, some robot tells us that one million people or 400,000 people or 17 million people were affected by what we did. But we have no objective human way to believe that, right? [Laughs] You didn't see 17,000 or 17 million people stop by your store. It seems like a myth.

We're human beings doing work and we get no real world, natural world feedback. Maybe it's because of that vacuum that we end up with ego and crazy ideas and seagulls and hippos and this tension of meetings where no one can agree.

It's different than when you rearrange the front of the store and more people walk into your store. You know that worked. You rearranged the front of your store and no one comes in to your store, you quickly rearrange the front of your store again.

Gerry

[01:14:28] You put your finger on it there, Jen.

We've got the metrics that says 60% success rate. But then we try and create videos. We find where the biggest failure was. It was on step 7. We say, "John, an engineer from Minneapolis, step 7. Show him at step 7 for 60 seconds. Show him clicking on the wrong link." Then we go to Mary, product designer from Dublin. At step 7, she clicks on the same wrong link. Then somebody else. We pull these little videos together. Usually no more than three minutes for a top task. We say, "60% success rate. Let me show you this video. This was the core reason." You see three people failing. That can have more of an impact. The penny drops, so to speak. You're exactly right; we forget these are real human people who are on these pages.

We once saw on a tax website, one of the busiest period was after 9:00 at night. Why was that? Kids were in bed. People were doing their personal taxes. A hard day's work. We could get that sort of empathy: here's three tired parents who have just put their kids to bed, failing on your website. [Jen laughs]

But we don't get that, Jen. We have hits. We have users. Three things in common: users, traffic and hits. We lose touch with the humanity of our customers.

Touch points are disappearing. Touch points are oxymorons. Touch points are replacing touch points. Touch points are where the customer touches the screen but you don't touch the customer any more. That customer touching the screen used to be one of your staff talking to the customer. Either physically or on the phone. Touch points are oxymorons. Organizations are closing themselves off from contact with their customers.

There's this massive darkness that's descending. There's good reasons; we're saving money, self-service, etc. But we can't close down all the views we have with our customer. Then we're in the dark. We don't know our customer anymore. As a result of that, customers are much less loyal. They're switching. They're much more ruthless with you. Because you have no empathy for them and they have no empathy for you.

Jen

Yeah. That's a whole other show to talk about. [Both laugh]

I hope people have found this helpful. I hope people go off and try this out.

Gerry

Thanks, Jen, for having me. I hope it's useful. The A List Apart article is called Top Task Management. That's a good overview, certainly at the first stage of it.

Jen

I'll put that in the show notes for the show, which are at thewebahead.net/106. It's good because there are pictures in here, which we did not include in the audio podcast. [Laughs]

If people want to find you, you have a website: gerrymcgovern.com.

Gerry

Yeah, they can get me at that website.

Jen

You've written five books?

Gerry

Five books, yeah. The last one is basically a manual of how to do this. It's called The Stranger's Long Neck. I love talking about these things but I don't promote often. [Laughs] The Stranger's Long Neck is basically a how to. How do you do task identification? How do you do task measurement? It's a real manual of the dos and don'ts of how you actually do this.

Jen

Your company, Customer Care Words, is customercarewords.com. They could hire you, if they've got a big enterprise company. It sounds like that's what you're doing. You're running around to big enterprise companies and doing this.

Gerry

And we have partners in six countries — user experience companies that implement these methods. We support a partner network in Norway, Sweden, Holland, Belgium and Canada. And to a degree, in the US. Also, for the very large projects, with the Ciscos or IBMs, we do those ourselves.

Jen

Nice. Thank you so much for coming on the show.

Gerry

Thank you for having me, Jen. If you're interested, I've got a newsletter, which I've been doing since 1996. I think it's great that there are so many people. We just put ideas out there and hopefully they're useful. Hopefully they help people do their jobs better and make for a better experience on the web.

Jen

Yeah, because it's hard enough already. I like that we totally help each other as much as we possibly can. It makes it better. If I need to download some Cisco software, I'm going to be glad that you were there before me and made it easier for me to do that.

Gerry

Let's hope.

Jen

If people want to follow The Web Ahead on Twitter, you can do so at @thewebahead. Please review and rate the show in iTunes. That will be super helpful. If you want to see both me and Gerry McGovern speak, we'll both be at An Event Apart in Chicago next week. Well, we might be on stage by the time this show comes out. [Laughs] But if people want to find out other places that I'm speaking and come to a conference that I'll be at, you can go to jensimmons.com and there's a list of events that are upcoming. I'm constantly adding more.

Thanks to Media Temple for sponsoring the show. I think that's it. Again, show notes at thewebahead.net/106. Until next time. Thank you for listening.