Putting Web Site Quality and Accessibility into Context - Purpose and Objectives of Websites

Why do organizations build Web sites? What do they hope to achieve with a Web site? In the early days of the Web, it was considered bold to spend money on building a Web site, but over time, this evolved to the point where it was extremely important to have a Web site — even if purely for the sake of being able to say you had one, that you were seen as “keeping up with the Joneses.” Frankly, it was embarrassing not to have one! Finally, by now we have all calmed down, and there are actual, clear business objectives for Web sites giving us good reasons to build and maintain them. Because of this, we can actually measure if a Web site has achieved the purpose for which it was built, that is, if the Web site is successful. But what does success have to do with quality? Strictly speaking, a Web site could be successful but be considered of low quality when tested against XHTML compliancy. A site could be considered to have low usability but be extremely popular and therefore successful in terms of the purpose of the site but considered low quality against industry standards.

The important point to understand is that quality isn’t objective and shouldn’t be seen as objective. Quality is contextual. A “quality” meal for someone who is suffering from malnutrition is going to be very different from a “quality” meal for a restaurant critic. If you look at the WebQual standards referred to by Eleanor in the previous article, you’ll see that even though they are set standards, their relative importance can change over time. In 2004, response time was more important than it is now; on the other hand, trustworthiness has increased in importance. When looking at the quality of a Web site, we should therefore first consider the purpose of the Web site and then evaluate what quality factors are directly related to that purpose.

Main Purpose

When asked to define the main purpose of a Web site, respondents in our survey selected two purposes well above the rest: to provide information to current and potential clients (30%) and to promote and market the organization (27%). These two purposes accounted for more than the rest combined (see Graph 9). What this tells us is that the Web is considered primarily a means of communication, first to inform and second to promote.

Graph 9 - Main purpose of website

These results suggest that for a Web site, getting a message across is the most important thing it can do. If this is the case, then what does it matter if a Web site does or does not strictly comply to XHTML coding standards? Does it matter if the site is readable? Even if it does not adhere to one of the few real standards, a Web site could still fulfill its main purpose. In fact, taking this point further, if the current and potential clients of an organization do not include people with disabilities, a Web site that fails to adhere to accessibility standards could actually still achieve its main purpose. The difficulty with such a statement though is that, as referred to later, we may believe our target market doesn’t include people with disabilities, but we really can’t be sure unless we test this assumption. The results of our survey show that this is one area we as an industry do very poorly. The point still remains that we need to be sure what the purpose of the site is in order to establish what standards need to be met.

From a commercial perspective, money spent to implement a level of quality that does not achieve a business goal makes little sense (unless there is a legal requirement to do so). Therefore, a successful Web site and a quality Web site go hand in hand. The success of a Web site will depend on it achieving its purpose. The quality of the Web site will depend on that purpose. To simply apply standards against a Web site without considering the context will provide a misguided result.

Targets Defined

It’s one thing to retrospectively decide the main purpose of a Web site by choosing from a generic list, it’s another to be specific about the targets of the Web site up front: to define the targets before the Web site is built and to then assess the Web site against those targets. This is where the level of success — and therefore the true quality of the outcome — can be better measured.The question is whether organizations are clear about the targets.

The survey results are not very inspiring with only 4% of respondents stating that there are precise targets defined before a Web site is built (see Graph 10). This is staggering! This means that for only one in 20 Web sites built does the organization actually take the time and effort to define what targets it hopes to achieve before it builds its Web site. On the surface, this is a crazy result. It says that most organizations don’t bother to think about what they are doing before they do it. There are many reasons for this; the one that I come across the most is lack of effort. The people in charge of getting the Web site built simply fail to make the effort to work out exactly what they hope to achieve. And the reason why is twofold: (1) it’s hard, and (2) it results in accountability. If you don’t set a target, you can’t fail. But, going back to the first reason, it is much harder than it sounds to set meaningful and measurable targets. In doing so, you would be required to ask lots of questions about the organization and its objectives before you can work out how a Web site may help to achieve those objectives. It means dealing with different departments in an organization since a Web site will represent many elements of the organization. It means you need to get people to make decisions as well as make decisions yourself that you’ll be measured against.

Graph 10 - Did you define targets for your website?

On the other hand, if you don’t make the effort, you can build a site without having to worry about all of those tricky questions and risk being seen as a failure should the targets not be met. This is a cynical view but as a project manager on many projects, I’ve asked my clients these sorts of questions, and very few are willing to answer them or find the answers; it’s much easier to stick with vague objectives than have strict, clear targets. Let me give you two examples. The first is a client that has a number of online stores selling specialized cycling equipment and accessories. I am in the process of building a new online store that will enable the client to have multiple stores, each with its own design and domain name — all to channel through to a single shopping cart and payment gateway. We have some stats as to the number of sales that the current sites have, and the numbers are not very high. When I query the client as to what it wants from the new online store, I don’t get a clear answer. The client has high ambitions but won’t set a target other than “more than the revenue it is currently getting.” This makes it very hard to make decisions regarding what to prioritize in the build of the store. There’s a big difference between an online store that has to accommodate 100 sales per month and 1,000 sales per month. The number of customers to be handled, the type of reporting, and the issues regarding load and backups all impact how the solution is architected, designed, built, and managed — all of which relate to how good the outcome is. In order to make decisions, the architect and I make assumptions on what we think is the best solution for a given issue in the absence of clear targets.

The second example is a client who just launched a Web site that enables users to rate universities in Australia.1 We had a tight deadline due to a marketing launch that had been organized and couldn’t be moved. I wrote the project brief and then got the architect to review the technical issues. The first question he asked me was how many ratings the client expected and hoped to get in a 12-month period. I asked the client, and we worked out that there were approximately 650,000 university students who would be eligible to give a rating per year. Of this, we calculated that if we could get 1% of students to rate, that would lead to approximately 6,500 ratings a year. This we agreed was ambitious so the client took a conservative approach and stated a target of 4,000 for the first 12 months. This clarity helps in many ways. It means we know the sort of traffic we will need to accommodate, the type of load the site could be under, the number of results that we could have on a ratings page, the types of marketing efforts that will be needed to get 4,000 ratings, the types of features the Web site will need to help to attract more ratings. Everything becomes clearer with a set target that is measurable.

It’s clear that many organizations are willing to invest time and money in a project despite not knowing what they hope to achieve or how they intend to measure it. I struggle to think of more than a handful of projects that I’ve worked on where my client gave me clear targets to achieve. For the most part, if there were targets, they were vague and open to interpretation. For example, a recent project I worked on had the following objective:

to promote greater levels of philanthropy in the community.”

It’s a noble goal but a very difficult one to measure. Perhaps if the number of donations increased over a period of time, then we could say the objective was achieved, but how do we prove this? Perhaps if the Web site had a donations facility and over 12 months it received $50,000 more over the previous year in donations, then maybe you could say it increased levels of philanthropy, but perhaps that just diverted donations from one place to another. Unless the targets are clear and focused, it’s not possible to accurately state if a Web site is successful. And, by consequence, any measures used to define quality will be equally subject to the vagueness of the success of the Web site. In short, without clear and defined targets, we can’t measure the success of a Web site and therefore can’t state the level of quality of the Web site.

Targets Achieved

Defining targets at the start of a project is clearly important in being able to measure the success of the Web site. The second part is to check if the targets have been met. The survey results show that not a single site “precisely” met the targets set. The data indicates that 56% of respondents met targets “for the most part,” and 40% met targets “somewhat” (see Graph 11). So, over all, this is a very ordinary result: not a single response that a Web site met, let alone exceeded, targets. The response is that we “mostly” got there.

Graph 11 - Has your website met the targets it was designed for?

This does not bode well for our industry. We fail miserably to set targets and then if we do, we only achieve them for the most part. You could be excused for saying that as an industry we are still stumbling around in the dark. It appears that we don’t really know what we are doing given we rarely bother to decide up front exactly what we hope to achieve. We seem to make it up as we go along and hope for the best, sometimes checking to see if we achieved what we thought we were trying to do. I believe we do this because we aren’t really sure of what we are doing or why. We fail to take the time to think things through up front. This is because setting targets up front is difficult; we have to make clear decisions and be accountable for them, which can be daunting. But what I fail to understand is how we can even begin to talk about quality when we are unable to, as a whole, go about setting targets let alone measuring against those targets.

From the service provider point of view, I have been asked to build many, many Web sites. I have rarely been given a precise target to achieve and have never been asked to measure that target later. If I get paid for the work done, should I care? Perhaps if I have ethics or morals I might care, and, in turn, being of moral fiber, that might bring me more work. In reality, it doesn’t seem to matter. The measures of whether the end result is Successful or of high quality are extremely vague and subjective, as subjective as whether the CEO’s son liked the look of it. I don’t wish to be a doomsayer but I can’t pretend that we can even begin to consider measures of quality when we are failing to define clear and measurable targets.