Collab-Or-Ate

While at CHI last month (our international conference on human-computer interaction (HCI)), I went to two panels (”Celebrating ‘The Psychology of Human-Computer Interaction’” and “Usability Evaluation Considered Harmful?“) that had really interesting discussions about what defines our research conference (CHI) and our field of study (HCI). I’m still synthesizing my thoughts around these panels and what I’ve been reading since, but based on them, here is how I think about the HCI field today:

1. The HCI field (and the CHI conference) began in the 1980’s with a strong grounding in computer science and cognitive science. Card, Newell, and Moran’s The Psychology of Human-Computer Interaction is considered the seminal textbook describing this approach.
2. Computer science and cognitive science guide us towards taking a systematic, scientific approach to building and evaluating software (for e.g. GOMS). This is a solid way to build systems and many of the early successful HCI research projects utilized this approach.
3. As the software and consumer electronics industries exploded over the last two decades, it has become obvious that there is something more going on here driving user adoption, in addition to computer science innovation and cognitive science usability. You could summarize this as “design” or “context” or “the third paradigm.” However you describe it, it has to do with human emotions, social dynamics and desire.
4. As Greenberg pointed out in his paper presentation, evaluating an early prototype in a systematic way, particularly in terms of usability, can kill the innovation process. Early design often gets things wrong, but it is a critical stage in the product innovation cycle and should not be stunted through rigorous evaluation. He claims that inappropriate evaluation is harming the quality of the work presented at CHI — read Greenberg and Buxton’s paper for more details.
5. The CHI community is struggling to find an identity that simultaneously supports a scientific process (so that there is a criteria for judging quality) and product innovation (so that CHI has an influence over the technology world, outside of academics).
6. The paper The Three Paradigms of HCI (Harrison, S. Tatar, D. and Sengers, P.) tries to define exactly what this “third” thing is that is missing from our traditional HCI education, calling it the “phenomenological matrix.” Research practices this third paradigm include are ethnography, action research, practice-based research, and interaction analysis, where the “goal is to grapple with the full complexity around the system.”
7. Because I’ve been working within the space of design, social psychology, and “context” for so long, this approach to building technology seems so logical, yet surprisingly hard to justify to CHI paper reviewers. But on the other hand, my response should not be to reject the CHI’s body of work as misguided or uninformed. I think a rejection of stringent evaluation techniques should not lead to a rejection of the innovations that have been born out of this structure.
8. My conclusion from this is that I should read more, spending time becoming more aware of and inspired by the work done before us. I’m all in favor of coming up with alternative evaluation methods or no evaluation criteria so that we can foster risky, exciting ideas within HCI. But I don’t want to abandon all the early work’s ideas.

Some Recommended Readings:


The Psychology of Human-Computer Interaction, Stuart K. Card, Thomas P. Moran, Allen Newell

Twenty-five years ago, Card, Moran and Newell’s book, “The Psychology of Human-Computer Interaction”, named our field and launched us into a new world of user-centered design and development. These pioneers believed that “a scientific psychology should help us in arranging [the human-computer] interface so it is easy, efficient, error-free – even enjoyable.”

Saul Greenberg & Bill Buxton’s paper “Usability Evaluation Considered Harmful (Some of the Time).“

Current practice in Human Computer Interaction as encouraged by educational institutes, academic review processes, and institutions with usability groups advocate usability evaluation as a critical part of every design process. This is for good reason: usability evaluation has a significant role to play when conditions warrant it. Yet evaluation can be ineffective and even harmful if naively done ‘by rule’ rather than ‘by thought’. If done during early stage design, it can mute creative ideas that do not conform to current interface norms. If done to test radical innovations, the many interface issues that would likely arise from an immature technology can quash what could have been an inspired vision. If done to validate an academic prototype, it may incorrectly suggest a design’s scientific worthiness rather than offer a meaningful critique of how it would be adopted and used in everyday practice. If done without regard to how cultures adopt technology over time, then today’s reluctant reactions by users will forestall tomorrow’s eager acceptance. The choice of evaluation methodology - if any - must arise from and be appropriate for the actual problem or research question under consideration.

The Three Paradigms of HCI, S Harrison, D Tatar, P Sengers

Informal histories of HCI commonly document two major intellectual waves that have formed the field: the first orienting from engineering/human factors with its focus on optimizing man-machine fit, and the second stemming from cognitive science, with an increased emphasis on theory and on what is happening not only in the computer but, simultaneously, in the human mind. In this paper, we document underlying forces that constitute a third wave in HCI and suggest systemic consequences for the CHI community. We provisionally name this the ‘phenomenological matrix’. In the course of creating technologies such as ubiquitous computing, visualization, affective and educational technology, a variety of approaches are addressing issues that are bad fits to prior paradigms, ranging from embodiment to situated meaning to values and social issues. We demonstrate the underlying unity of these approaches, and document how they suggest the centrality of currently marginal criteria for design, evaluation, appreciation, and developmental methodology in CHI work.

Thomas Erickson, David W. McDonald’s new book, HCI Remixed: Reflections on Works That Have Influenced the HCI Community

From Tom Erickson’s web page:

The goal of the HCI Remixed project is to produce a collection of essays in which researchers and practitioners reflect on a paper or other piece of work by someone else, that is at least 10 years old, and that has had a personal impact on their view of or approach to HCI.