A System for Maintaining an Online Community
Comments: 4 - Date: May 11th, 2009 - Categories: Social Networking, Social Tools, Workplace
Tomorrow, at the ACM Group conference, Rosta Farzan, PhD is going to be presenting a paper on the work we did together last summer.
R Farzan, JM DiMicco, B Brownholtz. (2009) “Spreading the Honey: A System for Maintaining an Online Community.” Full Paper, Proceedings of the ACM GROUP Conference, May 2009.
Last summer, when Beehive had been running for a full year, it had plenty of content — 100,000 pieces of content, in fact. So we realized the problem on the site was not generating new content, but rather finding the existing, interesting content. This problem is usually tackled in a few different ways: by displaying lists of recent content and most-viewed content (which we already did on Beehive) and by asking users to rate or vote on the best content.
We decided to design a custom system that encouraged a larger group of users to participate in the process of rating content than one usually sees in standard rating systems. We did this by picking a rotating board of users that has the power for one week to give “honey” to content they liked. Each board is picked based on their activity on the site and you can’t serve on the board more than once every four weeks.
We feel strongly that having a diverse group of users involved in selecting the best content brings a richness and diversity to the promoted content that reflects more of the IBM community. Because the Beehive community is large (>50,000) and IBM is even larger (>300,000), we didn’t want to have a small, and in some ways elite, group of enthusiastic raters driving up the visibility of a small set of content. Rather, we wanted to have the power to promote content distributed over a larger group, over a longer period of time.
To find out more about the system and, IMHO, impressive results, read the paper! The screenshot to the right is what you see on the home page of Beehive every time you log in and it shows you the content that this week’s “honey bees” picked as the best of the best.
Comment by James Morris - 12 May 2009 @ 12:25 am
I would love to see an experiment in which the thumb-up, thumb-down buttons that readers push are used to rate not just the articles but also *earlier* raters. This information could be used to (1) find raters I usually agree with and (2) somehow reward early raters that many people agree with. My fantasy is that good former newspaper editors can support themselves by reading and rating stuff they are expert on as it emerges. Good former editors-in-chief can support themselves by reading and rating the stuff the good former editors endorsed, etc.
Comment by Jake - 12 May 2009 @ 12:41 am
Neat paper, I just read it. Before I got through the first page, I thought about Flickr’s Explore page, featuring what they call the most “interesting” photos of the day, http://www.flickr.com/explore/interesting/2009/05/11/
When the service was introduced years ago, it was a revelation. And while it was a completely automated “secret” algorithm, before long the rankings were being gamed by users far & wide (search for silly sheep to see one fine example). Most users tolerated this, as most of the photos seemed to be deserving if a little narrow (lots of self-portraits featuring attractive women, sunsets, HDR, heavily processed photos). Lots more little history there.
But recently, it seems one group of users has perfected the gaming of the system, whereby up to 85% of the day’s 20 featured photos ( http://www.flickr.com/explore/ ) came from one small group of self-selected users. This angers a few and annoys many, as being on Explore is a nice way to make new contacts and interact with new people. Others were angry as they tried really hard to get onto Explore and couldn’t. But most were bored, and just stopped paying attention to Explore. Except, Flickr highlights Explore photos everywhere, such as on the mobile web app and many Google widgets. So now the photos that Flickr goes out of their way to highlight each day are repetitive and from a narrow group of users. Which, in the long run, seems to be counter to Flickr’s best interests.
I don’t really have a point, but thought you might be interested in this “problem. Here’s a long discussion about the recent, organized gaming of the system, http://www.flickr.com/groups/central/discuss/72157612926060266/ .
Comment by Joan DiMicco - 12 May 2009 @ 12:41 pm
@James Morris
Hi Dad, the system you’re describing is a bit like what slashdot has where a rotating group of people is given the power to rate the comments on posts. Those comments than get pushed up in the list. Cliff Lampe and Paul Resnick wrote a paper about it Lampe, C. and Resnick, P., Slash(dot) and burn: distributed moderation in a large online conversation space. in Conference on Human Factors in Computing Systems (CHI), (Vienna, Austria, 2004), ACM Press, 543-550.
Comment by Joan DiMicco - 12 May 2009 @ 12:43 pm
@Jake, thanks for reading the paper! In the previous work we did with rewarding points we had a lot of gaming of the system and then a lot of annoyed/angry users. So that is interesting to hear that Flickr has had gaming of something that seems hard to game, and that has also led to some annoyance. For whatever reason, hopefully because of how we designed it, we haven’t had gaming with the honey bees and, most importantly, we haven’t had a group of upset or angry users complaining about the system. It has been running for a year with mostly no trouble.