Posts filed under 'Research'
In mid-July I submitted the final version of my thesis to the library, and am now entitled to call myself Dr. Chawner. I don’t feel any different, though, and apart from the satisfaction of having completed (after far too long) a major piece of research, don’t expect my life to change. I’ve sent the abstract of my research to the people I interviewed, and to the discussion lists I sent invitations to complete the web-based survey to. In addition, I thought it might be a good idea to post a summary here, along with a few additional comments.
The purpose of this research was to identify factors that affect participants’ satisfaction with their experience of a free/libre open source software (FLOSS) project. The research built on existing models of user satisfaction from the information systems literature, and also incorporated two characteristics of FLOSS projects first identified by Ye, Nakakoji, Yamamoto, and Kishida (2005), product openness and process openness. The central research question it answered was, What factors influence participant satisfaction with a free/libre and open source application software project?
Richard Stallman’s reasons for setting up the GNU project and the Free Software Foundation arose from his frustration at being forced to be a passive user of software used for a Xerox printer. These suggest that being able to be an active participant in a FLOSS project is one factor that should be examined, and therefore the first sub-question this project answers is, What types of contributions do participants make to free/libre and open source software projects?
Several studies have shown that the extent of participation in a FLOSS project varies from individual to individual, and this variation leads to the second sub-question, Do the factors that influence satisfaction vary for different types of participation? If so, in what way?
A preliminary conceptual model of factors affecting participant satisfaction was developed, reflecting the key concepts identified in the literature. The main theoretical goal of this research was to test the model using empirical data.
The research used a sequential, mixed methods approach. The first, qualitative stage involved reviewing documents from selected projects and interviewing a purposive sample of FLOSS project participants. The second, quantitative stage involved an online survey of FLOSS project participants, and the data gathered were used to test the conceptual model.
The results of the first stage showed that participation in FLOSS projects was a more complex construct than previously reported in the literature. Seven categories of activities were identified:
- interaction with code;
- supporting the community;
- management; and
I illustrated the relationships between the categories in a diagram that places ‘use’ at the centre (where I think it belongs, since ’software is for use’, to paraphrase Ranganathan’s first law of library science), and groups the other six categories in three pairs. There is more detail about why I used this grouping in the full thesis. One of the reasons I was interested in identifying different types of participation, and how they relate to each other, is that I feel that the onion-style model many people use to discuss what happens in a FLOSS community (which usually places code at the centre) overemphasises the importance of coding, and diminishes the importance of the user. I don’t mean to imply that coding isn’t important, but I think that other aspects of particpation are also important.
Four attributes that modified these categories were also identified: organisational focus, role formality, remuneration, and time commitment.
Data from 154 responses to the online survey were used to test the model using stepwise multiple regression, which determined the effect of each of the variables on overall participant satisfaction. Moderated regression analysis was used to test the effects of three potential moderating variables. The results showed that that perceived system complexity had the largest effect, decreasing satisfaction if respondents perceived that the software was complex, while process openness and perceived developer communication quality accounted for the most variance in satisfaction. The final conceptual model, based on the data, is below.
Overall, the model explains 44% of the variance in satisfaction; the β values show the standardised effect of each predictor variable on satisfaction.
The main theoretical contribution of this research lies in its extension of satisfaction studies to FLOSS communities, showing that communication and openness are more important than in conventional software projects. Its practical contribution will help people involved in the management and governance of FLOSS projects to identify ways of increasing their participants’ satisfaction, which may in turn encourage them to contribute more.
The final version of the thesis is available in the VUW library’s research archive from:
Ye, Y., Nakakoji, K., Yamamoto, H., and Kishida, K. (2005). The co-evolution of systems and communities in free and open source software development. In Koch, S., editor, Free/open source software development, pages 59–82. Idea Group, Hershey, PA.
I’d welcome comments on the results here (while comments are open), or by email to me at email@example.com.
October 11th, 2011
Somehow ten months have gone by since I last posted. It’s been a busy year. I was the LIM Programmes Director until the end of June, and then was busy teaching and organising a national speaking tour for RMS, which finished with him in Christchurch, where he was a keynote speaker at the LIANZA annual conference. I’m really glad I took on the role of RMS tour organiser, but it was pretty intense at times, and definitely interfered with PhD progress.
Nonetheless,I’ve been chipping away at my research, looking at the data I so painfully converted into something usable, and am now on Research and Study Leave until the end of October 2010. It should come as no surprise to anyone that my top priority is getting my results written up as soon as possible, and then moving on to turn the PHD into two (or maybe more) journal articles. Watch this space for more regular updates.
November 16th, 2009
Somehow I never seem to have much to say about my research project, but that’s going to change over the next 6 months. I need to have the thesis finished by June 2009, which means that my priority for the first half of 2009 is going to be analysing the data and writing the thesis.
I was very pleased with the response to the online survey: I received 206 responses, from all over the world! Most of them look usable, and many more individual free/open source projects were represented than I expected, and one of the things I’ll do here is mention some of the ones that were new to me. The range of responses means that the data is more ‘random’ in some ways than I expected, which is probably a good thing. I’ll start summarising my findings here once I start working with the data.
One comment: the open source software I used to gather the data is from an abandoned project, and hindsight now tells me that I shouldn’t have relied on it, even though our systems administrator told me that other people had used it for big surveys. I used it for a shorter survey in 2007, and I thought that I understood its quirks well enough to use it for this one. Unfortunately, that wasn’t true, and I’ve spend the last two days converting the data from its rather idiosyncratic (and verbose) XML output to something I can work with more easily. The biggest challenge was the fact the it only output data for the questions that people had answered, so I had to identify the missing values before loading it into an anlysis tool. This was complicated by the order of the results, which were not sorted by question number, but were instead presented in the order I had defined them. But I have finally gotten the data in the form I want, through a combination of XSLT, a text editor, and a short php program. I’ve done enough double checking to be confident that everything’s there, and correct (whew!).
If I do something on this scale again, I’ll probably use LimeSurvey, which looks good and seems to have a very supportive community.
January 1st, 2009
This very short entry is just to record the TinyURL for the survey: http://tinyurl.com/4wgce7. I plan to print it on some small cards and hand them out at the LIANZA conference if anyone I talk to is a free/open source software user and is interested in completing the survey.
October 20th, 2008
Yesterday morning I issued the invitation to complete the survey that will gather the bulk of the data for my research. It’s surprisingly hard to stop tweaking and send it out for ‘the world’ to view.
I wouldn’t be surprised to get some feedback about the distribution method: I sent it to a number of project and library-related email discussion lists, rather than collecting email addresses and sending out individual invitations. Is it OK to send a message to (potentially) thousands of people, only a small fraction of whom might be interested in the topic of the survey? Of course, I think that my topic is so important that everyone who gets the invitation should respond, but I realise that’s not the case. So why use a list?
First, I have a problem with large-scale harvesting of email addresses from discussion lists. It seems to me that most people don’t post messages in order to broadcast their email address, and that this type of harvesting is dubious from an ethical perspective.
Second, and perhaps more importantly, I wanted to offer people who lurk on the lists an opportunity to complete the survey. There is no way of knowing who they are, and I suspect that their contributions will be very important to bring out a perspective that we usually don’t see in the list conversations. Time will tell how successful this will be.
I suppose I should also post the link to the survey here, just in case anyone is reading this and hasn’t received a copy of the invitation. If you use or are otherwise involved with a library/information management free/open source software project, and would like to contribute to research into factors that influence participant satisfaction with the software, you’ll find it at:
The survey will be available until 14 November 2008.
October 17th, 2008
I am nearly ready to invite people to complete the online survey for Stage 2 of this project (yay!). I started designing the questionnaire a week ago, had the first draft ready on Monday, and spent the rest of the week asking people from different backgrounds to give me feedback on the questions.
It’s now at the stage where I don’t think it’s possible to make any more changes—if I tweak the questions any more, it will start going downhill. Today I wrote a draft of the invitation email, and completed the Human Ethics Committee application. Assuming all goes well, the invitations should be sent by the end of the week. Then I will have the pleasure of seeing how many responses I get. While I’m waiting, I’ll experiment with the trial data, and set up an analysis template.
I have to confess that it does feel good to be getting somewhere at last. The current approach seems to be working, at least for now.
October 12th, 2008
Despite my good intentions in June, life got in the way of my research again (sigh). However, the good news is that for the last few months, I have indeed been spending at least “15 minutes a day” on my research, and have finally managed to make some progress. I am currently finalising the questionnaire for the next stage of my research. I probably won’t be able to issue it for a couple of weeks, since I need to find a few people to use as pilot testers, make any changes they identify, and apply for Human Ethics approval. But once that’s done, it will be ‘all systems go’.
This time I am feeling much more motivated to move on and get this project finished. I have even written the first sentence of my thesis, and prepared a mental outline of the first few chapters. As long as I can keep finding a few minutes a day to think/plan/read, and spend some time each the weekend writing, I should be able to continue to make progress.
October 8th, 2008
I didn’t mean to disappear for quite such a long time. It’s been a busier year than I expected, and my research project has always taken a back seat to other things I felt I had to do. Things need to change, and so today is day 1 of a new regime. I am going to try the ‘15 minutes a day’ approach, where I need to keep myself moving, by doing something, no matter how small, to move my project along. The idea is that I will always identify the ‘next step’ and record it here, so that I always know what I will be doing next.
Today I am sorting through my transcripts and updating the records of my interviews. I thought I had been very good about recordkeeping, but when I looked at the spreadsheet, I could tell exactly where things started getting on top of me.
It does feel good to be getting things underway, even if it has taken too long to get started again. Watch this space for
tomorrow’s the next update.
June 11th, 2008
… which is why I’ve been quiet this month. Nonetheless I reached a milestone with my research today. I received the final response to the email interviews! Thank you to everyone who has participated.
Now I can get out my coloured pens and start the analysis. This should be fun.
March 14th, 2007
Today I started reading Barry Schwartz’s The Paradox of Choice. On page 3 of the prologue, he talks about the difference between ‘freedom from’ and ‘freedom to’. Schwartz came up with these phrases based on Isaiah Berlin’s essay “Two Concepts of Liberty”, published in 1969.
I immediately wondered how this applies to open source projects, and what I’ve been hearing in the various interviews I’ve been doing over the last few months. I’ve thought for a while that ‘freedom’ is an undervalued concept in the open source movement. It’s unfortunate that many people first think of cost when they hear term ‘free software’, hence the widely used advice to ‘think free kitten, not free beer’
The FSF’s four software freedoms are all ‘freedoms to’, and they definitely represent the developer perspective. Some of the people at the ‘user’ end of the spectrum have focused more on ‘freedoms from’. Three of them are:
- freedom from license fees;
- freedom from compulsory upgrades; and
- freedom from vendor lock-in.
Since I finished the last face-to-face interview transcript yesterday, and am just waiting for two of the email interviews to be completed (I’ll send out follow up messages again tomorrow), I’m finally ready to start analysing the data. I’m looking forward to seeing what additional ‘freedoms from’ and ‘freedoms to’ I find, if any.
I’m still mulling over how Ranganathan’s second law, ‘Every person his or her book’ can be applied in the context of open source software. Watch this space for my conclusion. I also have a movie/DVD catch up post to write. We’ve recently seen Borat, Pride and Prejudice (the 2005 movie), A Prairie Home Companion, and The Corpse Bride. Eclectic is probably the only word to describe the list.
March 1st, 2007