Until recently, I’ve always hand-coded my online surveys, especially those requiring survey logic or special features (snowball sampling, name-generators, etc.).  Recently, I decided to try the LimeSurvey open source survey software (formerly phpsurveyor) as it looked like it had matured sufficiently to meet most of my needs. I am basing my review on LimeSurvey 1.80 RC1 (November 2008), using PHP5 and MySql 4+ hosted on a shared imHosted.com web host account.

I used LimeSurvey for two simultaneous surveys, sent to executives and professionals in two different high-tech industry sectors in Florida to support economic cluster analysis and industry development activities.  Collectively, about four hundred private sector participants responded to the survey using the LimeSurvey interface, accessing the survey via custom forwarded domain names I purchased just for the surveys. Now that the surveys have completed, here are my reactions:

Strengths:

• Skinnable, with a selection of decent out-of-the-box templates.
• Exportable, duplicatable survey structures that can definitely cut down labor required to set up repeat or similar surveys.
• Versatile dashboard for setting quotas, monitoring survey statuses, and exporting results into a variety of formats incluting native Excel spreadsheets.
• Powerful and intuitive “survey logic” controls for setting boolean survey conditions (”…if respondent answers this, then show this question”).
• Versatile survey flow options (question grouping, and display of questions by groups, singly, or in combination).
• Customizable question structures, and an online library of user-contributed question types.
• Save and auto-resume features readily available for users who choose not to complete the survey in one sitting.
• Supports custom “success / thank you” landing pages.
• Automatically generated PDF and “printable” survey versions.

Weaknesses:

• Limited documentation forces you to teach yourself the interface largely through clicking every dashboard icon to see what it does (at least, that is how I ended up teaching myself).
• The “auto save” feature breaks on many corporate computer systems, I presume in networks that control browser cookies, sessions, and private data.  I had many users step away from a partially completed survey, who upon return were asked to provide a login and password - which could not be generated by the survey administrator, and appears to only be generated by respondents who explicitly “save” their partial surveys.
• The question structure customization process is very non-intuitive and not well documented.  I never was able to figure out how to program a two-dimensional x by y table of numerical text inputs (I had to be satisfied with breaking it into x separate questions with y rows each).
• LimeSurvey happily recorded completely empty survey results (although it does designate such responses as “incomplete”).  It would be useful to be able to designate at which point the survey is considered “complete”, or to auto-discard completely empty responses.
• There really aren’t any features for identifying or preventing duplicate responses.
• The automatically generated “printable” and PDF versions don’t resemble each other or the online survey at all, and are not formattable (at least, not through the dashboard, or via CSS that I was aware).  The layout of the PDF version in particular appeared very amateurish (it looked better to create a PDF of the “printable” version).

Of LimeSurvey’s weaknesses, the missing “login/password” problems were the most problematic, and likely to result in incomplete surveys and frustrated survey participants.  It was encountered by at least six or seven of my survey respondents (at least, those that took the time to contact me as survey administrator); I don’t know how many more of the incompletes suffered the same problems.   It’s unacceptable to require a survey participant to ’start over’ a long or detailed survey, without a “lost password” or other interface to help them resume where they left off.  Next time I use it, I am going to take some very explicit precautions to improve the experience of my survey respondents.

Overall, I found LimeSurvey to be a solid and valuable survey tool, and one I intend to utilize again.  It more than met my requirements, and had a lot of advantages over my old methods for most straight-forward survey needs.  It has a few significant flaws, but it is open-source and I’ll bet that they’ll be addressed before too long - perhaps by version 2.0, which is already under development.   I’d strongly recommend the software to anyone with at least a fundamental level of server-side scripting knowledge (installing server-level scripts and databases). Give it a shot!

| Permalink | Trackback