Recently, I was called to document quality control / quality assurance steps for a public relations / communications research proposal. I thought it was a useful topic, and worth summarizing some of what I’ve learned over 15 years of scientific research administration and working as a PR / marketing research consultant. In no particular order:

1. Documentation

One of the most important quality control categories involves documentation during the research process. There are different tools and strategies for managing documentation.

• Project management / mapping tools. Business project management tools are used by most engineers and large businesses for very good reasons. GANTT charts, PERT and critical path charts help to document the progress and potential sources of error and difficulty in a research process, and help to delineate expectations and responsibilities. Once created however, their potential is only manifested if they are tracked and updated.
• Internal WIKIs and blogs are becoming increasingly used to foster document sharing and collaborative knowledge building.
• Meta notes - my term for the “field journal”, the lead researcher’s ongoing journal of events, progress, and most importantly, observiations, questions, thoughts and insights that are generated during the research process. Generally, these are private documents, but can be critical for consistently documenting, preserving, and fostering potential analytical insights which can develop throughout the research.
• Research plan. An important, but sometimes understated or overlooked, component is a documented research plan that states the data collection and analytical methodologies to be used, the expected outcomes of each method, the objectives of the research, and what form the findings will take. Inevitably, a project that involves applied, “real world” research will have to be adjusted “on the fly”, but it helps to work from a documented baseline. The technical / research plan can usually be incorporated into the proposal or statement of work.

2. Communication and Collaboration

The scientific research process has a wide body of literature oriented toward improving outcomes in the collaborative research process, particularly when responsibilities are divided across organizational and stakeholder boundaries. The lessons are relevant to marketing, public relations, and organizational research as well.

• Champions. It is necessary to clearly identify, and get commitments from, champions within each key organization that have a vested interest in the successful outcome of the research. It is also important to ensure that these champions have the blessing of their superiors, resources and time necessary to follow through, and an identified backup / replacement if they are removed from the project for any reason.
• Regular communication schedules. Champions and representatives from key stakeholders must meet on a regularly established schedule to review progress and discuss development. The meetings should be frequent enough to not be formalities and to foster relaxed relationships between the participants, but not so frequent as to be an excessive burden or to interfere with performance.
• Milestone reviews. Periodic reviews with key stakeholders, champions, and clients should be organized to review progress toward milestones, and discuss if modifications need to be made to schedules and research plans. However, this does not mean that preliminary research findings need to be shared.
• Managing client expectations. The most critical part of a research process often begins prior to the research itself - a focus group with clients and client stakeholders to reveal potential obstacles, political barriers, expectations, and a careful understanding of how the research findings will be utilized.

3. Transparency

Another key quality control category speaks to transparency of the research process, the research methodologies, and the activities, framework and guidelines from which the researchers operate.

• Transparent methodology. It’s important to ensure that no proprietary or undocumented methodologies are used during a research process. Some consultants utilize a “black box” or proprietary research process, and claim that the proof is in the results. However, the proof is also inevitably unverifiable in such cases. It’s always preferable to use research methods that are open to examination, replication, discussion or critique.
• Estimations of confidence and error. Any research methodologies that seeks to make observations about large populations (extrapolations from census or survey research) should include estimations of statistical confidence and error intervals, and statistical significance measures where appropriate.
• Review of vested interest and bias. Although academia has long discarded the mythical archetype of the truly objective researcher, the lay public still clings to the idea of the impartial, objective journalist or scientist. Every individual has a vested interest or bias in certain outcomes prior to research onset, and it is the responsibility of the researchers to examine their own motives prior to initiation and make any significant or relevant motives “up front” to stakeholders. Although researchers may resist such a step, consumers of research are often suspicious about potential bias and may dismiss credibility of findings purely upon such suspicions. Researchers may actually benefit more by being up-front about what the end-user will already suspect, thereby earning a degree of trust.

4. Multimodality

It’s almost always preferable to use multiple, complimentary research methods to research the same subject matter, where such is possible. When results from interviews, random sample surveys, and focus groups all point to and support the same general findings, the researcher can be confident about the analysis.

5. Peer Review / Second Opinions

It never hurts to have a second expert not otherwise directly involved in the process review the research plans, methodologies, and findings for overall soundness and critique.

6. Compliance and standards

• Published and academically accepted standards / methodologies. One of the most solid guarantees of quality is to document a peer-reviewed or academic standard, and adhere to it during the research process. It helps to frame the research findings, and make the potential strengths and weaknesses of the method available for discussion. Where less-standard methods are utilized, it helps to document the justification, advantages and disadvantages of the alternative method.
• ISO standards. There is an international ISO standard for marketing and public opinion research - ISO20252. It has broader acceptance in the EU, and is generally adopted by larger organizations. Like all ISO standards, it is at its base a documentation, certification and business process standard.
• Professional standards. Most professional and academic organizations have an adopted standard of ethics and behavior to which all members are expected to embody. These standards are not intended to speak to process quality assurance, but are relevant as they speak to maintaining a standard of diligence and performance by the people who perform that research.

| Permalink | Trackback