We have written about the advantages of modifying questions or the selections to them while a survey is still in the field. If that constitutes the only changes you make, your cleanup tasks will not be burdened. However, if you drop a question from the survey or add a question, your program code will become more convoluted.
One reason for this additional burden arises if you rename your variables to short names (doing so makes code easier to write and improves the look of axis titles on graphs, among other advantages).
One of the methodological risks that a compensation survey particularly confronts is that respondents don’t enter correct information. It is possible, for example, that a Chief Operating Officer deliberately enters inflated values where the survey asks for salary or bonus. They may nurture a hope that if they and others put in numbers that are higher than what is true, the resulting averages and medians in the report will be too high.
During the past six months I have been conducting a survey of law firm Chief Operating Officers, Executive Directors and equivalent roles. The outreach invited hundreds of them to take part in a quick, no-cost survey of compensation – a price, investment of time, and topic that should attract nearly everyone. Don’t we all want to know whether we are paid “fairly.”
But the response rate has been smallish. Why might that be?
Here is the Table of Contents of the draft Report that we plan to send to all participants. The send date will be during the last week of August. We plan to publish an updated report, with data from respondents between late August and mid-October, shortly after the October closing date.
From the 129 respondents to date (with considerably more expected in the next few weeks), here are quartile figures for their total base and bonus compensation.
Many designers of surveys like questions that ask respondents to indicate their opinion by choosing from a scale. Likert scale questions (named after their creator, American social scientist Rensis Likert) are popular because they are one of the most reliable ways to measure opinions, perceptions, and behaviors. Such topics can’t be understood with one question. Instead, you collect data on multiple indicators — questions that help you understand the concept you’re trying to measure.