Sponsors of external surveys wish the whole world would clamor for their report and find it easily. The report is a marketing tool and the wider the dissemination the better. “Feel free to forward this report to your colleagues and peers!” They want to harvest emails and phone numbers from a multitude of folks who might want their services.
By contrast, reports based on surveys on compensation or proprietary information like total legal spending are intended only for the companies and firms that gave data.
My enthusiasm for vivid graphics that convey a survey’s finding knows no bounds. If the reader’s eye immediately grasps the take-away of an attractive plot, the sponsor has done well. If, on the other hand, a plot causes confusion, delay, and page turning, that effort has come to naught.
With that ideology made clear, let’s critique the plot below, which comes from the 2023 Annual Litigation Trends Survey Report by Norton Rose Fulbright (pg.
A negatively phrased question one year followed the next year by the inverse positively phrased would mitigate the echo-chamber effect we have discussed elsewhere. “What are the biggest obstacles in your firm to knowledge management?” followed the next year with “What are the biggest incentives in your firm for knowledge management?” The technique of what I call “paused inverses” in a single survey also nails down key opinions and helps spot respondents who misread or miss-applied instructions.
For most surveys, the questionnaire develops as the sponsor thinks of information or opinions they would like to collect. Only when the sprawl of brainstormed questions peters out does the project team step back and think about the flow of the questions in terms of their logic, priorities, time demands, and effort needed.
My vote goes to starting with easy questions, such as demographics. I am against delaying demographics, such as gender, to the end.
Many survey sponsors ask respondents to rank a set of selections, such as to allocate 1 to 7 for seven selections. The average of each selection’s rankings becomes findings in the report, often visualized as a bar or column chart (a bubble chart offers an alternative perspective). The charts usually display the results in declining average rank order. But often whoever prepares the report fails to focus on whether the gaps between the averages are roughly equal.