Ideally, once you export to a spreadsheet the data that has been collected by your hosting software, you will read it into the scripting language of your choice and carry out every step of cleaning the data. You will remove disqualified responses, standardize titles and educational levels or other demographic questions that have an option for “Other,” correct incorrect answers (such as a compensation figure obviously missing a zero), and perhaps add calculated variables such as total compensation when you’ve asked for base and bonus or convert foreign currencies to a standard currency.
We continue here to discuss NLP techniques for extracting meaning from free-form text comments. The first post in the series is here while the second post is here.
Calculate Word Associations
Correlation is a statistical technique that can tell how strongly distributions of number resemble each other. For example, the correlation between the number of lawyers in a firm and the average hourly billing rate is strong; as one rises, the other tends to rise.
The first part of this series explains typical steps that a survey analyst takes to prepare free-text comments for Natural Language Processing (NLP). In this part, we describe four methods by which a cleaned and standardized corpus lets NLP reveal insights.
Create a Word Cloud of Common Words
From the document-term matrix, software can total how many times each word appears in the corpus as a whole (all of the text questions combined text).
If a law firm or law department decides to pursue an online survey, the project team will need to select a software vendor. Using the software and hosting services of that vendor, a team member or consultant will create the questionnaire form that is sent, as a URL link, to invitees. In this early step of the project, the team might enlist assistance from the information technology department when they evaluate survey software.
In the appendix to the report on your survey findings, your law firm or law department has the opportunity to say “thank you” to those who helped and “my bad” for any shortcomings you recognize. Many wheels must turn in a survey project, with lots of cogs and gears; an acknowledgement, therefore, can call out a range of contributors. Based on my familiarity with many law-related surveys, I have listed below the most common items in acknowledgements.