In our first blog post on intranet surveys, we discussed the importance of an intranet evaluation survey, and how it is a key tool in gauging how your employees feel about your intranet.
Now we are going to take it one step further to help you design and launch an intranet survey to ultimately measure your social intranet satisfaction.
Applying an evaluation survey like the one below on a recurring basis will provide comparable data to show change over time. In addition to documenting improvements, the recurring evaluations can show the strengths and weaknesses of your intranet and trends in how it is being used and can provide guidance on how and where your intranet team can focus its efforts.
Step 1: Designing your social intranet satisfaction survey
Understanding your intranet’s purpose
Before we get into the specifics of which questions you want to ask on your survey, you need to first identify your intranet’s purpose. This may seem obvious, but it is often overlooked. Your intranet’s purpose can help raise the project above departmental politics and provides guidance on priorities. It should also link to your company’s missions, vision, values, and strategy.
The intranet purpose statement should clarify your intranet’s core goals, for which you want to measure impact. To do that you can align the survey questions with the intranet purpose statement.
Most social intranets aim to:
- improve communication
- improve access to information
- improve collaboration
- increase connections and knowledge sharing
- streamline specific processes and
- improve the sense of company community.
These, or whatever elements make up your intranet’s purpose, are the factors the intranet satisfaction survey should aim to measure and should be at the core of the survey questions.
Remember that the survey aims to measure how employees feel about the intranet. You don’t need feedback on every feature of the intranet, but rather on its general impact. Use questions that link to the intranet’s value proposition while keeping the survey short to respect colleagues’ schedules.
Understand who the users are (demographics)
It’s good to know where your employees are located within the company. If you’re using a third-party survey tool, you will want to start the survey with a set of demographic questions. These can be questions about location, job level, department or any other factors relevant to your company. Keep the demographic questions short and easy to answer.
If you’re using ThoughtFarmer FormFlow, user profile information is automatically collected as part of the survey questions.
Should your survey be anonymous? In some organizations, some users may prefer to provide anonymous feedback. However, this misses out on opportunities to follow up with users if an addressable issue is raised. Generally speaking, we recommend intranet surveys to not be anonymous, however, reported survey results should be anonymized.
Regardless of which option you choose, you should make it clear to employees whether or not the information they provide will be personally identifiable and what you will be doing with it.
Keep the survey short
The core of the survey should be 4-6 questions with consistent rating scales (this doesn’t include the demographic questions). It’s helpful to ask an additional open-ended question at the end to capture specific comments and feedback. This last question may be the one you spend the most time analyzing as it provides mostly qualitative data that’s harder to measure and compare over time.
Be consistent in your format
Use a consistent rating scale for core questions. We suggest a five-point or four-point rating scale for the main questions. Come up with five answers for each question that fall along a scale from bad to good, negative to positive.
The below is an example survey for an organization that prioritizes intranet usage, and overall design and hierarchy. If your intranet placed a higher priority on improving knowledge sharing or streamlining processes, you would want to ask more questions about things like how employees share information, and less about their overall satisfaction with their intranet.
Note: If you’re using ThoughtFarmer FormFlow, you can install this sample survey directly on your intranet by downloading it from our community site.
- Demographic data
- In what Division do you work?
- Sales & Marketing | R&D | Operations | Etc.
- What is your primary work location/office?
- San Francsco | Kuala Lumpur | Rome | Vancouver | Etc.
- Assistant | Officer | Manager | Director | VP | Etc.
- What is your approximate job position level?
- In what Division do you work?
- Question #1: Approximately how often do you use the intranet?
- Several times a day
- Once a day
- Once a week
- Once a month
- I rarely or have never used it
- Question #2: How useful is the intranet for finding the information you need to do your job? (e.g. financial forms, HR tools, contact info, strategic documents)
- Critical to my work
- Very useful
- Moderately useful
- A little useful
- No use at all
- Question #3: How useful is the intranet for collaboration within your teams?
- Critical to my work
- Very useful
- Moderately useful
- A little useful
- No use at all
- Question #4: How helpful is the intranet in building a sense of work community and connection with colleagues?
- I can’t imagine life without it!
- Very helpful
- Somewhat helpful
- No effect
- Question #5: How satisfied are you with the user-friendliness of the intranet and available help (instructions & in-person)?
- Very satisfied
- Very unsatisfied
- Question #6: If you could choose one thing to improve about the intranet, what would it be and how would you improve it?
- Open-ended, text box
Step 2: Running the survey
Even if your intranet implementation is part of a coordinated effort to reduce email traffic you’ll likely want to announce the survey to all employees via email. Consider planning three emails: The first to kick off the survey, the second halfway through the survey period, and the third the day before the survey closes. If your executives are supportive, have the survey go out from the CEO or a VP.
Run your intranet survey for one to two weeks—more if necessary. The length of time depends on your company’s culture (do people typically respond quickly and in large numbers?) as well as factors such as how many staff travel regularly, and how quickly you can get the word out to all employees. Generally speaking, you will see a spike in initial submissions, followed by smaller spikes after the reminder emails go out.
Incentives can be tricky. In many companies, you won’t need incentives because employees are already interested in having a good intranet. Potential incentives could include:
- The opportunity to pilot the new features on the intranet
- A lunchtime pizza party for the department with the highest percentage of submissions
- Offer to donate $1 to a charity for every completed survey response
In many companies employees receive employee engagement surveys, branding surveys, or surveys about where to hold the company holiday party. The prospect of survey fatigue is another reason to keep the intranet survey short and sweet. Also be sure to include in your communications the message that this survey is about a tool employees use daily, and that their input will have a concrete impact.
If you’re ok with a bit less confidence in your survey results, you don’t need everyone in the company to fill out the survey. Even for companies of 10,000 people in size, you can reach a 95% confidence level with a 5% margin of error on about 380 survey responses. Consider how much you need to be confident to go back to your boss, your intranet steering committee, and the company itself in moving forward with actions.
Step 3: Analyzing the results
Your survey will reveal two sets of data. The first is quantitative. All the questions with rating scales provide measurable data you can analyze, present in graphs and charts, and use to show change over time. Quantitative data can easily be visualized.
The second set of data comes from the open-ended question and is purely qualitative. People will respond with written comments, sometimes not even in full sentence form. This is fine. It just means that you will have to spend time reviewing the comments for patterns and trends.
Percentage changes matter
When analyzing the quantitative data and putting it into visual forms, focus on the percentage change rather than raw numbers. While the number of people who check “Several times a day” for how often they use the intranet may go up, that could simply reflect an increase in the size of your workforce.
Compared to a 10% increase in the size of your workforce in a year, a 5% increase in frequent use actually represents a downward trend. Similarly, if one survey gets many fewer responses than a previous survey you may get lower numbers across the board. But the percentages may still show improvements. Comparing percentages over time will show a clear picture of changes in use levels and satisfaction.
The value of the open-ended comments & feedback
The quantitative data that’s easy to analyze suggests trends in how employees feel, but it doesn’t tell stories or provide specific, actionable feedback.
Comments from an open-ended question provide the narrative that explains the changes you see in the quantitative data. Open-ended comments will also likely highlight specific areas of the intranet’s navigation that frustrate users. You’ll also likely hear comforting kudos that highlight really successful areas of the intranet and valued efforts by your team.
Don’t underestimate the effort or focus required to derive meaningful improvements from the survey results. The quantitative data on its own says only so much. It’s the contextual analysis that provides deeper insights. The open-ended comments will need to be mulled over and read through several times to derive actionable meaning.
Remember that you start by simply observing the trends in the data. You’ll need to make a leap of faith from the data to your conclusions about what it means. These conclusions will rely on your existing knowledge of the intranet and its users combined with the feedback left in comments.
Step 4: Reporting the findings
Does one report size fit all? Not really. At a minimum your audiences will include:
Intranet stakeholders: Need to see that their efforts are worthwhile and understand their role in the intranet’s success
- Executives: Concerned with how the intranet impacts the company’s goals, strategy, and bottom line
- Core intranet team: Wants to know if its efforts are having a positive impact or are all in vain
- All employees/average users: Want to know that you’re listening to them and offering responsive service
Got bad news? Report it. The more honest you are the more likely people will trust the overall results. Bad news, or weak areas of the intranet uncovered by the survey, provide useful opportunities.
Bad news may highlight a problem you’ve already been trying to get support for. Or it will provide you with insights into something that might have been overlooked. As long as you’re able to make observations and come up with recommendations for how to address it, then the bad news is a good thing.
Step 5: Developing a satisfaction program
There’s little point in measuring if you won’t use the insights to make changes. You don’t have to address every issue, just report honestly on the findings and identify which of the issues you will address. As a survey is just one step in a continuing ongoing process, you need to determine how soon to repeat the survey.
Quarterly is likely too frequent, as most intranet improvement projects require at least a couple months to implement, followed by several more months before the impacts are felt. An annual survey could be too long since a lot can happen during the course of a year. We recommend running the survey every six months. This can provide a steady stream of data without overwhelming employees. You might also want to coordinate with HR, IT and Internal Communications to potentially piggy-back the short intranet satisfaction survey onto another existing survey.
It’s the recurring measurement that provides valuable data. The survey results are merely interesting after the first implementation. The second time the survey is applied you’ll have some results you can compare. But the survey only provides really substantial information after the third application of the survey, when clear trends start to emerge.
You may want to add a question or two that target pain points highlighted by the previous survey. For example, if the second survey received a large number of comments about the poor organization of content on group pages, then on the third survey include a question targeted at that problem. An open-ended question, such as “What one change to content on your group pages would you like to see?” could help gather qualitative feedback on the issue.
Baselining intranet satisfaction when re-launching
By applying the intranet satisfaction survey just before launching a new intranet you can capture employees’ feelings about the old intranet. This baseline of data will provide a clear reference point against which to compare the new intranet.
Try to apply the baseline survey for the old intranet no more than a month before the launch of the new intranet. Following up in six months and then a year with two more surveys, both for the new intranet, will expose any major changes in employee satisfaction that you can attribute to the new social intranet.
Combination with usage over time from web analytics
The approach to measuring intranet satisfaction relies on gathering self-reported data from your users, which can be unreliable. Responses could vary from one day to the next based on moods or recent experiences, or they could underestimate or overestimate how frequently they used the intranet.
Also, be careful when collecting self-reported data. As we know from user research, what people say and what people do aren’t the same thing. To build a complete and reliable picture of intranet use and satisfaction, combine the results of the intranet satisfaction survey with data from your web analytics software.
ThoughtFarmer Analytics offers extensive data on usage. You can dig deeply into data on intranet search terms and results, identify trends in how much people are using the intranet, and explore top content and contributors. Once you combine this information with the results of your survey you can build a clear and comprehensive picture of intranet satisfaction and use.
Combine with ad-hoc usability testing
Another way to supplement the user-reported survey data is to run usability tests on your intranet. Usability testing can be run with minimal resources and will provide live, observable experiences that you can compare with the perception of content findability reported in the survey. If you’re unfamiliar with content focused usability testing, you can read our introduction to task testing.
The questions you ask, how they link to the intranet’s goals and the amount of time you ask of employees all are important variables in designing an intranet satisfaction survey.
But raw data from a survey needs context and a narrative. The data alone tells no story, and measuring is not of value in and of itself. We measure so we can learn, report, and improve. Once you capture data, you still have to analyze it, make assumptions, and come up with an action plan for improvement.
If you consistently measure employee satisfaction with the intranet and consistently report, make changes, measure and report again, you will build trust among stakeholders and users. And the outcome will be something that benefits everyone—an ever-improving intranet.
Have questions? Get in touch! We're always happy to hear from you.
November 8, 2018