by In last week’s article we delved deeper into Statistically Valid Data, Data You Can Trust and looked at different examples as to why it is a critical tool that enables us to make good business decisions based on good profit and customer loyalty.
This week we are going to sift through more information, segmentation and exactly what is a statistically valid data sample.
If you don’t talk to your customers, chances are the only data you tend to focus on is complaints and what is being said on social media, which is hard to verify and validate.
What is said on social media can be really damaging. Maybe, you are lucky and getting loyal customer reviews and some accolades as well. But most people, are more likely to complain, just because that’s the way we’re wired.
Telephone research provides the opportunity to converse, to probe and to clarify responses. If you want to make a difference to your customers experience. You need to understand and focus on the areas that have the biggest impact.
You want to make sure you are researching the right customers. The more the better but think about their value to you and whether they are decision makers, influencers, and end users. All of these groups’ feedback is critical to ensuring you keep these clients in the future.
It is important to use a proven methodology. The Net Promoter Score (NPS™) is a globally proven method for measuring customer word of mouth. Our preferred method.
Segment Your Customers – Clarity Is Power
One of the challenges we see a lot is that we only get overall feedback. And we don’t break that down to the different levels (Segments). See below as example. It is recommended, where possible, to go down at least three levels.
You need to research a good sample at each level. Clarity is power; you’ll get much more detailed findings if you go down at least three levels.
Business decisions must be made on statistically valid data, data that you can trust.
What Is A Statistically Valid Data Sample?
All the information relating to this Sample Size Calculator is presented as a public service of Creative Research Systems survey software.
You can use it to determine how many people you need to interview in order to get results that reflect the target population as precisely as needed. You can also find the level of precision you have in an existing sample.
Before using the sample size calculator, there are two terms that you need to know. These are: confidence interval and confidence level.
To learn more about the factors that affect the size of confidence intervals. To read more go to https://www.surveysystem.com/sscalc.htm#two
Enter your choices in a calculator below to find the sample size you need or the confidence interval you have. Leave the Population box blank if the population is very large or unknown.
Sample Size Calculator Terms: Confidence Interval and Confidence Level
The confidence interval (also called margin of error) is the plus-or-minus figure usually reported in newspaper or television opinion poll results. For example, if you use a confidence interval of 4 and 47% percent of your sample picks an answer you can be “sure” that if you had asked the question of the entire relevant population between 43% (47-4) and 51% (47+4) would have picked that answer.
The confidence level tells you how sure you can be. It is expressed as a percentage and represents how often the true percentage of the population who would pick an answer lies within the confidence interval. The 95% confidence level means you can be 95% certain; the 99% confidence level means you can be 99% certain. Most researchers use the 95% confidence level. When you put the confidence level and the confidence interval together, you can say that you are 95% sure that the true percentage of the population is between 43% and 51%. The wider the confidence interval you are willing to accept, the more certain you can be that the whole population answers would be within that range.
For example, if you asked a sample of 1000 people in a city which brand of cola they preferred, and 60% said Brand A, you can be very certain that between 40 and 80% of all the people in the city actually do prefer that brand, but you cannot be so sure that between 59 and 61% of the people in the city prefer the brand.
Factors That Affect Confidence Intervals
There are three factors that determine the size of the confidence interval for a given confidence level:
Sample size
Percentage
Population size
Sample Size
The larger your sample size, the surer you can be that their answers truly reflect the population. This indicates that for a given confidence level, the larger your sample size, the smaller your confidence interval. However, the relationship is not linear (i.e., doubling the sample size does not halve the confidence interval).
Percentage
Your accuracy also depends on the percentage of your sample that picks a particular answer. If 99% of your sample said “Yes” and 1% said “No,” the chances of error are remote, irrespective of sample size. However, if the percentages are 51% and 49% the chances of error are much greater. It is easier to be sure of extreme answers than of middle-of-the-road ones.
When determining the sample size needed for a given level of accuracy you must use the worst case percentage (50%). You should also use this percentage if you want to determine a general level of accuracy for a sample you already have. To determine the confidence interval for a specific answer your sample has given, you can use the percentage picking that answer and get a smaller
interval.
Population Size
How many people are there in the group your sample represents? This may be the number of people in a city you are studying, the number of people who buy new cars, etc. Often you may not know the exact population size. This is not a problem. The mathematics of probability prove that the size of the population is irrelevant unless the size of the sample exceeds a few percent of the total population you are examining. This means that a sample of 500 people is equally useful in examining the opinions of a state of 15,000,000 as it would a city of 100,000. For this reason, The Survey System ignores the population size when it is “large” or unknown. Population size is only likely to be a factor when you work with a relatively small and known group of people (e.g., the members of an association).
The confidence interval calculations assume you have a genuine random sample of the relevant population. If your sample is not truly random, you cannot rely on the intervals. Non-random
samples usually result from some flaw or limitation in the sampling procedure. An example of such a flaw is to only call people on landline phones during the day and miss almost everyone who works.
For most purposes, the non-working population cannot be assumed to accurately represent the entire (working and non-working) population. An example of a limitation is using an opt-in online poll, such as one promoted on a website. There is no way to be sure an opt-in poll truly represents the population of interest.
Get Professional Analytics Done – Hand Over To A Third Party
As mentioned in a previous Blog, 10 Actions To Implement A Customer Loyalty Research Program now you have this great data it is not about reading and reacting to individual comments. It’s about the patterns that good analysts can identify. If you want more customers recommending you to their friends and colleagues, then you need to know what makes them return.
It’s very difficult to do the analytics yourself, if you don’t know how to do ‘good analytics’.
Learning to understand – How do you identify best practice drivers? Or how do you find your improvement drivers? For example, takes time and a particular skill set.
The need to read every customer response is critical.
Text mining tools don’t drill down to the level required to fully understand the voice of your customer. Or the behaviours that drive customer loyalty.
They cannot capture what you do that has a customer describe you as professional, or responsive. It is only by reading each response that you can clearly see the examples mentioned.
For example, by returning a telephone message within 2-3 hours had a customer describe you as very responsive. But how many said that?
You don’t change for one person you look for the overall view from your statistically valid findings.
Invest In Conversations
I highly recommend you invest your money in telephone or in-person conversations with customers, rather than relying on just online surveys.
Online surveys are easier, cheaper, and unfortunately increasingly common but they don’t provide rich detail.
Individual conversations cost more in the short term, but they provide the opportunity to probe and dig deeper into the detail you need to drive performance improvement.
For example, we were working with a $50 million business, and improved their business by 41% over two years. With those returns, the cost of in-depth research was insignificant – just a small fraction of a single percentage point on the gain.
By talking to people, you adjust the number and level of questions appropriately. You interpret the results of the initial questions to choose the correct follow-up questions, without leading the customer or asking irrelevant questions.
In fact, I believe a good survey should take around three to five minutes – and no longer. When we survey customers for a client, most of them agree to do the survey because we make it clear we will only take a few minutes of their precious time.
Even in such a short time, we still gather a lot of important information.
The most important thing is to probe their responses and ask the right next question.
For example, if they say, “We think you’re very friendly” a skilled interviewer will respond with, “What do we do that would have you describe us as being friendly?” If they say, “I think you can improve your timeliness” we will respond with, “Can you tell us exactly what would we need to do to improve timeliness? What do you feel is an appropriate timeline to ….?”
The point is that you will discover what’s important to them.
I recommend that, if possible, you ask this question in conversation, for example, in a phone call, or a meeting, as it gives you the chance to probe their response.
For example, if somebody says, “I thought the salesperson was very professional” you don’t exactly know what “professional” means.
It could mean they turned up on time, responded to queries promptly, dressed in a suit, took time to listen, sent a beautifully-formatted proposal, gave them the best price, or something else entirely!
You won’t know for sure exactly what they mean unless you probe further. The level of detail collected, and the analysis of your customer feedback is critical.
When you know the detail, you know what to replicate next time.
If you would like to have a chat with me about ‘Statistically Valid Data’ contact me I look forward to hearing from you.
Warm regards,
Craig