Read the method before the headline
Statistics Canada’s questions-to-ask guidance links the conventional margin of sampling error to probability sampling. It also distinguishes sampling error from problems such as non-response. A large sample does not by itself establish that an opt-in poll represents the wider population.
Look for who commissioned the survey, who conducted it, how respondents were selected and the exact question. A programme’s summary can be shorter or less qualified than the wording respondents actually answered.
Keep dates and groups attached
Field dates identify when responses were collected, which can precede the broadcast. A developing event after collection may affect how relevant the result is to a current discussion. Do not describe the poll as a live measurement simply because the programme is live.
Statistics Canada notes that smaller subgroups generally have greater sampling uncertainty than the overall sample. If a graphic highlights one age group or region, check the subgroup size and its own methodological information rather than copying the full sample’s margin.
Also inspect the denominator. Results among all respondents can differ from results among decided respondents or another filtered group. A percentage-point difference is not automatically evidence of a meaningful change without the relevant uncertainty analysis.
Share the scope, not just the number
Record the population, method, field period, question, sample size and published uncertainty information. Use the survey organisation’s own explanation if the methodology involves modelling or a non-probability sample; do not attach a conventional margin that the source does not justify.
For two polls, compare the wording and methods before drawing a trend line. Different surveys can measure different questions even when the television labels look similar. A careful viewing note attributes the finding to the survey and preserves its limits, rather than presenting a rounded headline as the settled opinion of every person in a country.
A phone-in programme vote measures its participating audience under that process; it should not automatically be relabelled as a national opinion poll.
If a question allows multiple answers, percentages can exceed one hundred when added. Read that instruction before assuming the published results contain an arithmetic error.
Related questions
Does a large sample remove every source of bias?
No. Selection, question wording and non-response still matter.
Can I use the overall margin for any subgroup?
Check the subgroup’s own sample and methodology.
Sources and scope
Scope notes distinguish published documentation, supplied information and editorial methods. They do not report a hands-on service test.
- Statistics Canada: survey questions to ask — Probability sampling, subgroup uncertainty and non-response.
- Statistics Canada: sampling error — Explains sampling uncertainty and confidence intervals.
