Data collection methods

What data collection actually means

Data collection is the deliberate, documented capture of evidence against a question you wrote down first. The order matters: a question, then a method chosen because it can answer that question, then an instrument — a questionnaire, an interview guide, an observation sheet — and only then respondents. Most weak studies invert it. They start with a tool somebody already had, collect whatever it produces, and look for a question the data happens to fit.

Three decisions determine whether the result is usable. The unit of analysis: are you describing people, sessions, transactions or days? Mixing them is the most common reason two analysts get different numbers from the same file. The instrument: fixed wording, fixed options and fixed units, frozen before fieldwork opens. The selection rule: how people or records got into the dataset, which decides what you are entitled to claim at the end.

Everything after that is execution. A clean instrument with a broken selection rule produces confident nonsense; a rough instrument with an honest selection rule produces a useful caveat. Both are better than a survey that was never written down, sent to whoever answered, and then reported as a percentage.

Primary vs secondary data

Primary data is collected by you, for your question, with an instrument you control. You choose the wording, the sample and the timing, so the data fits the question exactly — and you pay for all of it in time, recruitment and fieldwork.

Secondary data was collected by somebody else for their own purposes: official statistics, published research, industry reports, administrative records, platform analytics. It is faster, cheaper, usually far larger and frequently better collected than anything you could field alone. The catch is definitional. Their age bands, reporting periods and thresholds were chosen for their question, not yours, and a mismatch you notice late can invalidate an entire analysis.

In practice most good projects use both: secondary data to establish context and benchmarks, primary data for the specific thing nobody has measured. Keep a Secondary Data Log from the first download — publisher, reference period, licence, access date — because the provenance question always arrives later than the download.

Quantitative vs qualitative methods

Quantitative methods produce values you can count, compare and test: how many, how often, how much, how likely. Qualitative methods produce language and observation you can interpret: how something works, why somebody did it, what they thought was happening. The distinction is not rigour — both can be done well or badly — it is the kind of claim each one supports.

The failure modes are mirror images. Quantitative work fails by measuring a proxy precisely and calling it the thing itself. Qualitative work fails by generalising twelve conversations into a population claim. Mixed-method designs exist to cover for each other: interviews first to find out what the questions should be, a survey to find out how common each answer is, then a handful of interviews again to explain the result that surprised you.

The ten methods, compared

Cost and speed below assume you run the fieldwork yourself with no agency. "Main bias" is the failure this method is most exposed to — not the only one, but the one that most often survives into a published finding.

Sampling methods

Sampling is the selection rule, and it decides what your numbers are allowed to mean. Probability sampling gives every member of the population a known, non-zero chance of selection, which is what makes generalisation and margins of error legitimate. Non-probability sampling does not — it can still be the right choice, but the claim at the end has to shrink to match.

For quota and purposive designs, the selection happens in the screener. The Focus Group Screener is built for exactly this: it asks the qualifying questions before it explains the study, so nobody can work out which answer gets them in.

How big does your sample need to be?

For a quantitative estimate, sample size comes from three inputs: how precise you need to be (margin of error), how sure you need to be that the true value sits inside that range (confidence level), and how large the population is. The standard result surprises people — for any large population, ±5% at 95% confidence needs about 385 completed responses whether the population is fifty thousand or fifty million. Population size only matters when it is small.

Qualitative work sizes differently. There is no formula; you sample until new sessions stop producing new themes. In practice that is usually 8–15 interviews for a fairly homogeneous group, more if you are comparing distinct populations, and 3–6 focus groups before the discussion starts repeating itself. For diary studies, plan for attrition rather than pretending it won't happen: recruit 10–40 people, expect a fifth to drift, and check compliance on day three rather than day ten.

Writing questions that don't skew the answer

Most bias in survey data is introduced by the person who wrote the questionnaire, not by the people who answered it. Six rules prevent the majority of it.

Likert scales, done properly

A Likert item is a statement plus a symmetric agreement scale — not any old rating question. Keep two negative points, a genuine neutral midpoint and two positive points, use identical wording on every item, and reverse-word a couple of statements in each block so that respondents ticking straight down the page become visible in the export. Offer "not applicable" separately from the midpoint: "no opinion" and "does not apply to me" are different answers. Five points is enough for most general samples; seven adds variance for expert respondents and little else. When you report, publish the distribution as well as any mean — a bimodal set of answers has no meaningful average, and 3.72 on a five-point scale implies precision the instrument does not have. The Likert Scale Survey ships with all of this built in, reverse-worded items included.

Which method should you use?

Method choice is mostly constraint arithmetic: what you need to know, who you can reach, what you can spend and how long you have. Answer four questions and the chooser names a design plus the claim it will not support.

Tools and instruments

You need four capabilities, and they are less glamorous than the software category suggests: an instrument you can version, capture without re-typing, storage you can export from, and an audit trail of consent.

• The instrument. A form builder for questionnaires, screeners, consent, logs and field sheets. The advantage over a document is that logic runs at capture time — a follow-up appears only when the answer needs one, and required fields cannot be skipped.
• Capture. Anything that avoids transcription. Paper and spreadsheets both introduce copying errors; a form on a phone that queues offline and syncs later does not.
• Analysis. A spreadsheet is enough for frequencies and cross-tabs. R, Python, SPSS or Stata for anything inferential. Qualitative coding can be done in a spreadsheet up to about thirty transcripts before dedicated software pays for itself.
• Governance. Consent records stored apart from the data, keyed by participant code, plus a stated retention period and a route to withdraw.

HelloForms covers the first two and the last: conditional logic, offline-tolerant capture, CSV and PDF export, and a Research Consent Form with each permission recorded separately. For readings taken in the field, the Data Collection Sheet fixes units in the labels so a day of measurements cannot arrive in three different scales.

Ethics, consent and data protection

None of this is legal advice, and your ethics committee or data protection lead has the final word. But five points come up in almost every review.

• Consent is several decisions, not one. Taking part, being recorded, being quoted, having data reused and being contacted again are separate permissions. Bundling them into a single tick box is not informed consent, and it costs you the participants who would have said yes to three out of five.
• Collect less than you can. Every field you add is a field you must justify, store and eventually delete. If an answer will not change what you do, it does not belong on the form.
• Anonymity and confidentiality are different promises. Anonymous means you cannot identify the respondent even if you want to. Confidential means you can but you won't. Say which one you are offering, and never say 'anonymous' on a form that carries an email field.
• State retention and withdrawal in concrete terms. Give a period, not 'as long as necessary', and name the point after which data can no longer be pulled out of an analysis. A vague promise is worse than an honest limit.
• Special category data raises the bar. Health, ethnicity, religion, sexuality, biometrics and political opinion carry extra obligations under UK and EU data protection law. If you are collecting them, you need a documented lawful basis before you field the form.

Nine mistakes that ruin a dataset

• Treating a self-selected sample as representative. Report it as what it is — the people who chose to answer — and compare their characteristics with the population you claim to describe.
• Quoting a margin of error on a convenience sample. Margins of error assume random selection. Without it, report counts and distributions and drop the ± figure.
• Changing the instrument mid-study. Freeze the wording once fieldwork starts. If you must change it, treat the data as two datasets and say so.
• Mixing description with interpretation in field notes. Two boxes: what was seen, and what the observer thinks it meant. Descriptions can be re-analysed; conclusions cannot.
• Leaving units to free text. Put the unit in the field label and accept numbers only. Half your readings otherwise arrive in the wrong scale.
• Losing the pairing between rounds. Issue matching codes at enrolment so pre and post entries can be paired without storing names.
• Asking demographics first. Move age, income and ethnicity to the end. At the top they raise abandonment before anyone knows why you're asking.
• Not recording the access date on a source. Log publication date and access date separately. Online sources change without notice.
• Peeking at an experiment and stopping when it looks good. Fix the sample size and the end date before launch. Early stopping manufactures significance.

Frequently asked questions