Measurement
Part of Creator discovery platforms measurement: quality controls
Choosing an evidence method for creator discovery, from observation to causal evaluation
Compare creator discovery evidence methods from observation to causal evaluation, with shared criteria, prerequisites and honest limitations.
Creator discovery platforms attribution methods answer different questions. Observation can show that an event was recorded. Reconciliation can join authorised states. A credit rule can allocate an outcome. Only a suitable causal evaluation can estimate whether the intervention changed it. No method is the general winner.
What to take away
- Observation, reconciliation, credit rules and causal evaluation answer different questions, so no single method wins.
- Only a controlled causal evaluation can estimate whether an intervention changed an outcome.
- Specify the evidence contract before choosing a method, so incompatible inputs cannot hide behind a label.
- Label each result by what its method can support, and withdraw a causal claim when the comparison group fails.
- Test the chosen method with synthetic cases before release, because a failure blocks the report.
Six evidence methods compared: observation to controlled causal evaluation
Desk comparison, 6 September 2026, for one England buyer workflow. It compares published method documentation and platform credit rules rather than hands-on tests.
Criteria: question answered, identity unit, required evidence, treatment of time and principal failure. Permitted conclusion is the sixth. Costs, thresholds and method weights are not compared here, because no public price list exists and platform windows and deduplication settings are configured per account.
Privacy, PECR, safeguarding, rights and advertising gates sit outside and cannot be traded away. The advertising and disclosure gate is set by the CAP Code, enforced by the Advertising Standards Authority, together with the Competition and Markets Authority's guidance on social media endorsements. Security and accessibility gates are the same.
Six evidence methods compared
Question and identity
- Direct observation
- Did a versioned event occur for an event ID?
- Workflow reconciliation
- Do source, contract, publication and correction states agree for an internal identity?
- Rule-based or platform credit
- Which eligible event receives credit under rule [ ]?
- Descriptive association
- Do measured variables vary together in the stated sample?
- Modelled contribution
- What contribution follows from assumptions [ ]?
- Controlled causal evaluation
- What changed against a credible counterfactual?
Evidence and time
- Direct observation
- System record, provenance, timestamp, timezone and cut-off
- Workflow reconciliation
- Versioned joins, identity rules and exception log
- Rule-based or platform credit
- Defined window, rule version, eligible population and deduplication
- Descriptive association
- Defined population, period, model or summary and uncertainty
- Modelled contribution
- Model version, inputs, counterevidence and sensitivity
- Controlled causal evaluation
- Assignment or comparison design, outcome definition, period and uncertainty
Main failure
- Direct observation
- Missing visibility or duplicate event
- Workflow reconciliation
- False merges or unmatched late records
- Rule-based or platform credit
- Several systems credit the same outcome
- Descriptive association
- Selection, confounding and reverse direction
- Modelled contribution
- Hidden or unstable assumptions
- Controlled causal evaluation
- Contamination, attrition or weak comparison
Permitted conclusion
- Direct observation
- The named system recorded the event
- Workflow reconciliation
- The declared records reconcile, subject to exceptions
- Rule-based or platform credit
- Credit under the stated rule, not effect
- Descriptive association
- An association in that sample
- Modelled contribution
- Conditional estimate, not observed truth
- Controlled causal evaluation
- Effect within the studied design and population
For any row of that table, the credit rule must state its window, its attribution model, whether duplicate identifiers are collapsed, and whether the result is claimed or verified. A modelled contribution must set out its assumptions and the uncertainty threshold that triggers a correction. Where the rule, the assumptions or the threshold are not documented in the platform account, treat the row as unspecified and label the result accordingly.
No public price list exists for these methods. Effort is driven by logging coverage for direct observation, the number of sources joined for reconciliation, and platform settings for credit rules. For descriptive association it is sample size, for modelled contribution it is model review, and for causal evaluation it is whether a comparison group already exists.
Illustrative worked example. A weekly dashboard reports 1,240 credited sign-ups. The platform rule credits the last eligible click inside a seven-day window. Deduplication removes 180 duplicate events, leaving 1,060. Reconciliation against the subscription ledger matches 940 of those, with 120 unmatched. Report 1,060 as credit under the stated rule and version, not as an incremental effect.
Platform credit rules you will meet in practice: Google Ads commonly reports on a 30-day click window and a 1-day view window. Meta Ads commonly defaults to a 7-day click and 1-day view. TikTok Ads typically uses the same 7-day click and 1-day view default.
Treat these as typical defaults to verify in the account, because the window, the rule version and the deduplication setting all change the credited total.
How creator discovery and campaign platforms attribute
Creator discovery and campaign platforms such as CreatorIQ, Traackr, HypeAuditor, Modash, Upfluence, GRIN, Aspire, impact.com and Partnerize report a credited outcome rather than an incremental one, and each relies on one or more of four evidence paths: a creator or discount code entered at checkout, a tracked affiliate or campaign link, creator UTM tagging on a landing page, or paid amplification through whitelisted placements and Spark Ads inventory.
Those paths answer different questions. A code supports a claimed redemption; a tracked link supports a click or a session; UTM tagging groups traffic by source and campaign; a whitelisted placement is credited under the ad platform's own click and view windows. None of them on its own shows that the intervention changed the outcome.
Affiliate and partnership networks such as impact.com and Partnerize credit a first or last eligible click inside a lookback window that is set per programme, so verify the window in the account rather than assume it. A creator platform's own default window is likewise a per-account setting, not a universal rule.
Decision rule for picking a row:
- Was an event recorded? Use direct observation.
- Do the records agree? Use workflow reconciliation.
- Who receives credit? Use a credit rule, and name its version and window.
- Do variables vary together? Use descriptive association.
- What contribution follows from the assumptions? Use modelled contribution.
- Did the intervention change the outcome? Use a controlled causal evaluation.
Apply one evidence contract
For each row, specify: decision; eligible population; identity unit; source and provenance; numerator and denominator or other unit. Then period and timezone; inclusions and exclusions; duplicate, missing and late handling; claimed or verified status; privacy authority. Also query or model version; uncertainty; owner; correction trigger; and the threshold that triggers it.
Evidence contract fields
- Decision
- Eligible population
- Identity unit
- Source and provenance
- Numerator and denominator
- Period and timezone
- Inclusions and exclusions
- Duplicate, missing, late handling
This stops method labels hiding incompatible inputs.
The Government Analysis Function's guidance on communicating uncertainty calls for limitations and possible bias to be explained. It addresses official statistics but offers an appropriate reporting discipline for this commercial investigation.
Do not cross the causal boundary by wording
A profile found is not a suitable creator. Contact is not response. Contract is not campaign approval. Publication is not audience attention. Paid placement is not earned editorial response. Platform-attributed activity is not reconciled retained value. None is automatically an incremental effect.
HM Treasury's Magenta Book distinguishes process, impact and value-for-money evaluation. Its impact-evaluation quality guide explains the role of a counterfactual and why monitoring a change cannot show that an intervention caused it. These public-sector sources provide methodological boundaries, not evidence about this buyer.
Use the weakest accurate label. If the data show only an observed publication, report that. If a platform applies a rule, name the rule and version. If a comparison group fails, retire the causal claim. Preserve the prior report, correction and reason. The final decision can be HOLD even when a credited outcome looks favourable.
Test the chosen method before release
Run synthetic cases for a duplicate identity, a late event, missing source provenance, conflicting credit and a withdrawn publication. Record the expected result and the actual query output. A failure blocks the report until the method owner corrects the logic and reruns every affected case.
This test checks implementation only; it cannot validate the underlying creator claim or establish causation.
Before you act
- Keep privacy, PECR, safeguarding and rights gates outside the trade.
- Run the synthetic cases and record the expected result against the actual query output.
- Hold release until every failed case is corrected and rerun.
Common questions
What does a controlled causal evaluation estimate that other methods cannot?
It estimates what changed against a credible counterfactual, within the studied design and population. Observation only shows a system recorded an event, reconciliation joins authorised states, and a credit rule allocates an outcome. None of those establishes whether the intervention changed the outcome.
What should happen when a comparison group fails?
The causal claim no longer holds when the comparison group fails. Preserve the prior report, correction and reason, and be willing to record a HOLD even when a credited outcome looks favourable.
