Response Disposition Dashboard

Last updated: Jul 18, 2026, 7:47 AM EDT

Live quality triage results for the Technology Adoption Barriers Survey (2026). Data is updated automatically by the disposition pipeline.

Overview

864

Total Participants

476

Approved (55.1%)

225

Auto-Exclude (26.0%)

365

Flagged for Review (42.2%)

Disposition Breakdown

Auto-Exclude(AUTO-EXCLUDE)

225 (26.0%)

Failed 2+ attention checks or speed + attention compound failure.

Clean(CLEAN)

159 (18.4%)

All quality and auth checks passed: finished, all 3 IRIs correct, duration ≥ 9 min (Smeal benchmark), reCAPTCHA ≥ 0.5, no straightlining, no partial straightlining, Prolific auth checks passed. Approved on Prolific.

Flag: Single IRI(FLAG-SINGLE-IRI)

158 (18.3%)

Missed 1 of 3 attention checks at normal speed. Manual review.

Incomplete(INCOMPLETE)

115 (13.3%)

Survey not completed.

Flag: Smeal Benchmark(FLAG-SMEAL)

109 (12.6%)

Below 9-minute doctoral SME benchmark but passed all attention checks.

Flag: Partial Straightlining(FLAG-PARTIAL-STRAIGHTLINING)

65 (7.5%)

Near-identical responses in one or more question blocks (SD < 0.5).

Flag: reCAPTCHA(FLAG-RECAPTCHA)

17 (2.0%)

reCAPTCHA score below 0.5, indicating potential bot activity.

Flag: Speed(FLAG-SPEED)

15 (1.7%)

Completed in under 5 minutes but passed all attention checks.

Flag: Full Straightlining(FLAG-STRAIGHTLINING)

1 (0.1%)

Same answer for every item in an entire question block.

Auto-Exclude Breakdown

The 225 auto-excluded submissions are broken down by severity. Failed all 3 IRIs with speed violation is the strongest exclusion signal.

Failed 3 IRI + Speed

All 3 attention checks wrong AND under 5 minutes

11% of auto-excludes

Failed 3 IRI

All 3 attention checks wrong, normal speed

24% of auto-excludes

Failed 2 IRI + Speed

2 of 3 attention checks wrong AND under 5 minutes

10% of auto-excludes

104

Failed 2 IRI

2 of 3 attention checks wrong, normal speed

46% of auto-excludes

Speed + 1 IRI

Under 5 minutes AND 1 attention check wrong (compound signal)

9% of auto-excludes

Action Status

Action	Count
Approved on Prolific	476
Rejected on Prolific	55
Returned	334
Timed Out	9

Attention Check (IRI) Pass Rates

Three instructed-response items are embedded in the survey to detect inattentive responding. Each requires selecting a specific answer.

Barrier IRI

59.8%

Block: 19 items | Expected: “Major Barrier”

Readiness IRI

69%

Block: 18 items | Expected: “Low Readiness/Capability”

Maturity IRI

82.1%

Block: 9 items | Expected: “Level 2: Developing/Repeatable”

Result Group Types - The Four Analysis Datasets

Every TABS metric is computed independently across four primary result groups (datasets). Each group applies progressively fewer quality filters. Researchers choose which dataset to use based on their tolerance for data quality risk. All four are refreshed daily by the analysis pipeline. Full statistics for each group are available on the Descriptive Statistics, Scale Reliability, and Sensitivity Analysis pages.

✅Prolific API and pipeline Ns match

Prolific API “Approved” tab: 476 · Pipeline “Prolific Accepted” sample: 476

Both numbers should be identical. A mismatch indicates the approve automation failed to send commands to Prolific, or Prolific enrichment is using stale data.

Conservative Clean

N = 157

Definition: Prolific APPROVED + all quality checks (3 IRIs, duration ≥ 540s, reCAPTCHA, no straightlining, auth)

Used for: Primary CRP analysis - all published findings

Construct	Mean	SD	Cronbach’s α
Barriers	2.8705	0.6676	0.8660
Readiness	3.0373	0.5945	0.8688
Maturity	3.0672	0.7074	0.8204

Correlations: B-R = -0.40, B-M = -0.17, R-M = 0.57

Flexible Clean

N = 253

Definition: Prolific APPROVED + basic quality (3 IRIs + duration ≥ 480s)

Used for: Sensitivity analysis - verifies findings hold with lighter filters

Construct	Mean	SD	Cronbach’s α
Barriers	2.8897	0.7220	0.8846
Readiness	3.0603	0.6605	0.9083
Maturity	3.0689	0.8057	0.8815

Correlations: B-R = -0.43, B-M = -0.28, R-M = 0.69

Prolific Accepted

N = 476

Definition: All deduplicated V2 rows with Prolific APPROVED status (must match Prolific UI)

Used for: N ceiling - maximum usable sample, platform truth

Construct	Mean	SD	Cronbach’s α
Barriers	2.8913	0.7203	0.8858
Readiness	3.0940	0.6517	0.9093
Maturity	3.1215	0.7846	0.8779

Correlations: B-R = -0.32, B-M = -0.24, R-M = 0.71

All V2 Finished

N = 749

Definition: All finished responses with duration ≥ 120s (any Prolific status)

Used for: Full-population baseline - includes returned/rejected/awaiting

Construct	Mean	SD	Cronbach’s α
Barriers	2.8962	0.7811	0.9071
Readiness	3.2057	0.7188	0.9257
Maturity	3.2312	0.8036	0.8866

Correlations: B-R = -0.26, B-M = -0.23, R-M = 0.72

Conservative Clean ⊂ Flexible Clean ⊂ Prolific Accepted. All V2 Finished overlaps with Prolific Accepted but is not a strict subset (it includes non-APPROVED responses; Prolific Accepted includes incomplete-but-approved responses).

Note: Disposition CLEAN (159) vs. Conservative Clean (157)

Disposition CLEAN is the operations concept: a response that passes all quality checks in the waterfall (below). It is used to auto-approve participants on Prolific. It does not check Prolific_Status.

Conservative Clean is the analysis concept: Prolific_Status == APPROVED plus all quality checks. This is the primary analysis sample.

Expected: After the daily auto-approve workflow runs, all Disposition CLEAN participants should have Prolific_Status == APPROVED, making the counts converge. Any persistent gap indicates the approve automation failed to send commands to Prolific. The gap is currently 2, explained by 56 partial-straightlining responses that were manually approved on Prolific but excluded from Conservative Clean.

Disposition Waterfall

Each response is evaluated through this waterfall - the first matching step determines the disposition:

Step	Disposition	Criteria	Action
0	INCOMPLETE	Survey not finished	-
1	AUTO-EXCLUDE: IRI3+Speed	All 3 IRI checks wrong AND under 5 minutes	Auto-reject
	AUTO-EXCLUDE: IRI3	All 3 IRI checks wrong, normal speed	Auto-reject
	AUTO-EXCLUDE: IRI2+Speed	2 of 3 IRI checks wrong AND under 5 minutes	Auto-reject
	AUTO-EXCLUDE: IRI2	2 of 3 IRI checks wrong, normal speed	Pending
	AUTO-EXCLUDE: Speed+IRI	Under 5 minutes AND 1 IRI failure (compound)	Pending
2	FLAG-SPEED	<5 min but all 3 IRIs passed	Message + review
3	FLAG-SINGLE-IRI	1 IRI failure at normal speed	Message + review
4	FLAG-SMEAL	5-9 min (below Smeal eDBA benchmark of 9 min)	Message + review
5	FLAG-RECAPTCHA	reCAPTCHA score < 0.5	Message + review
6	FLAG-STRAIGHTLINING	Qualtrics: same answer for every item in a block	Auto-reject
7	FLAG-PARTIAL-STRAIGHTLINING	Within-person SD < 0.5 in any question block	Message + review
8	CLEAN	All checks passed	Auto-approve