Response Disposition Dashboard

Last updated: Apr 19, 2026, 7:00 AM EDT

Live quality triage results for the Technology Adoption Barriers Survey (2026). Data is updated automatically by the disposition pipeline.

Overview

507

Total Participants

268

Approved (52.9%)

134

Auto-Exclude (26.4%)

197

Flagged for Review (38.9%)

Disposition Breakdown

Auto-Exclude(AUTO-EXCLUDE)

134 (26.4%)

Failed 2+ attention checks or speed + attention compound failure.

Clean(CLEAN)

97 (19.1%)

All quality and auth checks passed: finished, all 3 IRIs correct, duration ≥ 9 min (Smeal benchmark), reCAPTCHA ≥ 0.5, no straightlining, no partial straightlining, Prolific auth checks passed. Approved on Prolific.

Flag: Single IRI(FLAG-SINGLE-IRI)

88 (17.4%)

Missed 1 of 3 attention checks at normal speed. Manual review.

Incomplete(INCOMPLETE)

79 (15.6%)

Survey not completed.

Flag: Smeal Benchmark(FLAG-SMEAL)

58 (11.4%)

Below 9-minute doctoral SME benchmark but passed all attention checks.

Flag: Partial Straightlining(FLAG-PARTIAL-STRAIGHTLINING)

39 (7.7%)

Near-identical responses in one or more question blocks (SD < 0.5).

Flag: Speed(FLAG-SPEED)

8 (1.6%)

Completed in under 5 minutes but passed all attention checks.

Flag: reCAPTCHA(FLAG-RECAPTCHA)

4 (0.8%)

reCAPTCHA score below 0.5, indicating potential bot activity.

Auto-Exclude Breakdown

The 134 auto-excluded submissions are broken down by severity. Failed all 3 IRIs with speed violation is the strongest exclusion signal.

Failed 3 IRI + Speed

All 3 attention checks wrong AND under 5 minutes

10% of auto-excludes

Failed 3 IRI

All 3 attention checks wrong, normal speed

24% of auto-excludes

Failed 2 IRI + Speed

2 of 3 attention checks wrong AND under 5 minutes

8% of auto-excludes

Failed 2 IRI

2 of 3 attention checks wrong, normal speed

46% of auto-excludes

Speed + 1 IRI

Under 5 minutes AND 1 attention check wrong (compound signal)

12% of auto-excludes

Action Status

Action	Count
Approved on Prolific	268
Rejected on Prolific	35
Awaiting Review	43
Returned	169
Timed Out	6

Attention Check (IRI) Pass Rates

Three instructed-response items are embedded in the survey to detect inattentive responding. Each requires selecting a specific answer.

Barrier IRI

58.4%

Block: 19 items | Expected: “Major Barrier”

Readiness IRI

69.4%

Block: 18 items | Expected: “Low Readiness/Capability”

Maturity IRI

82.2%

Block: 9 items | Expected: “Level 2: Developing/Repeatable”

Result Group Types - The Four Analysis Datasets

Every TABS metric is computed independently across four primary result groups (datasets). Each group applies progressively fewer quality filters. Researchers choose which dataset to use based on their tolerance for data quality risk. All four are refreshed daily by the analysis pipeline. Full statistics for each group are available on the Descriptive Statistics, Scale Reliability, and Sensitivity Analysis pages.

⚠️Prolific API and pipeline Ns do NOT match - investigate

Prolific API “Approved” tab: 268 · Pipeline “Prolific Accepted” sample: 265 (Δ = 3)

Both numbers should be identical. A mismatch indicates the approve automation failed to send commands to Prolific, or Prolific enrichment is using stale data.

Conservative Clean

N = 93

Definition: Prolific APPROVED + all quality checks (3 IRIs, duration ≥ 540s, reCAPTCHA, no straightlining, auth)

Used for: Primary CRP analysis - all published findings

Construct	Mean	SD	Cronbach’s α
Barriers	2.8371	0.6195	0.8512
Readiness	3.0694	0.5674	0.8676
Maturity	3.0772	0.6999	0.8290

Correlations: B-R = -0.41, B-M = -0.15, R-M = 0.59

Flexible Clean

N = 144

Definition: Prolific APPROVED + basic quality (3 IRIs + duration ≥ 480s)

Used for: Sensitivity analysis - verifies findings hold with lighter filters

Construct	Mean	SD	Cronbach’s α
Barriers	2.8153	0.7061	0.8741
Readiness	3.0965	0.6561	0.9154
Maturity	3.0811	0.8041	0.8855

Correlations: B-R = -0.44, B-M = -0.30, R-M = 0.71

Prolific Accepted

N = 265

Definition: All deduplicated V2 rows with Prolific APPROVED status (must match Prolific UI)

Used for: N ceiling - maximum usable sample, platform truth

Construct	Mean	SD	Cronbach’s α
Barriers	2.7957	0.7063	0.8755
Readiness	3.1309	0.6690	0.9174
Maturity	3.1602	0.8051	0.8887

Correlations: B-R = -0.34, B-M = -0.27, R-M = 0.72

All V2 Finished

N = 428

Definition: All finished responses with duration ≥ 120s (any Prolific status)

Used for: Full-population baseline - includes returned/rejected/awaiting

Construct	Mean	SD	Cronbach’s α
Barriers	2.7773	0.7595	0.8975
Readiness	3.2249	0.7142	0.9297
Maturity	3.2615	0.8017	0.8893

Correlations: B-R = -0.30, B-M = -0.32, R-M = 0.72

Conservative Clean ⊂ Flexible Clean ⊂ Prolific Accepted. All V2 Finished overlaps with Prolific Accepted but is not a strict subset (it includes non-APPROVED responses; Prolific Accepted includes incomplete-but-approved responses).

Note: Disposition CLEAN (97) vs. Conservative Clean (93)

Disposition CLEAN is the operations concept: a response that passes all quality checks in the waterfall (below). It is used to auto-approve participants on Prolific. It does not check Prolific_Status.

Conservative Clean is the analysis concept: Prolific_Status == APPROVED plus all quality checks. This is the primary analysis sample.

Expected: After the daily auto-approve workflow runs, all Disposition CLEAN participants should have Prolific_Status == APPROVED, making the counts converge. Any persistent gap indicates the approve automation failed to send commands to Prolific. The gap is currently 4, explained by 34 partial-straightlining responses that were manually approved on Prolific but excluded from Conservative Clean.

Disposition Waterfall

Each response is evaluated through this waterfall - the first matching step determines the disposition:

Step	Disposition	Criteria	Action
0	INCOMPLETE	Survey not finished	-
1	AUTO-EXCLUDE: IRI3+Speed	All 3 IRI checks wrong AND under 5 minutes	Auto-reject
	AUTO-EXCLUDE: IRI3	All 3 IRI checks wrong, normal speed	Auto-reject
	AUTO-EXCLUDE: IRI2+Speed	2 of 3 IRI checks wrong AND under 5 minutes	Auto-reject
	AUTO-EXCLUDE: IRI2	2 of 3 IRI checks wrong, normal speed	Pending
	AUTO-EXCLUDE: Speed+IRI	Under 5 minutes AND 1 IRI failure (compound)	Pending
2	FLAG-SPEED	<5 min but all 3 IRIs passed	Message + review
3	FLAG-SINGLE-IRI	1 IRI failure at normal speed	Message + review
4	FLAG-SMEAL	5-9 min (below Smeal eDBA benchmark of 9 min)	Message + review
5	FLAG-RECAPTCHA	reCAPTCHA score < 0.5	Message + review
6	FLAG-STRAIGHTLINING	Qualtrics: same answer for every item in a block	Auto-reject
7	FLAG-PARTIAL-STRAIGHTLINING	Within-person SD < 0.5 in any question block	Message + review
8	CLEAN	All checks passed	Auto-approve