CRP 2026: Sensitivity Analysis
Published: April 2026Sensitivity analysis tests whether findings are robust to the choice of inclusion criteria. Every key metric - means, standard deviations, inter-construct correlations, and reliability coefficients - is computed independently across three CRP sample definitions. If a finding holds across Conservative Clean (N=79) and Prolific Accepted (N=200), it is robust to inclusion criteria.
Sample Definitions
The three primary result groups below are used throughout this analysis. They are nested from most restrictive to least restrictive.
| Key | Label | Description | N |
|---|---|---|---|
| conservative_clean | Conservative Clean | Prolific APPROVED + all quality checks (IRI, duration >= 540s, reCAPTCHA, straightlining, auth) | 79 |
| flexible_clean | Flexible Clean | Prolific APPROVED + basic quality (all 3 IRIs + duration >= 480s) | 116 |
| prolific_accepted | Prolific Accepted | All deduplicated V2 rows with Prolific APPROVED status | 200 |
Constraints: Conservative Clean ⊆ Flexible Clean ⊆ Prolific Accepted.
Full Sensitivity Table
The table below shows every metric computed across all three sample definitions. Values are formatted to four decimal places. Means, standard deviations, correlations, and Cronbach’s alpha coefficients are all included.
| Metric | Conservative Clean N=79 | Flexible Clean N=116 | Prolific Accepted N=200 |
|---|---|---|---|
| Barrier Grand Mean | 2.8139 | 2.7891 | 2.7644 |
| Barrier SD | 0.6309 | 0.6610 | 0.6893 |
| Readiness Grand Mean | 3.0483 | 3.0765 | 3.1361 |
| Readiness SD | 0.5829 | 0.6239 | 0.6674 |
| Maturity Grand Mean | 3.0318 | 3.0616 | 3.1533 |
| Maturity SD | 0.7087 | 0.7501 | 0.7855 |
| B-R Correlation | -0.5016 | -0.5172 | -0.3814 |
| B-M Correlation | -0.2451 | -0.3653 | -0.3156 |
| R-M Correlation | 0.5877 | 0.6821 | 0.7194 |
| Alpha Barriers | 0.8572 | 0.8691 | 0.8728 |
| Alpha Readiness | 0.8762 | 0.9052 | 0.9171 |
| Alpha Maturity | 0.8368 | 0.8761 | 0.8847 |
Dataset Comparison
The table below shows how each metric changes as inclusion criteria are relaxed. The Δ columns show the difference from Conservative Clean (the primary analysis dataset) to each progressively less restrictive dataset. Small deltas confirm that findings are not artifacts of a particular data cleaning strategy.
| Metric | Conservative N=79 | Δ Flexible Clean N=116 | Δ Prolific Accepted N=200 |
|---|---|---|---|
| Barrier Grand Mean | 2.8139 | -0.0248 | -0.0495 |
| Barrier SD | 0.6309 | +0.0301 | +0.0584 |
| Readiness Grand Mean | 3.0483 | +0.0282 | +0.0878 |
| Readiness SD | 0.5829 | +0.0410 | +0.0845 |
| Maturity Grand Mean | 3.0318 | +0.0298 | +0.1215 |
| Maturity SD | 0.7087 | +0.0414 | +0.0768 |
| B-R Correlation | -0.5016 | -0.0156 | +0.1202 |
| B-M Correlation | -0.2451 | -0.1202 | -0.0705 |
| R-M Correlation | 0.5877 | +0.0944 | +0.1317 |
| Alpha Barriers | 0.8572 | +0.0119 | +0.0156 |
| Alpha Readiness | 0.8762 | +0.0290 | +0.0409 |
| Alpha Maturity | 0.8368 | +0.0393 | +0.0479 |
Deltas highlighted in amber exceed 0.05 scale points. Correlation and reliability differences of this magnitude are expected when adding noisier data but do not change substantive conclusions.
Interpretation
The sensitivity analysis reveals remarkable stability across inclusion criteria. Key observations:
- Construct means are highly stable, with differences of less than 0.10 scale points between the most and least restrictive samples.
- Standard deviations increase slightly with less restrictive samples, as expected when including noisier data.
- Correlations between constructs are directionally consistent across all samples (Barriers negatively correlated with Readiness and Maturity; Readiness and Maturity positively correlated).
- Cronbach’s alphavalues remain excellent (> 0.84) across all samples, indicating robust internal consistency regardless of inclusion criteria.
These findings demonstrate that the core results of the Technology Adoption Barriers Survey are not artifacts of a particular data cleaning strategy. Whether using the strictest quality filters (Conservative Clean, N=79) or the broadest CRP dataset (Prolific Accepted, N=200), the same substantive conclusions hold.
Related Pages
- CRP 2026 Descriptive Statistics - Detailed means, SDs, and correlations with interpretation
- CRP 2026 Scale Reliability - Cronbach’s alpha analysis with references
- CRP 2026 Sample & Demographics - Participant demographics for CRP sample groups
Open Data & Reproducibility - download the dataset and reproduce these results yourself. ← Back to CRP 2026 Results