Scale Reliability

Last updated: May 29, 2026, 7:40 AM EDT

Scale reliability is assessed using (α), the most widely used measure of internal consistency. A coefficient of α ≥ 0.70 is generally considered acceptable for research purposes (Nunnally & Bernstein, 1994), while values above 0.80 indicate good to excellent reliability.

All three TABS constructs demonstrate excellent internal consistency, with every Cronbach’s alpha exceeding 0.82 across all five sample definitions. This indicates that the survey items within each construct are measuring the same underlying factor reliably, regardless of which inclusion criteria are applied.

Cronbach’s Alpha by Construct and Sample

Construct	Conservative Clean N=140	Flexible Clean N=231	Prolific Accepted N=440	All V2 Finished N=675	All V2 N=782
Barriers	0.8664	0.8859	0.8897	0.9089	0.9089
Readiness	0.8720	0.9123	0.9128	0.9267	0.9267
Maturity	0.8264	0.8864	0.8855	0.8924	0.8924

Bootstrap Confidence Intervals on Alpha

Point estimates of Cronbach's alpha can fluctuate with sample composition. Bootstrap percentile confidence intervals (Efron, 1979) quantify that uncertainty: if the entire CI sits above the .70 acceptability threshold (or .80 for "good"), the scale's reliability is well-supported across plausible resamples of the data.

Construct	Bootstrap mean	95% CI lower	95% CI upper	CI width
Barriers	0.8659	0.8322	0.8928	0.0606
Readiness	0.8704	0.8321	0.9002	0.0681
Maturity	0.8239	0.7719	0.8655	0.0936

Bootstrap percentile CIs computed from 1,000 resamples (Efron, 1979). Narrow CIs that do not cross the .70 acceptability threshold provide stronger evidence of scale reliability than a single point estimate.

Alpha if Deleted

For each construct, alpha-if-deleted reports the value Cronbach's alpha would take if a given item were removed from the scale. Items whose removal would noticeably raise alpha are candidates for revision in a future version of the instrument; items whose removal would lower alpha are well-aligned with the construct.

Barriers (base alpha = 0.8664)

No items would raise alpha if removed. Closest call: Vendor/Partner Difficulty (alpha if deleted = 0.8646, change -0.0018). The scale is robust to single-item removal.

Readiness (base alpha = 0.8720)

1 item would raise alpha if removed. None of these are large enough to motivate dropping the item, but they are the candidates a future revision would target first.

Budget Adequacy: alpha if deleted = 0.8727 (change +0.0007)

Maturity (base alpha = 0.8264)

No items would raise alpha if removed. Closest call: Tech Risk & Resilience (alpha if deleted = 0.8185, change -0.0078). The scale is robust to single-item removal.

Reliability by Demographic Group

Cronbach's alpha computed within each organizational subgroup. Comparable values across SMB vs Enterprise and For-Profit vs Non-Profit / Government provide evidence that the scales work reliably regardless of organizational context, supporting cross-segment comparisons in downstream analyses.

Group	N	Barriers	Readiness	Maturity
SMB (<1000 emp)	83	0.8861	0.8754	0.8433
Enterprise (>=1000 emp)	57	0.8301	0.8541	0.7960
For-Profit	99	0.8694	0.8870	0.8134
Non-Profit / Gov	41	0.8652	0.8234	0.8543

Cronbach's alpha computed independently within each demographic group. Stable values across groups demonstrate that the scales perform reliably regardless of organizational size or sector.

Interpretation

Several key observations emerge from the reliability analysis:

All alphas exceed 0.82, well above the commonly cited 0.70 threshold, indicating excellent internal consistency across all constructs and samples.
Reliability coefficients are stable across sample definitions, meaning the scales perform consistently whether computed on the strictest clean sample or the full dataset.
Alpha values tend to increase slightly with larger sample sizes, which is expected behavior and does not indicate measurement problems.
The Readiness construct shows the highest alphas (up to 0.9267), while Maturity shows the lowest (still above 0.8264), which may reflect the smaller number of items in the Maturity scale (9 vs. 18-19).

The consistency of these results across five sample definitions demonstrates that the TABS scales are reliable instruments regardless of the inclusion criteria applied. This robustness is critical for supporting any downstream inferential analyses.

References

Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297-334.
Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.). McGraw-Hill.

Descriptive Statistics - means, SDs, and correlations for each construct
Sensitivity Analysis - all metrics across five sample definitions
Data Quality Pipeline - how the samples are defined and validated
← Back to Results Overview