What the Tally Hid: Screening Live-Observation Codes for Saturation, Floor Effects, and Degeneracy Before Piloting
Abstract
We set out a pre-piloting screening protocol for live-observation coding instruments (Table 1), usable in any brief structured behavioural-coding task, with a worked case study and a Monte Carlo calibration. In 15 stranger dyads a ten-code sheet produced a healthy-looking composite (M = 26.63, SD = 9.64), yet four engagement codes ran at 74-94% of their protocol maximum, three withdrawal codes were floored at 0-8%, and only three carried usable variance. Resampling the deposited counts (40,000 pilots per size), detection under the all-code conjunctive rule is 100% at every pilot size; one code (monologue) was never observed, and the all-code result is reported as a primary finding, with false positives against no-pathology nulls reaching 32.3% at N = 2 and falling to 9.3% at N = 4. In a sensitivity analysis excluding monologue and silence, detection remained high but non-monotonic (91.0% at N = 4; 87.1% at N = 6), with a closely comparable false-positive profile (6.1% at N = 4). Threshold distance and degeneracy, rather than pilot size, govern screening. The practical rule that follows is to report, for each code, its mean as a percentage of its protocol maximum, its between-unit variance, and its signed distance to the nearest threshold, rather than a recommended pilot N. The protocol and its two generalizable principles are offered as provisional and unregistered, pending independent replication on a second bin-level dataset. All counts derive from a single non-blinded…
Read the paper
Related studies
- Regulation of nucleolus assembly by non-coding RNA polymerase II transcripts
- For a Healthy Histone Code, a Little SUMO in the Tail Keeps the Acetyl Away
- Discovering a binary CTCF code with a little help from BORIS
- Article Commentary: Stabilizing the Code–-Methods to Preserve RNA Prove Their Worth
- Foreword to the Special Issue for the RINTC (The Implicit Seismic Risk of Code-Conforming Structures) Project
- Commentary: Promoting Health and Well-being in Pediatric Primary Care Settings: Using Health and Behavior Codes at Routine Well-child Visits
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.