Agreement comparison with respect to data with good and poor sleep efficiency.

Agreement comparison with respect to data with good and poor sleep efficiency.</p

Hypnograms of the subject no. 4.

The hypnograms scored by fully manual scoring (scorer 2) (A), fully automatic staging (B) and the HCSS system (C).</p

Evaluation of the HCSS system.

(A) The agreement in overall, high-reliability and low-reliability epochs, along with the kappa coefficient between the manual scorings and the HCSS system collaborated scorings. (B) The average of the scoring time for one subject spent in manual and HCSS groups. Percentage of reduced manual scoring time with the assistance of the HCSS system; OA: overall, HR: high-reliability, LR: low-reliability.</p

Comparison of agreement between different automatic scoring methods.

Comparison of agreement between different automatic scoring methods.</p

The hypnogram and SCF values of subject no. 5.

The hypnograms scored by gold standard (A) and the automatic staging system (B). The value of feature SCF (C). The red lines indicate disagreement between the expert and the automatic scoring system.</p

The hypnogram and SCD value of PSG from subject no. 3.

The hypnograms scored by gold standard (A) and the automatic staging system (B). The value of feature SCD (C). The red lines indicate disagreement between the expert and the automatic scoring system.</p

Reliability analysis examples.

Disagreements can be detected by using the SWR (A), SCD (B) and SCF (C) features.</p

The architecture of the voting process.

According to the value of the SWR, SCD, and SCF features, the voting process determines a scored epoch as a high reliability or low reliability.</p

The flow chart of the human-computer collaborative sleep scoring system.

The system consists of (A) fully automatic scoring, (B) reliability analysis, and (C) human-computer collaboration.</p

Histogram of 12 sleep features in Wake, N1, N2, N3, and REM stages of 30 PSG data.

The X-axis represents the normalized feature values and the Y-axis represents the number of epochs. Features are power: (A) 0–30 Hz EEG, (B) 0–30 Hz EMG; power ratio: (C) 0–4 Hz / 0–30 Hz EEG, (D) 8–13 Hz / 0–30 Hz EEG, (E) 22–30 Hz / 0–30 Hz EEG; power (F) 0–4 Hz EOG; spectral frequency: (G) 0–30 Hz mean frequency EEG, (H) 0–30 Hz mean frequency EMG; duration ratio: (I) alpha ratio EEG, (J) Spindle ratio EEG, (K) SWS ratio EEG; amplitude: (L) mean amplitude EMG.</p