Behavior-based safety (BBS) systems have shown to be effective in decreasing injury rates in industry. Typical BBS systems use a peer-to-peer observation via a critical behavior checklist (CBC) to measure the rates of safe and at-risk behavior for observed employees. A key feature of these programs is quality participation. The current study sought to test if using a shorter CBC with a few behaviors (5-8) would increase quality participation in BBS systems. No such increases were observed. Instead, results indicated that many employees did not utilize the new eight-behavior CBC. Interviews with managers on each vessel revealed that implementation methods on each vessel varied considerably. Furthermore, many employees relied on memory, rather than using a CBC. The implications of inconsistent implementation and memory reliance are discussed