I'm not completely sure but it has to do with what powers we have. Those of us with orange collars have no magic or supernatural powers, wheras those with Violet do. But this isn't always the case.

[There was a few more moments of quiet while he took the completely destroyed pieces to the bin, before returning.] Early last year there was an experiment, or accident but I think it was most likely the former, where the liquid in our collars was mixed up. Mine. [He gestured to his orange collar] Turned Violet, I felt like there was a great weight on me, I couldn't move, talking was difficult and I feared that my body would just stop breathing.

Those who switched the other way reported that they felt lightheaded and their powers went out of control. So I believe there is more to it than just affecting and controlling any magic we may have.