A self-regulatory organization is an industry body given legal authority to police its members. In the United States finra supervises broker-dealers, the nfa supervises futures firms, and every exchange is itself an SRO for the firms that trade on it. Each one writes rulebooks, examines members, and can fine, suspend, or expel them.
SROs sit under a government regulator. The sec must approve FINRA rule changes and hears appeals of FINRA discipline; the cftc plays that role for the NFA. So an SRO rule is not softer than a government rule in practice, and breaking one can end a career.
For a retail trader the practical effect is that your broker answers to at least three bodies at once: its exchange, its SRO, and its statutory regulator. When you file a complaint, the SRO is usually the first door.
Related: finra, sec, nfa, finra-arbitration, cftc