Let's be direct: the danger isn't the water, it's "I think I can." Freediving does carry real risks, but it is a sport that rules can keep in check—when accidents happen, it is usually not about diving deep, but about bypassing the rules.
The danger comes mainly from four places:
Shallow-water blackout (BO) and loss of motor control (LMC) caused by hypoxia. These most often happen during the last 10 meters of the ascent and in the few seconds after surfacing—when the oxygen partial pressure in the body is at its lowest. So after surfacing you must complete the surface protocol, and your buddy must keep watching you for at least 30 seconds.
Ear barotrauma. Pinching your nose and blowing too hard, or forcing yourself down when uncomfortable, can injure the eardrum and middle ear. If you can't equalize, ascend. There is no room for discussion on this.
Hyperventilation. It makes you "feel" you can hold your breath longer, at the cost of wiping out the warning signs of hypoxia—like removing the fire alarm and then lighting a fire.
Diving alone. This is the common factor in many fatal accidents: with no buddy, if you pass out, no one is there to bring you back to the surface.
Making it safe is actually not complicated: always have a buddy with rescue skills present; one up, one down, taking turns to dive; build depth targets gradually; if any one of weighting, line, safety team or medical plan is not in place, don't dive to that depth; if you're not in good condition, don't get in the water.
So the answer is: freediving with no rules is dangerous; properly regulated freediving is a sport you can safely enjoy for the long term. Rules are not formalism—they are the only seatbelt this sport has.