I've been saying for a while (probably on this thread already) but for me the simple fix is to punish all handball offences with an indirect free kick, with the exception of when it's a clearly deliberate act (eg. Suarez v Ghana).
When a team are getting a penalty for the merest flick off a defender's arm, the sanction is far heavier than it should be.
(Also under current VAR stipulations it would mean most handballs wouldn't be pulled up by the video refs).