Possible downsides are:
It's a cynical thing to say, but the rich want that divide. It's natural to want status, so I hope it's not too controversial. Point is, there's a lot of pressure in keeping things unbalanced in one way or another.See you, space cowboy.
"public sector workers who don't actually pay tax, due to said tax being funded by taxation"
Wow. You should get in touch with this fucking retard. You'd have a lot in common:
Society benefits a whole lot more from the work of the teachers who educate kids, the doctors and nurses who save lives, the waterworks guys who keep our drinking water safe, food inspectors, and pothole repairmen than it does from a bunch of worthless grifters who abuse the trust of others for their own enrichment through varied inventive forms of financial trickery.
Of the two groups which suckle the most at the teat of the state and the expense of the public - civil servants and big business - only one has people who at least earned some of their money.
Does this also extend to private sector employees paid by firms that have government contracts and private sector firms that enjoy various types of public subsidies?