A lot of people have requested this. Not sure how it would be visually/functionally implemented, but IMO it's worth making a ticket due to the volume of requests.

Should ideally be per-server, possibly need to implement "clientside server settings" menu for this and other such things (mute whole server, notification settings, etc). When enabled should just remove the channels from the list entirely.