That is correct, the panel is riveted in on that one and you will have to carefully drill the heads out, then tap the shank through with a punch. The panel will remove and those are 1/8 aluminum rivets on that one, nothing fancy.
The issue is probably more in the wire connections as those Carline switches have a very high dependability and seldom fail in that situation.