They are saying that it's important for his future role that he is seen to be politically neutral, but that it doesn't matter if that's simply an inaccurate illusion. It's not, apparently, the holding of highly partisan views which makes him non-neutral, nor is it the sharing of those views with ministers in forthright terms. It's not even whether we know that he's written such letters or not which damages his neutrality - it's only the publication of his actual words which does that. 'Justice doesn't need to be done, it only needs to be seen to be done' would be an odd mantra for a minister to adopt, yet that seems to be a parallel.
From a more common-sense and everyday perspective than that possessed by government ministers - past and present; apparently Labour were also consulted before the decision was taken - it's surely the writing of the letters which damages any pretence at neutrality, not their publication.
Some years ago, I worked on an IT conversion project which was officially going to be 'transparent to the users'. That meant, said one of my colleagues, that 'the users will see right through it'. It seems to me that the transparency of Charles' neutrality fits that definition quite nicely.