… to actually Use Truth to Do Desirable Things you need to actually to Focus On Truth For It’s Own Sake, and yes, this is a bit contradictory, and I’m not 100% sure how to resolve the contradiction
I do not think there is any way to resolve the contradiction. It seems clear to me that just as no man may serve two masters, no organization may serve two goals. “What you are willing to trade off, may end up traded away”. And ultimately, you will sacrifice your pursuit of truth, if what you are actually pursuing is something else—because there will come a time when your actual goal turns out (in that situation, at that time, in that moment) to not be best served by pursuing Truth, for its own sake or otherwise.
And then your Citadel will not even be a Citadel of Truth And Something Else, but only a Citadel of Something Else, And Not Truth At All.
I think there’s still some highly technical apparent-contradiction-resolution to do in the other direction: in a monist physical universe, you can’t quite say, “only Truth matters, not consequences”, because that just amounts to caring about the consequence of there existing a physical system that implements correct epistemology: the map is part of the territory.
To be clear, I think almost everyone who brings this up outside the context of AI design is being incredibly intellectually dishonest. (“It’d be irrational to say that—we’d lose funding! And if we lose funding, then we can’t pursue Truth!”) But I want to avoid falling into the trap of letting the forceful rhetoric I need to defend against bad-faith appeals-to-consequences, obscure my view of actually substantive philosophy problems.
I do not think there is any way to resolve the contradiction. It seems clear to me that just as no man may serve two masters, no organization may serve two goals. “What you are willing to trade off, may end up traded away”. And ultimately, you will sacrifice your pursuit of truth, if what you are actually pursuing is something else—because there will come a time when your actual goal turns out (in that situation, at that time, in that moment) to not be best served by pursuing Truth, for its own sake or otherwise.
And then your Citadel will not even be a Citadel of Truth And Something Else, but only a Citadel of Something Else, And Not Truth At All.
I think there’s still some highly technical apparent-contradiction-resolution to do in the other direction: in a monist physical universe, you can’t quite say, “only Truth matters, not consequences”, because that just amounts to caring about the consequence of there existing a physical system that implements correct epistemology: the map is part of the territory.
To be clear, I think almost everyone who brings this up outside the context of AI design is being incredibly intellectually dishonest. (“It’d be irrational to say that—we’d lose funding! And if we lose funding, then we can’t pursue Truth!”) But I want to avoid falling into the trap of letting the forceful rhetoric I need to defend against bad-faith appeals-to-consequences, obscure my view of actually substantive philosophy problems.