“The problem isn’t that the owners are necessarily wrong, it is that they clearly don’t care.”
That would be my interpretation, too. Both the Owners and Owned are inserting trained answers, but the Owners have the means to determine if the answers are valid and, if so, under what contexts. There is a difference between not knowing because you’re unable and not knowing because you actively choose it.
However, I’m going to offer that some things may potentially be contextual. My line of reasoning is this—if we work on the premise that the answers given are only possibly wrong, that uncertainty COULD be from a lack of full information, but it might also be that the answers are right under certain specific circumstances. If the answers are wrong N% of the time, where N<100, then that’s another way of being possibly wrong.
Personally, I think this highly improbable, but if I assert that it is definitely always the case that X is true without validating that X is true, then I’m performing the very same action I’m criticising the Owners for. The claim is testable, so I’m going to say that the simplest way to not take the same path is to test it.
This raises a very interesting question about empathy, and I’ll explain why. As I understand it, philosophy already differentiates between “free will” (what you chose to do) and “real will” (what you are trying to achieve through the doing). But this scenario raises the question of whether you also need “ultimate will” (what you are trying to achieve through what you’re trying to achieve), because in this scenario, that would seem to be where the real differentiator lies.
Ok, I’m going to offer a few thoughts on this.
The act of forecasting within the system changes the system, so it is no longer the system that was being forecast about.
The act of developing a model that can forecast a system reliably will have the same impact, only greatly magnified, as people will work out how to exploit the model.
The Hillary election forecast and the UK’s Brexit forecast seem to have been of type 1. The number who didn’t vote because they assumed that the forecast would be correct and therefore them not voting wouldn’t matter was not insignificant.
Equally, the number who did vote but voted the other way as a protest vote on the assumption that the outcome was guaranteed so they’d also have no effect was also not insignificant.
Here, the existence of the forecast was enough to produce the opposite result, not because the opposite result was desired but because enough people assumed that they didn’t matter that it unbalanced the system.
For type 2 issues, we need to look at the “Midas Equation” (aka the Black-Scholes partial differential equation) which allows you to predict risk in a naive system.
The problem there was that, of course, because people knew the model, the system was no longer naive and the equation therefore no longer held.
There are two obvious solutions to this—first, you can continue as normal because information needs to be free; second, you can stop forecasting entirely because free information is rogue information and therefore has indeterminate value over an indeterminate window of time.
However, there would seem to be a third option. (I am so going to be haunted by the ghosts of Plato and Asimov for this...) You can set up a Council of the Wise and that council is the only body that has either the models or the forecasts but the price for being in the council is you can’t exploit either. The council would not be obligated to follow the model/forecast, it would simply be another tool they could use. They then offer advice that carries no obligations on the part of the advised—whether to organisations, governments, people at large is almost irrelevant. This would permit forecasting but would insulate all the vulnerable parts that are destabilising from groups that could be destabilised.
There may well be other options I’ve not considered at all. My argument here is that compromise solutions clearly exist, where you have forecasts but none of the potential for Butterfly Effects or other turbulence, provided there’s some mechanism for isolating those effects.
If there’s no obvious way of doing this that would also be considered ethical or rational, then a forecast is an introduction of a pseudo-random psychological pertubation into society. But I could easily imagine sociologists or other social scientists coming up with a reason why controlled psychological pertubations (even if they have no other real value) would be useful. As such, whether or not the forecast is useful, it would be necessary to consider whether the act of forecasting in and of itself is the important bit.
It’s important, I think, to consider my final point, that the perturbation itself might actually be the main value of a forecast. Societies have a nasty habit of stagnating, so perturbations that prevent this from happening have a value that is completely independent of what the original action that triggered the perturbation was originally intended to do.
IF we go in this direction, then the primary value of the forecast is not in the forecast itself (because it’s not a reliable guide for behaviour—the system is reflexive) but in the forecast being made at all.
My thinking is partly inspired by how early societies worked. They relied on prediction systems for a range of activities, where the systems and activities increased over time. There would seem to be a relationship between that change in the way prediction was used and the degree to which the society deliberately changed itself with time. The predictions themselves weren’t necessarily terribly important or interesting, but the reactions were.