Nice post! I broadly agree/endorse your thoughts, but would like to focus on a technical disagreement:
I think that the abstract concept of wealth as you discuss it doesn’t map well onto the ‘wealth’ that Garrabrant Inductors give traders. The construction algorithm given in the paper does ‘budget’ each trader, giving an upper bound on how much money that trader can lose in a given day. However, the ‘wealth’ assignments the traders get is arbitrary. More concretely, the subroutine “TradingFirm” can seemingly be built off of any computable enumeration of traders. The Inductor ‘doesn’t care’ which traders get what part of the budget; it only cares that the traders get budgeted somehow. This suggests to me that the assignment of ‘degrees of truth’ or ‘wealth’ isn’t too conceptually important for Garrabrant Induction (since any assignment works).
Ideally, we’d be able to say that Inductors ‘give’ wealth to successful traders as a way to encode trusted hypotheses, inductive biases, or priors. As it stands, however, Inductors lack the structure to justify such rich interpretations. This is related to how Inductors explicitly price in (and must compute) every trader on every day. They never truly rule out any trader based on wealth, which deprives them of the sort of cognitive structure that arises from eliminating hypotheses based on heuristics.
BRIAs are more semantically rich in this sense. For instance, the decision auction used to construct the agent disqualifies ‘broke’ hypotheses from even being computed. This makes the wealth-based dynamics you describe in the post map much better onto BRIAs. Indeed, I think that there’s valuable work to be done in modifying Garrabrant Inductors to describe acquired knowledge through the budgeting of traders. Relatedly, Garrabrant has previously written about how Inductors that can tamper with their own deductive process (through action) can get trapped in the 5 and 10. He frames this as a problem, but to me it signifies that Inductors do have some latent ability to express strong inductive biases – this is a desirable feature for descriptive notions of bounded rationality.
Nice post! I broadly agree/endorse your thoughts, but would like to focus on a technical disagreement:
I think that the abstract concept of wealth as you discuss it doesn’t map well onto the ‘wealth’ that Garrabrant Inductors give traders. The construction algorithm given in the paper does ‘budget’ each trader, giving an upper bound on how much money that trader can lose in a given day. However, the ‘wealth’ assignments the traders get is arbitrary. More concretely, the subroutine “TradingFirm” can seemingly be built off of any computable enumeration of traders. The Inductor ‘doesn’t care’ which traders get what part of the budget; it only cares that the traders get budgeted somehow. This suggests to me that the assignment of ‘degrees of truth’ or ‘wealth’ isn’t too conceptually important for Garrabrant Induction (since any assignment works).
Ideally, we’d be able to say that Inductors ‘give’ wealth to successful traders as a way to encode trusted hypotheses, inductive biases, or priors. As it stands, however, Inductors lack the structure to justify such rich interpretations. This is related to how Inductors explicitly price in (and must compute) every trader on every day. They never truly rule out any trader based on wealth, which deprives them of the sort of cognitive structure that arises from eliminating hypotheses based on heuristics.
BRIAs are more semantically rich in this sense. For instance, the decision auction used to construct the agent disqualifies ‘broke’ hypotheses from even being computed. This makes the wealth-based dynamics you describe in the post map much better onto BRIAs. Indeed, I think that there’s valuable work to be done in modifying Garrabrant Inductors to describe acquired knowledge through the budgeting of traders. Relatedly, Garrabrant has previously written about how Inductors that can tamper with their own deductive process (through action) can get trapped in the 5 and 10. He frames this as a problem, but to me it signifies that Inductors do have some latent ability to express strong inductive biases – this is a desirable feature for descriptive notions of bounded rationality.