Two agents can have the same source code and optimise different utility functions

I be­lieve that it is pos­si­ble for two agents to have the ex­act same source code while (in some sense) op­ti­mis­ing two differ­ent util­ity func­tions.

I take a util­ity func­tion to be a func­tion from pos­si­ble world states to real num­bers. When I say that an agent is op­ti­mis­ing a util­ity func­tion I mean some­thing like that the agent is “push­ing” its en­vi­ron­ment to­wards states with higher val­ues ac­cord­ing to said util­ity func­tion. This con­cept is not en­tirely un­am­bigu­ous, but I don’t think its nec­es­sary to try to make it more ex­plicit here. By source code I mean the same thing as ev­ery­one else means by source code.

Now, con­sider an agent which has a goal like “gain re­sources” (in some in­tu­itive sense). Say that two copies of this agent are placed in a shared en­vi­ron­ment. Th­ese agents will now push the en­vi­ron­ment to­wards differ­ent states, and are there­fore (un­der the defi­ni­tion I gave above) op­ti­mis­ing differ­ent util­ity func­tions.