Two agents can have the same source code and op­tim­ise dif­fer­ent util­ity functions

I be­lieve that it is pos­sible for two agents to have the ex­act same source code while (in some sense) op­tim­ising two dif­fer­ent util­ity func­tions.

I take a util­ity func­tion to be a func­tion from pos­sible world states to real num­bers. When I say that an agent is op­tim­ising a util­ity func­tion I mean some­thing like that the agent is “push­ing” its en­vir­on­ment to­wards states with higher val­ues ac­cord­ing to said util­ity func­tion. This concept is not en­tirely un­am­bigu­ous, but I don’t think its ne­ces­sary to try to make it more ex­pli­cit here. By source code I mean the same thing as every­one else means by source code.

Now, con­sider an agent which has a goal like “gain re­sources” (in some in­tu­it­ive sense). Say that two cop­ies of this agent are placed in a shared en­vir­on­ment. These agents will now push the en­vir­on­ment to­wards dif­fer­ent states, and are there­fore (un­der the defin­i­tion I gave above) op­tim­ising dif­fer­ent util­ity func­tions.