Good point, poor choice of words on my part. I meant for example:
UserMessage: Hello GPTMessage: Hello! How may I assist you? UserMessage: I need directions to the mall.
Is actually something like
A = [“Hello”], B = [“Hello”, “Hello! How may I assist you?”, “I need directions to the mall”]
And if you send A to the model with the same params you’ll get the same probability distribution back for every trial. (Same actual response for temperature==0, in practice mostly the same response for temperature >0). Ditto for B. The context is not independent but the model responses are independent of each other.
Good point, poor choice of words on my part. I meant for example:
UserMessage: Hello
GPTMessage: Hello! How may I assist you?
UserMessage: I need directions to the mall.
Is actually something like
A = [“Hello”], B = [“Hello”, “Hello! How may I assist you?”, “I need directions to the mall”]
And if you send A to the model with the same params you’ll get the same probability distribution back for every trial. (Same actual response for temperature==0, in practice mostly the same response for temperature >0). Ditto for B. The context is not independent but the model responses are independent of each other.