If you want to consider probabilities other than epsilon and 1 - epsilon then the distinction becomes: setting up and approximating the solution to the right Bellman equation is the problem stage; carrying out the indicated actions is the task stage.
If you want to consider probabilities other than epsilon and 1 - epsilon then the distinction becomes: setting up and approximating the solution to the right Bellman equation is the problem stage; carrying out the indicated actions is the task stage.