Figure 4.
Illustration of six trials of the full selective attention model in action. At t = 1 the model randomly picks one dimension-feature (d*, f *) pair, in this case (shapes, triangle), to attend to. Then the three stimuli {s}t+1 = {st+1(1), st+1(2), st+1(3)} are observed and ε-greedy choice probabilities are computed (in this example ε = 0.3). Next a stimulus is chosen, ct+1, and the reward outcome, rt+1, observed. The observed reward is used to update the log likelihood ratio LR(n). For clarity, the black bar indicates the initial log likelihood ratio and the orange bar represents the updated log likelihood ratio that takes into account the new reward value. If the log likelihood ratio crosses the threshold (dashed line), as occurs here on trial 4, the agent switches its attention randomly (in this case, to (textures, plaid)) and resets its log likelihood ratio to the initial value (black bar at trial 5).