Are AIs more likely to pursue on-episode or beyond-episode reward?

Consider an AI that terminally pursues reward. How dangerous is this? It depends on how broadly-scoped a notion of reward the model pursues. It could be:on-episode reward-seeking: only maximizing reward on the current training episode — i.e., reward that reinforces their current action in RL. This is what people usually mean by “reward-seeker” (e.g. in Carlsmith or The behavioral selection model…).beyond-episode reward-seeking: maximizing reward for a larger-scoped notion of “self” (e.g., all mo...