Wow, that's no optical illusion! I've never actually noticed this happen before but I just checked it and it does happen if you repeat those actions.
The first barrel doesn't decide to exit. The second one definitely decided to exit the screen because you're x pixels above it, but then decides not to exit when you start climbing down the ladder and comes back in.