I am biased to remember the times when I recall something disturbing, or forget something necessary (information for exams). The human brain is a bit messy; it was adapted from Darwinism. It cannot remember everything it wants to remember, or forget everything it wants gone. This is a pitfall of idealism. Rather, as the brain usually can still remember a lot of stuff selectively, the times when something unexpected happens can be shocking.
Rather, recalling unwanted things, which itself is not a wanted behavior. It can theoretically lead to an infinite loop. Detaching and being neutral can help in such a case, because with Determinism, the brain will just think of what it thinks, and you don't have to do anything but let the thoughts flow away... but this ironically can make... the... reverse.. happen.