I'd ask where you got this assumption from. Memories are exactly what define behavior and personality in itself. Biological instincts/evolution are memory passed down from previous generations. Before you even have your own consciousness, there's a form of memory already implanted inside of you that controls your behavior/actions baseline.
(small pre-concept)
Even regarding free will debates, there are arguments for predestination as arguments for having freedom of choice are different than arguments from freedom of manifestation. As humans, we do not manifest things in a random manner at all and what is the importance of "free will" if nothing manifests from it. Even something as simple as picking a random number has various micro thoughts inside of it such as "is this number too small", "is this number too common", "let me avoid numbers like this". It is the same process for computers when they generate randomness, they have a set of guidelines to mimic RNG that isn't actually RNG.
(main concept)
When you get into discussing the micro mechanics of how your thought works in itself, memory is the baseline of that. Memory determines what you find important, what you like, what you are afraid of, what you value, etc. You cannot like something you never experienced, you cannot value something you do not even know exists, you cannot think in a way that hasn't been shown to you directly or indirectly. We look for patterns and consistency in our existence baseline. That is why all human behavior is incredibly predictable with enough information and when someone loses enough of their memory, they cease to act in the same capacity and exhibit a different personality entirely from what they had before. A "person" that has no memory nor capacity for future memory cannot function in the world and is merely a husk that will eventually die because it won't even remember where or how to get basic elements of survival, it can't perform basic maintenance on itself, let alone to form goals or a purpose of life.
The idea that someone can be implanted with memories and behave "uniquely" than said memories' influence is a failure to understand the core of a thought process. I am not going to have memories of walking into a swamp and being attacked by snakes, and say "well, it's only a memory, let me walk into a swamp anyway, surely I won't be attacked by snakes." Memories that we consider real and not dreams, are not naturally ignored. That is how we make predictions and do experiements, to find variations from those baseline observations and assumptions that we create.