from what i've been reading lately, it seems that the soul fragments, which retreated during trauma, often go to a place that is far more "friendly" than real-life has been.
often the shaman doing the retrieval finds that the soul fragment refuses to come back, and so has to be persuaded, and even tricked, into coming back.
even when it has returned, it needs to be nurtured by the individual it was once a part of. retaining the memory of the original trauma, it may be unsure of it's safety, back in the real world.