Measurements of negative ion density after initial laser photodetachment show good agreement with a ballistic theory of negative ion return early in time, but show a much slower return than predicted by theory late in time. Here we develop an explanation based on the establishment of a hole in the negative ion velocity space, which would then be filled in by collisions on a slower time scale.