.Google's Gary Illyes discloses Googlebot gathers web links rather than "adhering to" them, negating official records.Googlebot picks up web links rather than following them in real-time.Gary Illyes discussed this information in Google's 'Look Off The Document' podcast.Google.com's documents negates the true spider behavior.