In the canonical version of events, he's only 3 years old when that takes place, just before the destruction of his home planet in 739. His year of birth is either 736 on the planet Vegeta in the manga, or 737 if we're going by the anime. Son Goku was born with the name Karakot, ostensibly sent to destroy Earth as an infant by his parents.
Essentially, there is a coherent timeline that runs through Dragon Ball, Dragon Ball Minus, and Dragon Ball Z – albeit with a few variations at certain points.