Background/Objective: Available evidence on infant body composition is limited. This study aimed to investigate factors associated with body composition at 6 and 24 months. Subjects/Methods: Multicenter study with data from a 0 to 6-mo cohort (Australia, India and South Africa) and a 3 to 24-mo cohort (Brazil, Pakistan, South Africa, and Sri Lanka). For the 0–6-mo cohort, body composition was assessed by air-displacement plethysmography (ADP) and for the 3–24-month cohort by the deuterium dilution (DD) technique. Fat mass (FM), fat-free mass (FFM), FM index (FMI), and FFM index (FFMI) were calculated. Independent variables comprised the Gini index of the country, maternal and infant characteristics, and breastfeeding pattern at 3 months. For the 3–24-mo cohort, breastfeeding, and minimum dietary diversity (MDD) at 12 months were also included. Crude and adjusted analyses stratified by sex were conducted by multilevel modelling using mixed models. Results: At 6 months, every 1 kg increase in birth weight was associated with an increase of 0.716 kg in FFM and 0.582 kg/m2 in FFMI in girls, whereas in boys, the increase was of 0.277 kg in FFM. At 24 months, compared to those weaned before 12 months, girls still breastfed at 12 months presented a decrease of 0.225 kg in FM, 0.645 kg in FFM and 0.459 kg/m2 in FFMI, and in boys the decreases were of 0.467 kg in FM, 0.603 kg in FFM and 0.628 kg/m2 in FFMI. Conclusion: Birth weight and breastfeeding are independent predictors of body composition in early life, irrespective of sex.