Many studies have shown that shorter sleep duration in childhood is associated with higher body mass index (BMI), and have proposed that it is due to an effect of sleep on adiposity. There is little evidence about the association of sleep with fat‐free mass. This study examined the association between child's sleep duration at age 3 years and fat and fat‐free mass at 4 years of age in a prospective cohort study of 302 boys and 285 girls. Study participants were taking part in the Southampton Women's Survey, a longitudinal study of mothers and children from preconception onwards. Total sleep duration at age 3 years was derived from parental report of night sleep and nap duration. Body composition was assessed by Dual‐energy X‐ray Absorptiometry (DXA) at 4 years. Mean total sleep duration was 11.5 hours. In linear regression analyses, adjusted for potentially confounding factors (maternal educational attainment, prepregnancy BMI, smoking during pregnancy, child's gestational age at birth, age at DXA, sex, age last breastfed, dietary quality at 3 years, TV watching and hours actively on the move and parental social class), shorter sleep in hours was associated with higher BMI (kg/m2) [β: −0.2340, 95% confidence interval (CI): −0.373 to −0.096], a greater fat mass index (kg) (β: −0.1182, 95% CI: −0.218 to −0.018) and a greater fat‐free mass index (kg) (β: −0.100, 95% CI: −0.185 to −0.015). Previous research suggested that the association between shorter sleep and higher body mass index is due to an effect on adiposity. Our findings are novel, suggesting that the relationship between sleep and BMI is also determined by an effect on muscle.