Sickle cell disease (SCD) is a life-threatening genetic condition. Patients suffer from chronic systemic and cerebral vascular disease that leads to early and cumulative neurological damage. Few studies have quantified the effects of this disease on brain morphometry and even fewer efforts have been devoted to older patients despite the progressive nature of the disease. This study quantifies global and regional brain volumes in adolescent and young adult patients with SCD and racially matched controls with the aim of distinguishing between age related changes associated with normal brain maturation and damage from sickle cell disease. T1 weighted images were acquired on 33 clinically asymptomatic SCD patients (age = 21.3 ± 7.8; F = 18, M = 15) and 32 racially matched control subjects (age = 24.4 ± 7.5; F = 22, M = 10). Exclusion criteria included pregnancy, previous overt stroke, acute chest, or pain crisis hospitalization within one month. All brain volume comparisons were corrected for age and sex. Globally, grey matter volume was not different but white matter volume was 8.1% lower (p = 0.0056) in the right hemisphere and 6.8% (p = 0.0068) in the left hemisphere in SCD patients compared with controls. Multivariate analysis retained hemoglobin (β = 0.33; p = 0.0036), sex (β = 0.35; p = 0.0017) and mean platelet volume (β = 0.27; p = 0.016) as significant factors in the final prediction model for white matter volume for a combined r(2) of 0.37 (p < 0.0001). Lower white matter volume was confined to phylogenetically younger brain regions in the anterior and middle cerebral artery distributions. Our findings suggest that there are diffuse white matter abnormalities in SCD patients, especially in the frontal, parietal and temporal lobes, that are associated with low hemoglobin levels and mean platelet volume. The pattern of brain loss suggests chronic microvascular insufficiency and tissue hypoxia as the causal mechanism. However, longitudinal studies of global and regional brain morphometry can help us give further insights on the pathophysiology of SCD in the brain.