Most studies reporting on the association of circulating testosterone levels with type 2 diabetes in men are of cross-sectional design. Reports on the relevance of altered testosterone levels in women are scarce. Here, we evaluate the role of low serum testosterone levels for incident diabetes in men and women in a population setting of 7706 subjects (3896 females). During a mean follow up time of 13.8 years, 7.8% developed type 2 diabetes. Significant correlations of testosterone with high density lipoprotein (HDL)-cholesterol (R = 0.21, p < 0.001), body-mass-index (R = -0.23, p < 0.001), and waist-to-hip-ratio (R = -0.21, p < 0.001) were found in men. No correlation was found with age in men; in women, the correlation was negligible (R = 0.04, p = 0.012). In men, low testosterone levels predicted high risk of type 2 diabetes, while in women this relationship was opposite. Men with low testosterone levels showed increased risk of future diabetes (hazard ratio (HR) 2.66, 95% confidence interval (CI) 1.91⁻3.72, p < 0.001 in basic model; HR 1.56 95%, CI 1.10⁻2.21, p = 0.003). In women, low testosterone levels indicated lower risk with (HR 0.53, 95% CI 0.37⁻0.77, p = 0.003), while the association lost significance in the fully adjusted model (HR 0.72, 95% CI 0.49⁻1.05, p = 0.09). Low levels of testosterone predicted future diabetes in men. A borderline opposite association was found in women.