Purpose: Sufficient 25-hydroxyvitamin D (25(OH)D) concentrations might prevent a decline in physical performance, and are considered important for the prevention of frailty. This study investigates the association of serum 25(OH)D concentration with physical performance and frailty status in Dutch older adults. Methods: This cross-sectional study included 756 men and women, aged ≥ 65 years. Serum 25(OH)D concentration and frailty status (Fried criteria) were assessed in the total population. Screening for frailty status included functional tests of gait speed and hand grip strength. In a subgroup (n = 494), the Timed Up and Go test (TUG) and knee-extension strength were measured. Associations of serum 25(OH)D status with physical performance were examined by multiple linear regression. Prevalence ratios (PR) were used to quantify associations between serum 25(OH)D deficiency (< 50 nmol/L) and frailty. Results: In total, 45% of the participants were vitamin D deficient. Participants with vitamin D status < 50 and 50–75 nmol/L had significantly lower scores on the TUG and gait speed test, compared to participants with vitamin D status > 75 nmol/L. No significant associations with serum 25(OH)D concentrations were observed for handgrip strength or knee-extension strength. Participants with serum 25(OH)D status < 50 nmol/L were about two times more likely to be frail compared to participants with serum 25(OH)D status ≥ 50 nmol/L. No significant associations were observed between the pre-frail state and serum 25(OH)D status. Conclusion: In this study, serum 25(OH)D concentrations were significantly associated with frailty status and measures of physical performance, including gait speed and TUG, but not with strength-related outcomes.