Fiber clustering has been used to improve the configuration of neuronal fiber tracts in white matter by labeling erroneous fibers in the tract. In this study, we propose a robust differential distance fiber similarity metric as the key component in accurate FC. Compared with the commonly used averaged distance fiber similarity metric, our defined differential distance fiber similarity metric measures differential distances instead of averaged distances between point pairs along two fibers within a tract. To validate the performance of our proposed approach, it was compared with the averaged distance fiber similarity metric on simulated fiber tracts and human data. The results showed that our FC method was superior to the averaged distance fiber similarity metric at correctly labeling the fibers in an individual tract.