Sea ice drift monitoring is important for climate change analysis, vessel navigation, and offshore maritime safety operations. The current methods of synthetic aperture radar (SAR) sea ice drift tracking are based on SAR intensity image analysis. However, intensity images are easily influenced by noise and have a high probability of mismatching and low matching accuracy for sea ice drift detection. Considering the above problems, this paper attempts to use the SAR sea ice texture features to enhance the drift detection performance. First, the enhancement performance of eight texture features for pattern matching in sea ice drift detection is analyzed, and the optimal texture feature is selected. Then the effects of sea ice type, incident angle, and resolution on the enhanced performance of texture features are analyzed. The results show that the mean value is the optimal texture feature, and compared with the SAR intensity image analysis approach, the proposed method improves the pattern matching accuracy by about 7%.