A Feasibility Study of Knee Joint Semantic Segmentation on 3D MR Images
-
Abstract: The segmentation of knee joint is of great significance for diagnosis, guidance and treatment of knee osteoarthritis. However, manual delineation is time-consuming and labor-intensive since various anatomical structures are involved in the 3D MRI volume. Automatic segmentation of the whole knee joint requires no human effort, additionally can improve the quality of arthritis diagnosis and treatment by describing the details more accurately. Existing knee joint segmentation methods in the literature focus on only one or few structures out of many. In this paper, we study the feasibility of knee joint segmentation on MR images based on neural networks and deal with the following challenges: (1) end-to-end segmentation of 15 anatomical structures, including bone and soft tissue, of the whole knee on MR images; (2) robust segmentation of small structures such as the anterior cruciate ligament, accounting for about 0.036% of the volume data. Experiments on the knee joint MR images demonstrate that the average segmentation accuracy of our method achieves 92.92%. The Dice similarity coefficients of 9 structures were above 94%, five structures were between 87% and 90%, and the remaining one was about 76%.
-
Keywords:
- deep learning /
- semantic segmentation /
- neural networks /
- MRI /
- knee osteoarthritis
摘要: MR三维图像的全膝自动分割对膝骨关节炎疾病的诊断、指导和治疗具有重要意义。然而，膝关节的三维MR图像中涉及多种多样的解剖结构，人工勾画费时耗力；全膝自动分割不但节省人力，且可以通过更准确的细节勾画来提高关节炎的诊疗质量。现有的膝关节分割方法只关注众多解剖结构中的一个或几个结构，无法提供全膝分割的结果。本文研究基于三维神经网络的全膝分割方法，并致力于应对以下挑战：①在三维 MR图像上对包括骨骼和软组织在内的全膝15个解剖结构进行端到端分割；②前交叉韧带等小结构的鲁棒分割，前交叉韧带仅占全膝体积的0.036％ 左右。在基于脂肪抑制三维各向同性中等权重VISTA序列的膝关节MR图像上，验证本文方法的平均分割精度为92.92％，其中9种结构的Dice相似系数在94％ 以上，5种结构在87％～90％ 之间，剩余1种结构在76％ 左右。 -
-
![]()
Figure 1. Histogram of voxels per class. The background occupies about 88.62%, the femur, tibia, cartilage and fibula occupy a higher percentage of voxels, while other soft tissues occupy few percentages of voxels. The anterior cruciate ligament only accounts for 0.036%. The anatomical structures of the whole knee vary in volume greatly
![]()
Figure 2. Overview of the SegResNet network architecture. The encoder network down-sample four times, and the residual block amounts of each stage are 1, 2, 2, 2, 4
![]()
Figure 3. (a) Whole volume inference using scan window, (b) Filter kernel of average smoothing strategy, (c) Filter kernel of Gaussian smoothing strategy, (d) A new filter kernel and smoothing strategy designed to eliminate artifacts and speed up inference. All kernels of 2D cases are shown for convenience
![]()
Figure 8. Segmentation results. From left to right are: bone, quadriceps tendon and patellar tendon, collateral ligaments, cartilage, meniscus & cruciate ligaments, and whole segmented knee joint
Table 1 Comparison of different inference strategies
Average smoothing Gaussian smoothing[22] Our method Inference time/s 36 126 38 DSC 0.9290 0.9292 0.9292 Prediction of tibia 
下载: 导出CSV
Table 2 Comparison of kernel parameters
$ s $/M $ \sigma $/M DSC 3/4 1/32 0.9083 3/4 1/64 0.8961 3/4 1/16 0.8966 1/2 1/32 0.8960 7/8 1/32 0.8967
下载: 导出CSV
-
[1] GUPTA S, HAWKER G A, LAPORTE A, et al. The economic burden of disabling hip and knee osteoarthritis (OA) from the perspective of individuals living with this condition[J]. Rheumatology (Oxford), 2005, 44(12): 1531−1537. DOI: 10.1093/rheumatology/kei049.
[2] NIEMINEN M T, CASULA V, NEVALAINEN M T, et al. Osteoarthritis year in review 2018: Imaging[J]. Osteoarthritis anc Cartilage, 2019, 27(3): 401-411. DOI: 10.1016/j.joca.2018.12.009.
[3] ECKSTEIN F, WIRTH W, CULVENOR A G. Osteoarthritis year in review 2020: Imaging[J]. Osteoarthritis and Cartilage, 2021, 29(2): 170−179. DOI: 10.1016/j.joca.2020.12.019.
[4] ECKSTEIN F, CICUTTINI F, RAYNAULD J P, et al. Magnetic resonance imaging (MRI) of articular cartilage in knee osteoarthritis (OA): Morphological assessment[J]. Osteoarthritis and Cartilage, 2006, 14(A): 46−75. DOI: 10.1016/j.joca.2006.02.026.
[5] BARR C, BAUER J, MALFAIR, D, et al. MR imaging of the ankle at 3 Tesla and 1.5 Tesla: Protocol optimization and application to cartilage, ligament and tendon pathology in cadaver specimens[J]. European Radiology, 2007, 17: 1518−1528. doi: 10.1007/s00330-006-0446-4
[6] RAYNAULD J P, MARTEL-PELLETIER J, BERTHIAUME M J, et al. Long term evaluation of disease progression through the quantitative magnetic resonance imaging of symptomatic knee osteoarthritis patients: Correlation with clinical symptoms and radiographic changes[J]. Arthritis Research and Therapy, 2006, 8(1): R21.
[7] HUNTER D J, MARCH L, SAMBROOK P N. The association of cartilage volume with knee pain[J]. Osteoarthritis and Cartilage, 2003, 11(10): 725−729. doi: 10.1016/S1063-4584(03)00160-2
[8] ENGLUND M, GUERMAZI A, LOHMANDER L S. The meniscus in knee osteoarthritis[J]. Rheumatic Diseases Clinics of North America, 2009, 35(3): 579−590. doi: 10.1016/j.rdc.2009.08.004
[9] LI R T, LORENZ S, XU Y, et al. Predictors of radiographic knee osteoarthritis after anterior cruciate ligament reconstruction[J]. The American Journal of Sports Medicine, 2011, 39(12): 2595−2603. doi: 10.1177/0363546511424720
[10] SHARMA L, DUNLOP D, CAHUE S, et al. Quadriceps strength and osteoarthritis progression in malaligned and lax knees[J]. Annals of Internal Medicine, 2003, 138(8): 613−619. doi: 10.7326/0003-4819-138-8-200304150-00006
[11] NORMAN B, PEDOIA V, MAJUMDAR S. Use of 2D U-net convolutional neural networks for automated cartilage and meniscus segmentation of knee MR imaging data to determine relaxometry and morphometry[J]. Radiology, 2018, 288(1): 177−185. DOI: 10.1148/radiol.2018172322.
[12] PRASOON A, PETERSEN K, IGEL C, et al. Deep feature learning for knee cartilage segmentation using a triplanar convolutional neural network[C]//Medical Image Computing and Computer-Assisted Intervention: MICCAI International Conference on Medical Image Computing and Computer-Assisted Intervention, 2013, 16: 246-253.
[13] GAJ S, YANG M, NAKAMURA K, et al. Automated cartilage and meniscus segmentation of knee MRI with conditional generative adversarial networks[J]. Magnetic Resonance in Medicine, 2020, 84(1): 437−449. doi: 10.1002/mrm.28111
[14] ZHOU Z, ZHAO G, KIJOWSKI R, et al. Deep convolutional neural network for segmentation of knee joint anatomy[J]. Magnetic Resonance in Medicine, 2018, 80(6): 2759−2770. doi: 10.1002/mrm.27229
[15] MYRONENKO A. 3D MRI brain tumor segmentation using autoencoder regularization[J]. Springer International Publishing, 2019.
[16] WU Y X, HE K M. Group normalization[J]. International Journal of Computer Vision, 2020, 128: 742−755. doi: 10.1007/s11263-019-01198-w
[17] TANG P, ZU C, HONG M, et al. DA-DSUnet: Dual attention-based dense SU-net for automatic head-and-neck tumor segmentation in MRI images[J]. Neurocomputing, 2021, 435: 103−113. doi: 10.1016/j.neucom.2020.12.085
[18] ZHU W, HUANG Y, ZENG, L, et al. AnatomyNet: Deep learning for fast and fully automated whole‐volume segmentation of head and neck anatomy[J]. Medical Physics, 2018, 46(4): 576−589.
[19] SALEH S S, ERDOGMUS D, GHOLIPOUR A. Tversky loss function for image segmentation using 3 D fully convolutional deep networks[J]. Machine Learning in Medical Imaging, 2017: 379−387.
[20] CHEN X. WILLIAMS B M, VALLABHANENI S R, et al. Learning active contour models for medical image segmentation[C]//2019 Conference on Computer Vision and Pattern Recognition (CVPR), 2019: 11624-11632.
[21] LIN T, GOYAL P, GIRSHICK R, et al. Focal loss for dense object detection[C]//2017 IEEE International Conference on Computer Vision (ICCV), 2017: 2999-3007.
[22] ISENSEE F, JAEGER P F, KOHL S A, et al. Automated design of deep learning methods for biomedical image segmentation[J]. arXiv: 1904.08128, 2019.
计量
- 文章访问数: 883
- HTML全文浏览量: 297
- PDF下载量: 159
