Title
题目
Liver lesion changes analysis in longitudinal CECT scans by simultaneous deep learning voxel classification with SimU-Net
通过SimU-Net进行同时深度学习体素分类的纵向CECT扫描肝病灶变化分析
01
文献速递介绍
影像学随访是对影像学研究的解读，以评估疾病状态和确定治疗效果，从而支持临床决策。随着癌症发病率的增加和患者寿命的延长，影像学随访在肿瘤学中的重要性日益增加（《全球医学影像市场报告》，2019；Signify Research，2018）。各种起源的癌症，如肝癌、肺癌、乳腺癌、前列腺癌、结直肠癌，虽然起源于不同的器官，但往往会转移到相同的二级器官，即肝脏、肺和大脑，其转移率分别为58%、62%和15%。医学肿瘤治疗包括化疗、生物药物和免疫疗法的组合。
Abstract
摘要
在纵向增强对比CT（CECT）扫描中识别和量化肝病灶的变化是评估疾病状态和确定治疗效果以支持临床决策所必需的。本文描述了一个全自动的端到端流程，用于分析肿瘤患者的连续（之前和当前）腹部CECT扫描中的肝病灶变化。三个主要创新点是：（1）SimU-Net，一个同时多通道3D R2U-Net模型，训练在每个患者的一对注册扫描上，基于病灶和健康组织外观差异识别肝病灶及其变化；（2）基于模型的二部图病灶匹配方法，用于在病灶级别分析病灶变化；（3）基于SimU-Net的方法进行患者一个或多个连续扫描的纵向分析，处理主要肝脏变形并结合先前分析的病灶分割。为了验证我们的方法，在一个独特的数据集上进行了五项实验研究，该数据集包含71名转移性疾病患者的218次临床腹部CECT扫描中735对共3491个肝病灶，这些病灶由一名专家放射科医师手动描绘。使用SimU-Net模型的流程，在385对扫描上训练和验证，在249对扫描上测试，产生的病灶检测召回率为0.86±0.14，精度为0.74±0.23，病灶分割Dice系数为0.82±0.14（病灶大于5毫米）。相比单独分析每个扫描的参考3D R2-UNet模型，其精度提升约50%，且在相同的训练和测试数据集上召回率和Dice系数相似。病灶匹配的精度为0.86±0.18，召回率为0.90±0.15。病灶分类的特异性为0.97±0.07，精度为0.85±0.31，召回率为0.86±0.23。我们的新方法提供了准确且全面的结果，有助于减少放射科医生的时间和精力，并改善放射肿瘤学评估。
Method
方法
The key novelty of our method is the simultaneous detection and segmentation of lesions and lesion changes in consecutive pairs of scans with SimU-Net, a simultaneous multi-channel 3D RU-Net voxel classification model. The simultaneous analysis performed by SimU-Net leverages on additional co-location information for the lesion voxels' classification: gray values of two patches per voxel instead of one, and on the relative lesion and healthy tissue appearance differences, instead of only the individual tissue appearance. This mimics the way radiologists currently detect and evaluate lesion changes. Fig. 2 shows a novel, fully automatic end-to-end pipeline for liver lesion changes analysis at the voxel and lesion levels in pairs of abdominal CECT scans of a patient that is based on SimU-Net. The input is a pair (prior and current) of CECT abdominal scans. The output is the lesion changes' analysis consisting of the lesions segmentation in the prior and current scans and the lesion and lesion changes report (Fig. 1d). Optionally, when the liver lesions segmentation in the prior scan are available, it is also input to the simultaneous lesion detection and segmentation step.
我们方法的关键创新在于使用SimU-Net，一种同时多通道3D RU-Net体素分类模型，对连续扫描对中的病灶及其变化进行同时检测和分割。SimU-Net进行的同时分析利用了病灶体素分类的额外共定位信息：每个体素的两个补丁的灰度值，而不是一个补丁，并且利用相对病灶和健康组织外观差异，而不仅仅是单个组织外观。这模仿了放射科医生目前检测和评估病变变化的方式。
图2展示了一种新颖的、完全自动的端到端流程，用于基于SimU-Net在患者腹部CECT扫描对中的肝病灶变化分析，包括体素和病灶级别的分析。输入是一对（前期和当前）CECT腹部扫描。输出是病灶变化的分析结果，包括前期和当前扫描中的病灶分割，以及病灶和病灶变化报告（图1d）。可选地，如果前期扫描中的肝病灶分割已经可用，则也作为同时病灶检测和分割步骤的输入。
Conclusion
结论
We have presented a new fully automatic method for longitudinal liver lesions follow-up in pairs of prior and current liver CECT scans. Our method is unique in providing radiologists both a classification of lesion changes at the lesion level (new, disappeared, existing) and a volumetric measurement of changes. We show that these measurements can be significantly improved by using the prior scan and, when available, its lesions segmentation, in the lesions detection and segmentation tasks. Our proposed pipeline can be used in radiological lesions follow-up to quantitatively evaluate disease progression, assess treatment efficiency, and analyze response to therapy. It may help radiologists to cope with the increase in the number of radiological follow-up radiological examinations for oncology patients and with the volumetric evaluation of scans obtained after treatment, which account for ~40% of all acquired scans worldwide.
我们提出了一种全新的全自动方法，用于在先前和当前的肝脏CT增强扫描（CECT）配对图像中进行纵向肝脏病灶的随访。我们的方法独特之处在于，它可以为放射科医生提供病灶变化的分类（新出现、消失、存在）以及体积变化的测量。我们展示了通过使用先前扫描及其病灶分割结果，可以显著改善病灶检测和分割任务中的测量结果。
我们提出的流程可以用于影像学病灶随访中，定量评估疾病进展、评估治疗效果和分析治疗反应。它可以帮助放射科医生应对肿瘤患者放射学随访检查数量的增加以及治疗后扫描体积评估的需求，而这些评估占全球所有获取扫描的约40%。
Figure
图
Fig 1. Illustration of radiological follow-up tasks: liver lesions detection, segmentation, matching and classification: (a) representative registered slices of prior (top) and current (bottom) liver contrast-enhanced CT scans of a patient acquired six months apart; (b) detection and segmentation of liver lesions (orange) in the prior (top) and current (bottom) scans; (c) identification and matching of the prior (top, numbers) and current (bottom, letters) liver lesions; colors indicate the lesion matching; (d) lesions matching graph (top): nodes correspond to lesions (left prior, right current); edges correspond to lesions parings; lesions changes summary (bottom): number of lesions in each scan, lesions changes type: existing, existing split, disappeared lesions (numbers and letters indicate pairings); volumetric measures omitted.
图1. 影像学随访任务的示意图：肝病灶检测、分割、匹配和分类：（a）患者在六个月间隔内获取的前（上）和当前（下）肝脏增强CT扫描的代表性注册切片；（b）前（上）和当前（下）扫描中肝病灶（橙色）的检测和分割；（c）前（上，数字）和当前（下，字母）肝病灶的识别和匹配；颜色表示病灶匹配情况；（d）病灶匹配图（上）：节点对应病灶（左侧为前，右侧为当前）；边对应病灶配对；病灶变化摘要（下）：每次扫描中的病灶数量，病灶变化类型：现存、现存分裂、消失病灶（数字和字母表示配对）；体积测量省略。
Fig. 2. Flow diagram of the liver lesion changes analysis pipeline in pairs of abdominal CECT scans. The inputs are the prior (blue) and current (green) scans of a patient. The outputs are the lesion segmentations in each scan and the lesion and lesion changes report. The steps are: (1) liver segmentation in the prior and current scans; (2) registration of the prior liver segmentation to the current liver segmentation; (3) simultaneous liver lesion detection and segmentation in the prior and current liver segmentations with SimU-Net; (4) lesion changes detection and classification at the lesion level; (5) lesions and lesion changes quantification.
图2. 腹部CECT扫描对中肝病灶变化分析流程图。输入为患者的前期（蓝色）和当前（绿色）扫描。输出为每个扫描中的病灶分割以及病灶和病灶变化报告。步骤包括：（1）前期和当前扫描中的肝脏分割；（2）将前期肝脏分割注册到当前肝脏分割；（3）使用SimU-Net对前期和当前肝脏分割进行同时肝病灶检测和分割；（4）在病灶级别检测和分类病灶变化；（5）病灶及病灶变化的量化。
Fig. 3. Illustration of the advantage of the simultaneous SimU-Net model over the standalone 3D R2U-Net model: (a) representative matching slices of a prior (top) and current (bottom) liver CECT scans of a patient acquired four months apart; (b) ground truth segmentation of a lesion (orange) whose volume has increased by 1.64cc; (c) standalone model result: the lesion was missed in the prior scan (top, false negative); it was detected and partially segmented in the current scan (bottom, Dice score of 0.40) and classified as new (red), together with two new false positive lesions (red); (d) SimU-Net model results: the lesion was correctly detected and segmented (orange) in both the prior (top, Dice score of 0.88) and current (bottom, Dice score of 0.90) scans; the computed volume increase is 1.57cc (− 4% of the ground truth value). Both models were trained with the same training set.
图3. 示范SimU-Net模型相对独立3D R2U-Net模型的优势：(a) 患者前后相隔四个月获取的前期（上）和当前（下）肝脏CECT扫描的代表性匹配切片；(b) 病灶（橙色）的地面真实分割，其体积增加了1.64cc；(c) 独立模型结果：病灶在前期扫描中未被检测到（上，假阴性）；在当前扫描中被检测到并部分分割（下，Dice系数为0.40），并被分类为新的病灶（红色），同时还有两个新的假阳性病灶（红色）；(d) SimU-Net模型结果：病灶在前期（上，Dice系数为0.88）和当前（下，Dice系数为0.90）扫描中被正确检测和分割（橙色），计算出的体积增加为1.57cc（地面真实值的-4%）。两个模型均使用相同的训练集。
Fig. 4. Configurations of SimU-Net for the four scenarios for lesion and lesion changes analysis: (a) Simultaneous single for individual scans; (b) Simultaneous for pairs of scans; (c) Simultaneous with prior for pairs of scans with prior lesion segmentations; (d) Longitudinal for studies with mutiple consecutive scans. M1 is a simultaneous two-channel SimU-Net trained on registered pairs of prior and current scans. M2 is a simultaneous three-channel SimU-Net model trained on triplets of registered current and prior scans and prior lesion segmentations.
图4. SimU-Net在四种病灶及病灶变化分析场景下的配置：(a) Simultaneous single，用于单个扫描的同时分析；(b) Simultaneous，用于扫描对的同时分析；(c) Simultaneous with prior，用于具有前期病灶分割的扫描对的同时分析；(d) Longitudinal，用于具有多个连续扫描的研究。M1是一个同时两通道的SimU-Net模型，训练在配准的前期和当前扫描对上。M2是一个同时三通道的SimU-Net模型，训练在配准的当前和前期扫描三元组以及前期病灶分割上。
Fig. 5. Partition of the liver lesion dataset D2. Nodes show the number of pairs, scans and patients of each set; the root shows the total. Dataset D2 is split into twodatasets: D2 reg consists of registered pairs with minor liver changes, and D2 noreg consists of unregistered pairs with major liver changes. Dataset D2 reg is further split into pairs for training (left) and testing (center). Three models were generated with D2 reg: M0 Standalone single, M1 simultaneous, and M2 Simultaneous with prior. Four test setups were established with D2 reg (center): two for individual scan analysis with M0 and M1 and two for simultaneous pairwise scan analysis with M1* and M2 without and with prior lesions segmentation. Two test setups were established with D2 noreg (left) for individual scan analysis with M0 and M1.
图5. 肝病灶数据集D2的分区。节点显示每个集合中的配对数、扫描数和患者数；根节点显示总数。数据集D2分为两个子数据集：D2 reg包含具有较小肝脏变化的配准配对，D2 noreg包含具有较大肝脏变化的未配准配对。数据集D2 reg进一步分为用于训练（左侧）和测试（中心）的配对。使用D2 reg生成了三个模型：M0独立单个、M1同时和M2同时带有前期。使用D2 reg建立了四个测试设置（中心）：两个用于单个扫描分析的设置，分别使用M0和M1，两个用于同时配对扫描分析的设置，分别使用M1和M2，其中包括前期病灶分割和不包括前期病灶分割。使用D2 noreg建立了两个测试设置（左侧），用于单个扫描分析，分别使用M0和M1。
Fig. 6. No registration: illustration of the advantage of the simultaneous single model over the standalone model: (a) representative matching slices of a prior (top) and current (bottom) liver CECT scans of a patient acquired three months apart; (b) ground truth segmentation of a lesion (orange) whose volume has increased by 1.26cc; (c) standalone model results: a false lesion was detected in the prior scan (top, false positive) and classified as disappeared (green), the true positive lesion was correctly detected and segmented correctly in the prior scan (top, Dice score of 0.92), but was incorrectly segmented in the current scan (bottom, Dice score of 0.68); (d) simultaneous single model results: the lesion was correctly detected and segmented (orange) in both the prior (top, Dice score of 0.92) and the current (bottom, Dice score 0.88) scans; the volume increase is +1.33cc (within 5% of the ground truth). Both models were trained with the same training set (Section 4.2).
图6. 无配准：示范同时单个模型相对独立模型的优势：(a) 患者前后相隔三个月获取的前期（上）和当前（下）肝脏CECT扫描的代表性匹配切片；(b) 病灶（橙色）的地面真实分割，其体积增加了1.26cc；(c) 独立模型结果：在前期扫描中检测到一个假阳性病灶（上，假阳性），并将其分类为消失（绿色），真阳性病灶在前期扫描中被正确检测并正确分割（上，Dice系数为0.92），但在当前扫描中被错误分割（下，Dice系数为0.68）；(d) 同时单个模型结果：病灶在前期（上，Dice系数为0.92）和当前（下，Dice系数为0.88）扫描中被正确检测和分割（橙色），体积增加为+1.33cc（与地面真实值相差不到5%）。两个模型均使用相同的训练集（第4.2节）。
Table
表
Table 1 Results of the lesion detection and segmentation in the current and prior scans for the simultaneous model and for the standalone model for three categories of liver lesions: lesions with diameter > 10 mm, lesions with diameter > 5 mm, and all lesions. Columns 3--4 list the mean (std) lesions detection precision and recall. Columns 5--8 list the mean (std) lesion segmentation Dice, ASSD, Mean SHD, and Max SHD. Numbers in boldface indicate the statistically significant best per-class results (* indicates no difference).
表1对于三类肝脏病灶（直径 > 10 mm、直径 > 5 mm 和所有病灶），同时模型和独立模型在当前和前期扫描中进行病灶检测和分割的结果。第3--4列列出平均（标准差）病灶检测的精度和召回率。第5--8列列出平均（标准差）病灶分割的Dice系数、平均表面距离（ASSD）、平均哈斯多夫距离（Mean SHD）和最大哈斯多夫距离（Max SHD）。粗体数字表示在每类别中具有统计显著的最佳结果（*表示无差异）。
Table 2 Results of the lesion detection and segmentation in the current scan for the simultaneous with prior model and for the simultaneous single model for three categories of liver lesions. The columns are as in Table 1.
表2 同时带有前期模型和同时单个模型在当前扫描中进行病灶检测和分割的结果，针对三类肝脏病灶。列的设置与表1相同。
Table 3 No registration: results for lesion detection and segmentation in the current scan for the simultaneous single model and for the standalone model M0 for three categories of liver lesions. The columns are as in Table 1. The test set consists of 34 individual scans of 23 different patients.
表3无配准：针对三类肝脏病灶，在当前扫描中进行病灶检测和分割的结果，使用同时单个模型和独立模型M0。列的设置与表1相同。测试集包括23位不同患者的34次单个扫描。