[]We use the Cornerstone JavaScript library to build the labeling UI that displays a DICOM image in modern web browsers that support the HTML5 canvas element, and use Cornerstone tools to support interactions and enable region of interest (ROI) annotations in different shapes. Additionally, we use the Cornerstone WADO Image Loader to retrieve Web Access to DICOM Objects (WADO) URI through the DICOMwebTM plugin in a remote Orthanc server.

[]The following two functions are important for retrieving and viewing the DICOM image:

• DownloadAndView() – Adds wadouri: to the beginning of the URL so that the cornerstone.js file can find the image loader
• loadAndViewImage() – Uses the Cornerstone library to display the image and allows additional functionalities such as zoom, pan, and ROI annotation

[]You should be able to annotate any DICOM image using the following custom HTML template, which lists the possible labels as multiple option selections for the image classification task.

Build an ML model using the MONAI framework

[]Now that we have a labeled dataset in the form of the output-manifest.json file, we can start the ML model building process. We created a SageMaker notebook instance earlier through our stack deployment. Alternatively, you can create a new notebook instance. The MONAI library and other additional packages are managed in requirements.txt, which is in the same folder of the training script.

Data loading and preprocessing

[]After you open the Jupyter notebook, load the output-manifest.json file that has the DICOM URLs and labels.

[]The following example from dicom_training.ipynb parses the labeled dataset from output-manifest.json:

# Get SageMaker Ground Truth labeling data from annotation file datadir = ‘data’ metadata = ‘{}/output-manifest.json’.format(datadir) s3 = boto3.client(‘s3’) s3.download_file(bucket, bucket_path, metadata) # Load Labels with open(metadata) as f: manifest = json.load(f) class_names = list(json.loads(manifest[0][‘annotations’][0][‘annotationData’][‘content’])[‘disease’].keys()) for i, j in enumerate(manifest): label_dict = json.loads(json.loads(manifest[i][‘annotations’][0][‘annotationData’][‘content’])[‘labels’]) image_url_list.extend([label_dict[‘imageurl’]]) image_label_list.extend([class_names.index(label_dict[‘label’][0])]) []Now that we have the DICOM URLs in image_url_list and corresponding labels in image_label_list, we can download the DICOM files from Orthanc DICOM server directly into Amazon S3 for SageMaker training:

import urllib3 import requests import os from io import BytesIO import contextlib; image_file_list = [] #Load DICOM images to S3 for i, j in enumerate(image_url_list): file_name = image_url_list[i].split(“/file”)[0].split(“instances/”)[1] + ‘.dcm’ response = requests.get(image_url_list[i], auth=(user, password), stream=True, verify=False) fp = BytesIO(response.content) s3.upload_fileobj(fp, bucket, file_name) image_file_list.append(file_name) []After you load the images to your S3 bucket, you can display a sample of the images with labels from the parsed JSON file using the MONAI framework:

import monai from monai.transforms import Compose, LoadImage, Resize, SqueezeDim import matplotlib.pyplot as plt #Display sample of DICOM Images trans = Compose([LoadImage(image_only=True), Resize(spatial_size=(108,96)),SqueezeDim()]) plt.subplots(1, 3, figsize=(8, 8)) for i in range(0,3): s3.download_file(bucket, image_file_list[i], datadir+’/’+image_file_list[i]) img = trans(datadir+’/’+image_file_list[i]) plt.subplot(1, 3, i + 1) plt.xlabel(class_names[image_label_list[i]]) plt.imshow(img, cmap=’gray’) plt.tight_layout() plt.show() []The following images show a sample output of DICOM images with labels.

[]

[]We use the MONAI transforms during model training to load the DICOM files directly and preprocess them. MONAI has transforms that support both dictionary and array formats, and is specialized for the high-dimensionality of DICOM medical images. The transforms include several categories such as crop and pad, intensity, I/O, postprocessing, spatial, and utilities. In the following excerpt, the Compose class chains a series of image transforms together and returns a single tensor of the image:

#Load and transform DICOM images class DICOMDataset(Dataset): def __init__(self, image_files, labels, transforms): self.image_files = image_files self.labels = labels self.transforms = transforms def __len__(self): return len(self.image_files) def __getitem__(self, index): return self.transforms(self.image_files[index]), self.labels[index] train_transforms = Compose([ LoadImage(image_only=True), ScaleIntensity(), RandRotate(range_x=15, prob=0.5, keep_size=True), RandFlip(spatial_axis=0, prob=0.5), RandZoom(min_zoom=0.9, max_zoom=1.1, prob=0.5, keep_size=True), Resize(spatial_size=(108,96)), ToTensor() ]) dataset = DICOMDataset(trainX, trainY, train_transforms) train_sampler = torch.utils.data.distributed.DistributedSampler(dataset) if is_distributed else None train_loader = torch.utils.data.DataLoader(dataset, batch_size=batch_size, shuffle=train_sampler is None, sampler=train_sampler, **kwargs) …

Model training

[]MONAI includes deep neural networks such as UNet, DenseNet, GAN, and others, and provides sliding window inferences for large medical image volumes. In the DICOM image classification model, we train the MONAI DenseNet model on the DICOM images loaded and transformed using the DataLoader class in the previous steps for 10 epochs while measuring loss. See the following code:

#Instantiate model device = torch.device(“cuda”) model = densenet121( spatial_dims=spatial_dims_num, in_channels=in_channels_num, out_channels=out_channels_num ).to(device) loss_function = torch.nn.CrossEntropyLoss() optimizer = torch.optim.Adam(model.parameters(), 1e-5) #Train DICOM model for epoch in range(epoch_num): model.train() epoch_loss = 0 step = 0 for batch_data in train_loader: step += 1 inputs, labels = batch_data[0].to(device), batch_data[1].to(device) optimizer.zero_grad() outputs = model(inputs) loss = loss_function(outputs, labels) loss.backward() optimizer.step() epoch_loss += loss.item() epoch_len = len(train_loader.dataset) // train_loader.batch_size epoch_loss /= step epoch_loss_values.append(epoch_loss) []The following example shows how to use the SageMaker Python SDK in a Jupyter notebook to instantiate a training job and run the DICOM model training script:

inputs = sagemaker_session.upload_data(path=datadir, bucket=bucket) from sagemaker.pytorch import PyTorch estimator = PyTorch(entry_point=’monai_dicom.py’, source_dir=’source’, role=role, framework_version=’1.5.0′, py_version=’py3′, instance_count=1, instance_type=’ml.m5.xlarge’, hyperparameters={ ‘backend’: ‘gloo’, ‘epochs’: 10 }) estimator.fit({‘train’: inputs}) []When you call the fit method in this example, it launches the training instance with a compute size based on the instance_type parameter, instantiates the built-in PyTorch deep learning container, installs the MONAI dependent packages found in requirements.txt, and runs the training script referenced by the entry_point=’monai_dicom.py’ parameter.

[]For more details on how to use the MONAI framework within SageMaker to build and deploy your medical image models, see Build a medical image analysis pipeline on Amazon SageMaker using the MONAI framework. To learn about other custom data labeling workflows, see Build a custom data labeling workflow with Amazon SageMaker Ground Truth.

Conclusion

[]This post showed how to build a DICOM custom labeling workflow with Ground Truth. Additionally, we showed you how to take the output manifest file generated from your labeling workflow and use it as a labeled dataset to build a DenseNet image classification model using the MONAI framework.

[]If you have any comments or questions about this post, please use the comments section. To find the latest developments to this solution, check out the GitHub repo.

About the Authors

[]Nihir Chadderwala is an AI/ML Solutions Architect on the Global Healthcare and Life Sciences team. His background is building big data and AI-powered solutions to customer problems in a variety of domains such as software, media, automotive, and healthcare. In his spare time, he enjoys playing tennis, and watching and reading about Cosmos.

[] 

[]Gang Fu is a Senior Healthcare Solutions Architect at AWS. He holds a PhD in Pharmaceutical Science from the University of Mississippi and has over 10 years of technology and biomedical research experience. He is passionate about technology and the impact it can make on healthcare.

[] 

[] 

[]Bryan Marsh is a Senior Solutions Architect in the Academic Medical Center team at Amazon Web Services. He has expertise in enterprise architecture with a focus in the healthcare domain. He is passionate about using technology to improve the healthcare experience and patient outcomes.

Source