M I D A G  at  U N C: Reminiscences and Philosophy

An invited paper written by Kenan Professor Stephen Pizer and the MIDAG project leaders, published in IEEE Transactions on Medical Imaging, January 2003, 22(1): 2-10.



UNC's Medical Image Display & Analysis research group (MIDAG) has been awarded a 5-year grant of approximately $8 million dollars from the National Cancer Institute for years 12-16 of research entitled Medical Image Presentation. The principal investigator of this grant is Kenan Professor Stephen Pizer of the Departments of Computer Science, Radiation Oncology, Radiology, and Biomedical Engineering, and leaders of grant projects and cores are Professor Edward Chaney (Radiation Oncology), Taylor Grandy Professor Guido Gerig (Computer Science & Psychiatry), Associate Professor Keith Muller (Biostatististics), Dr. Graham Gash (Computer Science), and Professor Pizer. The research for the next five years is entitled Structural Image Analysis and Medical Uses and will be carried out mainly in the Departments of Computer Science, Radiation Oncology, Psychiatry, Biostatistics, and Statistics. The objective of the research is to develop methods for the extraction of 3D anatomic objects from CT and MR images for radiation treatment planning and psychiatric diagnosis and for the statistical characterization of object shapes to support research in the development and diagnosis of schizophrenia. The research will further develop methodologies for 3D image based object extraction and shape characterization that are widely applicable and are based on m-reps, the novel means of representing deformable objects invented at UNC by Prof. Pizer and this group.

MIDAG is a multidisciplinary UNC research group of around 100 faculty, graduate students, and staff from 11 UNC departments.

Begun in 1974, MIDAG focuses on many problems of extracting and displaying information from CT, MR, ultrasound, X-ray, and nuclear medicine images to help physicians plan and deliver therapy and diagnose disease. Most of the recent work of MIDAG has been done on 3D medical images, and much of it involves describing the shape of anatomic objects of individuals and populations of individuals. The following describes many of the recent research projects of MIDAG.


Model-based Object Extraction for Radiation Treatment Planning
Participating Depts: Computer Science, Radiation Oncology, Psychiatry
Leaders: SM Pizer, EL Chaney

In radiation oncology cancerous tumor tissue is to be destroyed by radiation while sparing tissue of nearby normal organs. Planning how to form and aim the radiation beams therefore requires knowing where the tumor and the normal organs in the patient to be treated are. This information as to where the tumor and normal organs are is available in 3D CT or MRI images of the patient but must be extracted from the images to be usable for planning the beams.

M-reps are a form of geometric model that we have invented to represent deformable objects by their interior and to study populations of objects probabilistically. They lead to efficient and effective methods to deform atlases into individual patients measured by their 3D images and thus to extract the organs needed in treatment planning from the patient image. M-reps also allow following individuals as they change shape or measuring the shape differences between well and ill patient classes (see section below)

PlanUNC, built by Radiation Oncology based on MIDAG research, is the radiation treatment planning tool in clinical use there

As illustrated in the PlanUNC panel, radiation treatment planning requires the extraction of anatomic objects, which can be modeled by m-reps

Stages of extraction of kidney from CT in planning treatment of abdominal cancer.
Kidney model thereby deforms into the CT image data

Planning radiation treatment of lung cancer, recognizing breathing motion


Morphology of Brain Structures in Psychiatric Illness
Participating Depts: Computer Science, Psychiatry, Radiology
Leaders: G Gerig, JA Lieberman, S Joshi , SM Pizer

Improved understanding of disease mechanisms in psychiatric illness is a key factor for early diagnosis, better treatment, and development of new drug therapies. Measurements of morphologic change of brain structures imaged in 3D by MRI illuminate the nature of neurodegenerative diseases and/or disorders of abnormal neurodevelopment. We study the whole age-range to provide a broad picture of neurodevelopmental and neurodegenerative aspects of illness, especially schizophrenia.

We compare the size and shape of brain structures between healthy subjects and patients with or at risk for schizophrenia. We also measure disease progress by observing these changes over time. Our 3D model deformations efficiently extract anatomical structures, reducing time from several hours to a few minutes. New measures of shape statistics using m-reps provide insight into the natural variability of brain structures and thus help to describe difference from normal in clinical terms such as local growth and thinning, widening, or bending

Creation of a statistical model template of the hippocampus from a shape population

Overlay of shapes of the healthy hippocampus (curved) and its alteration (flat) in schizophrenia

Overlay of shapes of the healthy hippocampus (curved) and its alteration (flat) in schizophrenia

Shape variability of brain ventricles of identical twin pairs (top row) and nonidentical twin pairs (bottom row).


Patient-Specific Vascular Models for Surgical Planning and Guidance
Participating Depts: Surgery, Radiology, Computer Science, Radiation Oncology
Leaders: E Bullitt, S Aylward

Many surgical and all endovascular procedures require understanding the locations and connectivity of a patient's blood vessels. Our projects extract patient-specific vascular networks from 3D image data, to aid vascular procedures.

Projects include 1) interactive 3D visualizations for neurosurgical planning, 2) registration of high-resolution 3D pre-operative images with 2D intra-operative images for guidance of intra-vascular procedures, 3) definition of cutting planes for liver donors, 4) overlaying pre-operative findings and surgical plans onto intra-operative 3D ultrasound to guide biopsies, 5) delineation and hybrid surface/volume visualization of tumors and their surrounding/feeding vasculature, and 6) judging malignancy and extent of tumors via the pattern of blood vessels within and surrounding the tumor.

Tumor (white) segmented from MRA and shown in relationship to color-coded vessel trees. Some vessels pass through the tumor to supply normal brain, which is important to know for surgical planning.

Pre-operatively defined portal vessels (blue) are registered with a 2D image obtained during an endovascular procedure.

Lesion (orange), defined from a pre-operative CT, is superimposed upon an intra-operative 3D ultrasound image to enable fast and accurate ultrasound guided biopsy.

Blood vessls in relation to a tumor. Their structure can help extract which tissues is tomor and diagnose malignany.

Medical Augmented Reality
Participating Depts: Computer Science, Radiology, Surgery
Leaders: H Fuchs, ED Pisano
We aim to enable physicians to achieve more accurate miminally invasive access to internal patient structures by viewing real-time medical data registered with the patient. These visualization methods use images such as ultrasound echograms or laparascopic video and augmented reality technology such as see-through head-mounted displays (HMDs) and motion trackers. The physician sees a dynamic, stereoscopic image of the patient, enhanced with registered live intra-operative imagery. Our work reaches from the design and construction of video see-through HMDs and prototype 3D laparoscopes to human subject studies comparing conventional and AR methods for ultrasound-guided breast biopsies. More information can be found at

AR guidance system in use on a breast biopsy model (above) and on a human subject (below).

Parametric design tool for video see-through HMDs (left), resulting design (center) and finished device (right): Camera and eye positions are matched geometrically via mirrors.