{ "cells": [ { "cell_type": "markdown", "id": "c51dcfb5", "metadata": { "collapsed": false, "jupyter": { "outputs_hidden": false } }, "source": [ "# Working with MultiLabelSegmentation\n", "\n", "This notebook shows how to create, transfer, and modify multi-label segmentations between Python and a running MITK Workbench.\n", "\n", "You will learn how to:\n", "- Build a `MultiLabelSegmentation` from scratch (groups, labels, pixel data)\n", "- Upload it via `node.set_data()` and download it back via `node.get_data()`\n", "- Inspect and edit label metadata (name, color, visibility)\n", "- Dilate a label region using a SimpleITK filter (optional)\n", "- Remove a label including its pixels via `remove_label()`\n", "- Validate consistency and push the modified segmentation back\n", "\n", "**Prerequisites:** A running MITK Workbench instance with the REST API enabled." ] }, { "cell_type": "markdown", "id": "5c128f5a", "metadata": { "collapsed": false, "jupyter": { "outputs_hidden": false } }, "source": [ "## 1. Connect to a running Workbench" ] }, { "cell_type": "code", "execution_count": 1, "id": "86f03fb7", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Connected to: MITK Workbench REST API (MITK 2025.12.99-23b4e629) at http://localhost:8080\n" ] } ], "source": [ "import numpy as np\n", "import mitk_workbench_remote as mw\n", "from mitk_workbench_remote import Label, Image, MultiLabelSegmentation, LABEL_DTYPE\n", "\n", "instances = mw.discover(timeout=2) #longer timeouts to make the example more robust for latencies.\n", "if not instances:\n", " raise SystemExit(\"No running MITK Workbench found. Start one and try again.\")\n", "\n", "wb = instances[0]\n", "print(f\"Connected to: {wb.info.name} (MITK {wb.info.mitk_version}) at {wb.url}\")" ] }, { "cell_type": "markdown", "id": "d0c19a2f", "metadata": { "collapsed": false, "jupyter": { "outputs_hidden": false } }, "source": [ "## 2. Build a synthetic segmentation\n", "\n", "`MultiLabelSegmentation` organises labels into named groups. Label values are globally unique across all groups; value 0 is reserved as the unlabeled background.\n", "\n", "Here we create two groups:\n", "- **Anatomy** — two anatomical structures (Liver, Spleen)\n", "- **Findings** — one pathological structure (Tumor)\n", "\n", "Each group's pixel data is a separate 3-D integer array stored as an `Image`. Voxels that belong to a label carry its integer value; unlabeled voxels are 0." ] }, { "cell_type": "code", "execution_count": 2, "id": "11fbc009", "metadata": { "collapsed": false, "jupyter": { "outputs_hidden": false } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", " Liver value=1\n", " Spleen value=2\n", " Tumor value=3\n" ] } ], "source": [ "SHAPE = (80, 80, 80) # voxels\n", "SPACING = (2.0, 2.0, 2.0) # mm per voxel\n", "\n", "ref_img = Image(np.zeros(SHAPE, dtype=LABEL_DTYPE), spacing=SPACING)\n", "seg = MultiLabelSegmentation.create(shape=SHAPE, spacing=SPACING)\n", "\n", "# --- Group 0: Anatomy ---\n", "g_anatomy = seg.add_group(\"Anatomy\")\n", "liver = seg.add_label(Label(None, \"Liver\", color=(0.82, 0.40, 0.12)), group=g_anatomy)\n", "spleen = seg.add_label(Label(None, \"Spleen\", color=(0.30, 0.60, 0.80)), group=g_anatomy)\n", "\n", "# --- Group 1: Findings ---\n", "g_findings = seg.add_group(\"Findings\")\n", "tumor = seg.add_label(Label(None, \"Tumor\", color=(1.0, 0.1, 0.1), locked=True), group=g_findings)\n", "\n", "print(seg)\n", "print(f\" Liver value={liver.value}\")\n", "print(f\" Spleen value={spleen.value}\")\n", "print(f\" Tumor value={tumor.value}\")" ] }, { "cell_type": "code", "execution_count": 3, "id": "sxj909t59tg", "metadata": {}, "outputs": [ { "data": { "text/plain": "", "text/html": "
ValueNameColorVisibleLocked
Anatomy
1LiverTrueFalse
2SpleenTrueFalse
Findings
3TumorTrueTrue
" }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "seg" ] }, { "cell_type": "code", "execution_count": 4, "id": "354626fe", "metadata": { "collapsed": false, "jupyter": { "outputs_hidden": false } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Consistency check: OK\n", "Voxel counts — Liver: 23795, Spleen: 4169, Tumor: 925\n" ] } ], "source": [ "def draw_sphere(arr: np.ndarray, center: tuple, radius: float, value: int) -> None:\n", " \"\"\"Paint a filled sphere into arr at center with the given integer label value.\"\"\"\n", " cx, cy, cz = center\n", " x, y, z = np.ogrid[:arr.shape[0], :arr.shape[1], :arr.shape[2]]\n", " arr[(x - cx)**2 + (y - cy)**2 + (z - cz)**2 <= radius**2] = value\n", "\n", "# --- Paint ref_img ---\n", "draw_sphere(ref_img.array, center=(40, 38, 40), radius=40, value=100)\n", "\n", "# --- Paint Anatomy group ---\n", "arr_anatomy = np.zeros(SHAPE, dtype=LABEL_DTYPE)\n", "draw_sphere(arr_anatomy, center=(40, 38, 40), radius=18, value=liver.value)\n", "draw_sphere(arr_anatomy, center=(58, 50, 35), radius=10, value=spleen.value)\n", "seg.set_group_image(g_anatomy, arr_anatomy)\n", "\n", "# --- Paint Findings group ---\n", "arr_findings = np.zeros(SHAPE, dtype=LABEL_DTYPE)\n", "draw_sphere(arr_findings, center=(36, 44, 42), radius=6, value=tumor.value)\n", "seg.set_group_image(g_findings, arr_findings)\n", "\n", "issues = seg.validate()\n", "print(\"Consistency check:\", issues if issues else \"OK\")\n", "\n", "liver_px = int((arr_anatomy == liver.value).sum())\n", "spleen_px = int((arr_anatomy == spleen.value).sum())\n", "tumor_px = int((arr_findings == tumor.value).sum())\n", "print(f\"Voxel counts — Liver: {liver_px}, Spleen: {spleen_px}, Tumor: {tumor_px}\")" ] }, { "cell_type": "markdown", "id": "5504e575", "metadata": { "collapsed": false, "jupyter": { "outputs_hidden": false } }, "source": [ "## 3. Upload to the Workbench\n", "\n", "MITK's Segmentation view expects a segmentation to be a **child node** of its reference image in the DataStorage tree. We therefore upload in two steps:\n", "\n", "1. Create a node for the reference image and upload it.\n", "2. Create the segmentation node **as a child** of the image node by passing `parent=img_node`.\n", "\n", "`set_data()` serialises each object to NRRD and transfers it. The transfer mode — direct HTTP body or local file reference for large data on localhost — is negotiated automatically.\n", "\n", "After uploading, `wb.reinit([img_node])` fits the render windows to the image geometry." ] }, { "cell_type": "code", "execution_count": 5, "id": "15fa230d", "metadata": { "collapsed": false, "jupyter": { "outputs_hidden": false } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Uploaded 'Demo CT' (uid=node_1, type=Image)\n", "Uploaded 'Demo Segmentation' (uid=node_2, type=MultiLabelSegmentation)\n", "Children of img_node: ['Demo Segmentation']\n" ] } ], "source": [ "# Upload the reference image\n", "img_node = wb.storage.create(\"Demo CT\")\n", "img_node.set_data(ref_img)\n", "\n", "# Upload the segmentation as a child of the image node.\n", "# MITK uses the parent-child relationship to link a segmentation to its reference image,\n", "# which enables the Segmentation view to display both together correctly.\n", "seg_node = wb.storage.create(\"Demo Segmentation\", parent=img_node)\n", "seg_node.set_data(seg)\n", "\n", "wb.reinit()\n", "\n", "print(f\"Uploaded '{img_node.name}' (uid={img_node.uid}, type={img_node.data_type})\")\n", "print(f\"Uploaded '{seg_node.name}' (uid={seg_node.uid}, type={seg_node.data_type})\")\n", "print(f\"Children of img_node: {[n._name for n in img_node.children]}\")" ] }, { "cell_type": "code", "execution_count": 6, "id": "1jjkcema5m2", "metadata": {}, "outputs": [ { "data": { "text/plain": "", "text/html": "
uidnode_2
nameDemo Segmentation
data_typeMultiLabelSegmentation
path/Demo CT/Demo Segmentation
parent_uidnode_1
" }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "seg_node" ] }, { "cell_type": "markdown", "id": "me2sougx9b", "metadata": {}, "source": [ "## 4. Download and inspect the segmentation\n", "\n", "`get_data()` returns a `MultiLabelSegmentation` with complete pixel data and label metadata as stored in the Workbench. The rich HTML representation below lists every group and label with its value, color swatch, visibility, and lock state — a quick sanity check that the roundtrip preserved all metadata correctly." ] }, { "cell_type": "code", "execution_count": 7, "id": "265048e9", "metadata": { "collapsed": false, "jupyter": { "outputs_hidden": false } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "\n", "Group 0: 'Anatomy'\n", " value= 1 name='Liver' color=(0.8199999928474426, 0.4000000059604645, 0.11999999731779099) locked=False voxels=23795\n", " value= 2 name='Spleen' color=(0.30000001192092896, 0.6000000238418579, 0.800000011920929) locked=False voxels=4169\n", "\n", "Group 1: 'Findings'\n", " value= 3 name='Tumor' color=(1.0, 0.10000000149011612, 0.10000000149011612) locked=True voxels=925\n" ] } ], "source": [ "seg = seg_node.get_data()\n", "print(seg)\n", "\n", "for i, group in enumerate(seg.groups):\n", " print(f\"\\nGroup {i}: {group.name!r}\")\n", " for lbl in seg.get_group_labels(i):\n", " img = seg.get_group_image(i)\n", " voxels = int((img.array == lbl.value).sum())\n", " print(f\" value={lbl.value:2d} name={lbl.name!r:10s} \"\n", " f\"color={lbl.color} locked={lbl.locked} voxels={voxels}\")" ] }, { "cell_type": "code", "execution_count": 8, "id": "xe3msupzyub", "metadata": {}, "outputs": [ { "data": { "text/plain": "", "text/html": "
ValueNameColorVisibleLocked
Anatomy
1LiverTrueFalse
2SpleenTrueFalse
Findings
3TumorTrueTrue
" }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "seg" ] }, { "cell_type": "markdown", "id": "38cba29c", "metadata": {}, "source": [ "## 5. Edit label metadata\n", "\n", "Label properties such as color, visibility, and name are stored in the `MultiLabelSegmentation` object in Python. They are only written back to the Workbench when you call `node.set_data()` again (see step 8). Unlike node display properties, there is no live REST setter for individual label fields." ] }, { "cell_type": "code", "execution_count": 9, "id": "2fad4c45", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Liver color: (0.1, 0.85, 0.0)\n", "Spleen visible: False\n", "Tumor name: 'Primary Tumor'\n" ] } ], "source": [ "# Retrieve labels from the downloaded segmentation by value\n", "liver_lbl = seg.get_label(1)\n", "spleen_lbl = seg.get_label(2)\n", "tumor_lbl = seg.get_label(3)\n", "\n", "# Adjust Liver color and make Spleen semi-transparent in MITK's property sense\n", "liver_lbl.color = (0.10, 0.85, 0.0) # greenish\n", "spleen_lbl.visible = False # hide Spleen for now\n", "tumor_lbl.name = \"Primary Tumor\" # rename\n", "\n", "print(f\"Liver color: {liver_lbl.color}\")\n", "print(f\"Spleen visible: {spleen_lbl.visible}\")\n", "print(f\"Tumor name: {tumor_lbl.name!r}\")" ] }, { "cell_type": "code", "execution_count": 10, "id": "9umrquv8ubk", "metadata": {}, "outputs": [ { "data": { "text/plain": "", "text/html": "
ValueNameColorVisibleLocked
Anatomy
1LiverTrueFalse
2SpleenFalseFalse
Findings
3Primary TumorTrueTrue
" }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "seg" ] }, { "cell_type": "markdown", "id": "4ca10fcd", "metadata": {}, "source": [ "## 6. Dilate the tumor label with SimpleITK (optional)\n", "\n", "`get_group_image(index)` returns the group's pixel data as an `Image`. We convert it to a SimpleITK image, apply a binary dilation (growing the tumor mask by 2 voxels = 4 mm), and store the result back with `set_group_image()`.\n", "\n", "This pattern — download a group image, apply any filter, write back — is the general way to edit pixel content from Python.\n", "\n", "If SimpleITK is not installed, this cell is skipped gracefully." ] }, { "cell_type": "code", "execution_count": 11, "id": "71ca34cb", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Tumor before dilation: 925 voxels\n", "Tumor after dilation: 6127 voxels (+5202)\n" ] } ], "source": [ "try:\n", " import SimpleITK\n", "except ImportError:\n", " SimpleITK = None\n", " print(\"SimpleITK not installed — skipping dilation.\")\n", " print(\"Install with: pip install SimpleITK\")\n", "\n", "if SimpleITK is not None:\n", " findings_img = seg.get_group_image(g_findings)\n", " arr = findings_img.array\n", "\n", " # Build a binary mask: 1 where tumor, 0 elsewhere\n", " binary = (arr == tumor_lbl.value).astype(np.uint8)\n", " sitk_mask = SimpleITK.GetImageFromArray(binary)\n", " sitk_mask.SetSpacing([float(s) for s in seg.spacing])\n", "\n", " # Dilate by 2 voxels in each direction\n", " dilated = SimpleITK.BinaryDilate(sitk_mask, kernelRadius=[5, 5, 5])\n", " dilated_arr = SimpleITK.GetArrayFromImage(dilated)\n", "\n", " # Write dilated region back as tumor label value; leave other labels intact\n", " result = arr.copy()\n", " result[dilated_arr > 0] = tumor_lbl.value\n", " seg.set_group_image(g_findings, result)\n", "\n", " before = int(binary.sum())\n", " after = int(dilated_arr.sum())\n", " print(f\"Tumor before dilation: {before:5d} voxels\")\n", " print(f\"Tumor after dilation: {after:5d} voxels (+{after - before})\") " ] }, { "cell_type": "markdown", "id": "49db199b", "metadata": {}, "source": [ "## 7. Remove a label including its pixels\n", "\n", "`remove_label(value, clear_pixels=True)` removes a label from the registry **and** zeroes out every voxel carrying that value in the group's image. This is the cleanest way to delete a label — no orphaned pixel values remain.\n", "\n", "With `clear_pixels=False` the metadata is removed but the pixel values stay; useful when you intend to reassign them manually.\n", "\n", "Here we delete **Spleen** (value 2) from the Anatomy group." ] }, { "cell_type": "code", "execution_count": 12, "id": "dc98e507", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Before removal:\n", " Labels in Anatomy: [(1, 'Liver'), (2, 'Spleen')]\n", "\n", "After removal:\n", " Labels in Anatomy: [(1, 'Liver')]\n", " Spleen pixels remaining in image: 0 (expected 0)\n", " get_label(2) -> None\n" ] } ], "source": [ "print(\"Before removal:\")\n", "print(f\" Labels in Anatomy: {[(l.value, l.name) for l in seg.get_group_labels(g_anatomy)]}\")\n", "\n", "spleen_value = spleen_lbl.value # save before removal\n", "seg.remove_label(spleen_value, clear_pixels=True)\n", "\n", "print(\"\\nAfter removal:\")\n", "print(f\" Labels in Anatomy: {[(l.value, l.name) for l in seg.get_group_labels(g_anatomy)]}\")\n", "\n", "# Confirm pixels are gone\n", "arr = seg.get_group_image(g_anatomy).array\n", "remaining = int((arr == spleen_value).sum())\n", "print(f\" Spleen pixels remaining in image: {remaining} (expected 0)\")\n", "\n", "# Lookup after removal should return None\n", "print(f\" get_label({spleen_value}) -> {seg.get_label(spleen_value)}\")" ] }, { "cell_type": "code", "execution_count": 13, "id": "st8iwpi5r0m", "metadata": {}, "outputs": [ { "data": { "text/plain": "", "text/html": "
ValueNameColorVisibleLocked
Anatomy
1LiverTrueFalse
Findings
3Primary TumorTrueTrue
" }, "execution_count": 13, "metadata": {}, "output_type": "execute_result" } ], "source": [ "seg" ] }, { "cell_type": "markdown", "source": [ "## 8. Validate consistency and push changes back\n", "\n", "`validate()` checks internal consistency — that every pixel value in each group array maps to a registered label, that no value appears in multiple groups, and that the label-value space is free of gaps or overlaps. Run it before `set_data()` to catch mistakes early.\n", "\n", "After validation, `set_data(seg)` re-serializes the entire segmentation (all pixel groups plus label metadata) and uploads it. There is no partial-update endpoint — any label or pixel edit requires a full re-upload." ], "metadata": { "collapsed": false } }, { "cell_type": "code", "execution_count": 14, "id": "6286c684", "metadata": { "collapsed": false, "jupyter": { "outputs_hidden": false } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Validation OK — no consistency issues.\n", "\n", "Pushed modified segmentation back to 'Demo Segmentation'.\n", "The Workbench now shows:\n", " Group 0 'Anatomy': ['Liver']\n", " Group 1 'Findings': ['Primary Tumor']\n" ] } ], "source": [ "issues = seg.validate()\n", "if issues:\n", " print(\"Consistency warnings:\")\n", " for w in issues:\n", " print(f\" {w}\")\n", "else:\n", " print(\"Validation OK — no consistency issues.\")\n", "\n", "seg_node.set_data(seg)\n", "wb.reinit([img_node])\n", "print(f\"\\nPushed modified segmentation back to '{seg_node.name}'.\")\n", "print(\"The Workbench now shows:\")\n", "for i, group in enumerate(seg.groups):\n", " labels = seg.get_group_labels(i)\n", " print(f\" Group {i} '{group.name}': {[l.name for l in labels]}\")" ] }, { "cell_type": "markdown", "id": "6ei74n5mbvm", "metadata": {}, "source": [ "## 9. Save the segmentation to disk\n", "\n", "`save_data(path)` downloads the raw serialized bytes from the Workbench and writes them directly to a file without any parsing or re-encoding. The result is a standard MITK multilabel NRRD file containing the 4-D pixel array and the full label metadata, readable by MITK Workbench and any compatible tool." ] }, { "cell_type": "code", "execution_count": 15, "id": "829f5b75", "metadata": { "collapsed": false, "jupyter": { "outputs_hidden": false } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Saved to: demo_segmentation.nrrd\n", "File size: 9.7 kB\n" ] } ], "source": [ "from pathlib import Path\n", "\n", "output_path = Path(\"demo_segmentation.nrrd\")\n", "saved = seg_node.save_data(output_path)\n", "print(f\"Saved to: {saved}\")\n", "print(f\"File size: {saved.stat().st_size / 1024:.1f} kB\")" ] }, { "cell_type": "markdown", "id": "gpz21cuiiqb", "metadata": {}, "source": [ "## 10. Clean up\n", "\n", "Remove both nodes from the DataStorage. Passing `recursive=True` to the parent node (`img_node`) also removes `seg_node`, since it is registered as a child in the DataStorage tree." ] }, { "cell_type": "code", "execution_count": 16, "id": "5415b01b", "metadata": { "collapsed": false, "jupyter": { "outputs_hidden": false } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Done.\n" ] } ], "source": [ "img_node.remove(recursive=True) # removes img_node and its child seg_node\n", "output_path.unlink(missing_ok=True)\n", "print(\"Done.\")" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.11.0" } }, "nbformat": 4, "nbformat_minor": 5 }