The constantly changing and advancing 3D market also provides new innovative solutions for the medical and industrial sectors. The further development of the medical engineering industry is supported by the Federal Ministry for Economic Affairs and Energy (BMWi) with our latest project: 3DInMed.
Real-time depth and distance measurement by means of stereo endoscopyNew algorithms and demonstrators are being developed to enable a promising use of sterophotogrammetry for new application fields in the near future, within the framework of the 3D in Med project, which is funded by the BMWi. The article "3D-Endoskopie: Kinotechnik für die berührungslose Vermesseung", by Jean-Claude Rosenthal from the Fraunhofer Heinrich Hertz Institute HHI and Johannes Ruhhammer from SCHÖLLY FIBEROPTIC GmbH, was published in the German trade journal photonik. The article presents a new image-based stereomic system based on new algorithms. These algorithms allow real-time depth and distance measurement by means of stereoscopic endoscopy.
The, with stereo borescopes recorded, images can be displayed in 3D on conventional 3D displays via known stereo representation. The resulting improvement of the depth perception ensures an intuitive and quick navigation to the inspection site. The algorithms used for the depth and distance measurement is based on the principle of stereo triangulation, an established procedure in optical measurement.
An improved measuring methodology and easy user guidance through the in real-time generated disparity information are the advantages of stereoscopic measuring systems. Please find the detailed article available for download here.
3DInMed @ Control 2017Fraunhofer HHI and Schölly present a technical case study of a 3D endoscope used for online visual inspection tasks of hardly accessible parts at Control 2017.
Stereo image-based measurement for remote visual inspection
Download 3DInMed@Control 2017 Flyer
3DInMed @ Medica 2016Fraunhofer HHI presented first results of the 3IT project 3DInMed at Medica 2016. Digital stereo camera systems offer new possibilities to extract and visualize additional information resulting in significantly enhanced conditions for diagnostics and surgical interventions. Shown are the latest research results for measuring, tracking of medical instruments, augmented reality applications (AR) and optimized 3D playback.
A positive feedback was received from many sides, including engineers, surgeons and product managers. This result is a strong indicator that the developed methods and solutions of 3DInMed are of high relevance for digital 3D imaging technologies in medical applications.
Download 3DInMed@Medica 2016 Flyer
3DInMed describes an ongoing collaborated project of different partners, aiming to establish and develop immersive imaging technology for industrial (In) and medical (Med) applications. Including applications such as destruction-free industrial inspection- and measurement techniques, diagnostic and interventional radiology or the introduction of computer assisted designs (CAD) and further digitals tools for planning and production.
Existing technologies used by the entertainment industry, such as 3D-sensors, high-speed signal and imaging processors, high-resolution autostereoscopic displays, touchless interfaces and improved 3D-printers, are actuating those developments. The targeted technologies are more and more used in other application areas, such as planning, production or inspection processes in construction, trade and industry. Further, 3D imaging methods (e.g. endoscopy and microscopy) will change diagnosis and surgical processes strongly.
The project has essentially four main goals, including the development of new, solid processes of depth estimation and object detection as well as the design of high frequency and low latency transmission lines for 3D signals. Furthermore the development of real time processes to create endoscopic 3D panoramas through texture- and depth information from stereo-endoscopic views is essential. As well as the development of spatial mapping processes and measurement methods for 3D microscopy/endoscopy data and related AR-applications are planned.
Results and Partners
The main objective is to develop, evaluate and validate key technologies for 3D-capturing, -processing, -transmission and autostereoscopic visualization and integrate these technologies into and beyond the targeted application areas. The consortium is led by the Fraunhofer Society Munich and the Fraunhofer Institute for Telecommunications, Heinrich Hertz Institute. The project partners are ARRI Medical, C.R.S. iiMotion, Fraunhofer HHI, Fraunhofer IIS, Schölly, SeeFront and Solectrix. Ultimately the project has associated partners in particular the Technical University Munich / Hospital “rechts der Isar”, research group for Minimal Invasive, Interdisciplinary, Therapeutic Intervention (MITI); Ludwig-Maximilians-University Munich, Großhadern Campus, Cochlea-Implant-Centre; Neurosurgical clinic and polyclinic, hospital “rechts der Isar”, Technical University Munich; University Hospital Erlangen and Siemens Energy.
3D applications in industry are primarily used in the field of automotive and robotics by improving driver assistant systems, autonomous navigation and related capturing of surroundings. Touchless inspection and measurement processes of inaccessible tubular parts are another field of application. Supported by 3D visualizations in computer assisted designs, these inspection and diagnostic applications are used as well in 3D X-Ray or CT-Volume of chemical or biological structures.
3D applications in a medical environment enables doctors to record stereoscopic data and its three dimensional visualizations to guarantee an enhanced and improved workflow for minimal invasive operations. Associated technologies are surgery-microscopy, spatial approximation of operating areas, recognition and labelling of risk tissues (e.g. nerve tract) and robot or telemanipulator supported operations. Furthermore it aims at enabling the combination of preoperative and intraoperative diagnostic images, for documentation and education purposes.
Copyright: Fraunhofer IIS - (a) Left image of a typical 3D laparoscope (b) L-HRM disparity map (c) 3D reconstructed point cloud - 2 instruments have been removed
Fraunhofer HHI: (a) Real-time tracking result of instrument tip inside a temporal bone model. (b) Related result of real-time depth map estimation.
Events And ConferencesCARS 2017, Barcelona
CARS 2017-Computer Assisted Radiology and Surgery, 31th International Congress and Exhibition Barcelona, Spain, June 20-24, 2017.
New Tools for Digital Surgical Microscopy
N. Gard, S. Jurk, A. Schneider, P. Kauff, P. Eisert, J.-C. Rosenthal, Fraunhofer Heinrich-Hertz-Institut, Berlin; ARRI, München
Please note: the conference language was German therefore some of the following titles are also in German language.
DGE-BV MEETS ENDOSKOPIE LIVE in Berlin, 06.04 - 08.04.2017
Herausforderungen, Lösungen und Potenziale der Stereo- Bildverarbeitung für die 3D-Endoskopie
Jean-Claude Rosenthal, Niklas Gard
SID-ME Fall Meeting 2016 in Berlin, 03.11 - 04.11.2016
Depth-Based Quality Control for Medical Applications
Jean-Claude Rosenthal, Niklas Gard, Peter Kauff
15th CURAC Annual Conference in Bern, 29.09. - 01.10.2016
Session: Endoscopy & Microscopy
3D-Reconstruction by Polarization Imaging in Endoscopy
J. Sandvoss, T. Wittenberg, A. Nowak, J. Ernst
Evaluierung von 3D-Rekonstruktionsverfahren in der Stereo-Laparoskopie
D. Erpenbeck, T. Wittenberg, J. C. Rosenthal, P. Kauff, N. Lemke, T. Bergen
Special session: Optische 3D-Endoskopie & Mikroskopie
Johannes Ruhammer, Niels Lemke, Schölly Fiberoptic GmbH, Denzlingen
3D-Techniken in der Klinik
Dirk Wilhelm, Nils Kohn, TU München
Potential der digitalen 3D Operationsmikroskopie
Armin Schneider, ARRI Medical, München
Von der 2D zur 3D Panorama-Endoskopie
Daniel Erpenbeck, Thomas Wittenberg, Fraunhofer IIS, Erlangen
Tiefenbasierte Stereo-Bildverarbeitung für medizinische Anwendungen
Peter Kauff, Niklas Gard, Jean Claude Rosenthal, Fraunhofer HHI, Berlin
14th CURAC Annual Conference in Bremen, 17.09. - 19.09.2015
Session: Optische 3D-Modalitäten in der Chirurgie
3D-Verarbeitung und Visualisierung mit Optischen Modalitäten
Jean-Claude Rosenthal, Fraunhofer HHI, Berlin
Die Digitalisierung der Operationsmikroskopie
Christoph Bichlmeir, ARRI, München
Aspekte der 3D Endoskopie
Niels Lemke, Schölly, Denzlingen
Panorama Endoskopie - von 2D zu 3D
Thomas Wittenberg, Fraunhofer IIS
Technical University Munich, Hospital “rechts der Isar”, Research Group for Minimally invasive Interdisciplinary Therapeutic Intervention (MITI)
Neurosurgical clinic and polyclinic, hospital “rechts der Isar”, Technical University Munich
University Hospital Erlangen