Engineer Medical

Bal zs P V gv lgyi Mr

P R O F E S S I O NA L I N T E R E S T S

Image/Video Processing and Computer Vision; 2D/3D Medical Imaging; Real-time Data Processing and

Analysis; Computer Assisted Surgical Systems; Parallel Processing; Software Development; System

Design.

EXPERIENCE

3/2006-Present Sr. Software Engineer

Johns Hopkins University, ERC-CISST/LCSR, Baltimore MD, USA

My current employer, the Engineering Research Center for Computer-Integrated Surgical Systems and

Technology (ERC-CISST, http://www.cisst.org) is a multi-disciplinary research institution founded by

the National Science Foundation (NSF) and the Whiting School of Engineering at the Johns Hopkins

University (JHU). The institute develops surgical systems that integrate novel computer and human/

machine interface technologies that will revolutionize surgical procedures, extending the surgeon's

abilities to achieve better outcomes at lower costs. ERC-CISST/LCSR is one of the world s most

acknowledged and in uential research institutions on surgical robotics. As a member of the senior

engineering staff my responsibility is to contribute to the engineering research and facilitate the

implementation of novel medical applications. Based on my previous experience, I am mainly involved

in the real-time computer vision aspects of various projects. Our objective is to interpret the visual

information obtained from 2D and/or 3D image sources and help physicians during interventional

procedures.

Reference: Russell H. Taylor, Professor, ERC-CISST Director (***@***.***)

Peter Kazanzides, Assistant Research Professor, ERC-CISST/LCSR (****@***.***)

Gregory D. Hager, Professor, ERC-CISST Research Director (*****@**.***.***)

PROJECTS

CISST Software Infrastructure Development (ERC-CISST)

(Key words: Collaborative Cross-Platform Development, Real-Time Image Processing, Multithreading, Open

Source, C++, POSIX, Win32, X11, DirectShow, V4L2, LibDC1394, Matrox Imaging, CMake, Subversion)

The CISST software package (https://trac.lcsr.jhu.edu/cisst) is a collection of software libraries

designed to ease the development of computer assisted intervention systems. My main contribution

to the package is the computer vision and image processing infrastructure that provides a platform

independent interface (Windows, Linux, Mac OS X) for handling image sources, processing video,

and visualization. In early 2011 I have started a seminar series for graduate students on using the

CISST stereo vision libraries where they can learn the basics of using built in image lters and

develop new ones.

page 1 of 8

Microsurgical Workstation [5][22] (ERC-CISST, Wilmer Eye Institute of Johns Hopkins Hospital)

(Key words: System Design and Development, C++, Microsurgery, Vitreo-Retinal Surgery, Stereo Video

Microscopy, Computer Vision, Distributed Systems, Component Based Software Engineering)

The Microsurgical Workstation BRP Project is one of the largest projects of the CISST lab

currently. It involves the contribution of several medical and engineering faculty members and

number of graduate students. As member of the engineering staff I am responsible for managing

the design, development and integration of the stereo visualization system into the surgical

work ow. Due to the extreme precision required during microsurgery, accuracy and real-time

operation of the assisting computer system are paramount. The system also serves as a testbed for a

major system integration project which aims to connect all diverse computerized surgical devices in

a distributed, component based architecture.

Surgical Assistance Workstation [1][2][5][8][24] (ERC-CISST, Intuitive Surgical Inc.)

(Key words: System Design and Development, C++, Robotics, Computer Vision, Distributed Systems,

Component Based Software Engineering, VTK)

Built on the CISST software package, the Surgical Assistant Workstation (SAW) (https://

www.cisst.org/saw/Main_Page) provides a software framework that enables the integration of new

research results with surgical robotic systems. Although the da Vinci Surgical System from Intuitive

Surgical Inc. is a primary research platform used in the project, the SAW framework contains a

generic robot interface and can therefore be used with other robotic devices as well. The framework

consists of real-time robot control, tracking, real-time computer vision, and 3D visualization

functions. My responsibilities in the project include the design and implementation of video

capture, processing and live visualization functions with interactive 3D overlays.

Context Aware Surgical Assistance for Mentoring [1][2][9][10][11]][15][16][21] (ERC, TATRC)

(Key words: Computational Stereo, Dynamic Programming, 2D to 3D Registration, Real-Time Computer

Vision, Augmented reality)

Minimally invasive surgery (MIS) is a technique whereby instruments are inserted into the body via

small incisions (or in some cases natural ori ces), and surgery is carried out under video guidance.

While advantageous to the patient, MIS presents numerous challenges for the surgeon due to the

restricted eld of view presented by the endoscope. One means of overcoming some of these

limitations is to present the surgeon with addition visual information. The objective of the project

was to develop a system that provides the surgeon with a three-dimensional information overlay

registered to pre-operative or intra-operative volumetric data. The novelty of the system lies in its

use of stereo video data to perform the registration without recourse to an external tracking system.

A prototype version of the system for augmenting the surgical view during laparoscopic kidney

procedures has been demonstrated on video sequences recorded during animal and human

surgeries. Under the supervision of an engineering faculty member and an experienced cardiac

surgeon, I was the sole designer of the computer vision algorithms involved.

Visual Haptic Feedback [6][7][8][23] (ERC-CISST Haptic Exploration Laboratory)

(Key words: Tele-Operation, Stereo Video Augmented Reality, Computational Stereo, VTK, 3D Calibration of

the Robotic Vision System)

Currently the da Vinci tele-operated surgical robot lacks the ability to provide haptic feedback for

the surgeon. In the Haptic Exploration Laboratory we have developed a system that measures the

forces exerting upon the slave manipulator and visualizes the force magnitude on the stereoscopic

display of the master console. The augmented reality overlay is rendered in 3 dimensions on top of

the live endoscopic video and it is registered to the 3D robot frame, thus the overlay closely follows

the position of the surgical tool tip. In another implementation, the 3D geometry of the cameras

view is reconstructed using a dense stereo reconstruction and the resulting 3D triangle mesh is

displayed on top of the live video. The mesh is later color-coded according to the measured stiffness

values, eventually resulting in a stiffness map of the anatomy. My contribution to the project was the

design and development of the entire computer vision and visualization system and the registration

between the robot, the anatomical phantoms and the video streams.

page 2 of 8

Data Acquisition System [14] (ERC-CISST)

(Key words: Windows, MFC GUI, Multi-Channel Video Capture, External Trackers, da Vinci)

Recording live surgical data from a variety of data sources represent signi cant challenges. First, the

raw amount of lossless data requires a high performance pipeline that is capable of handling

multiple uncompressed multi-channel video and tracking streams simultaneously. On the other

hand, the labeling of different streams has to be done at acquisition time so that the streams remain

synchronized even after hours of continuous recording. Synchronization is crucial for successful

post-processing of multi-modality data set. I developed a recording system that consist of a high

performance Windows workstation, image acquisition boards and a speciality software that is

capable of interfacing with external optical (Polaris, OptoTrak) and electro magnetic (Aurora)

trackers, video capture sources, and the da Vinci surgical robot. The developed software and the

workstation has already been used to successfully record hundreds of hours of live surgeries, in

order to analyzing the Language of Surgery in a related research project.

3/2008-5/2008 HTML, DOM, Javascript Software Development

Freeware Web Applications for Apple s iPhone & iPod Touch

I developed three Web Applications for Apple s handheld devices. Like all other Web Applications, each

of these games run inside the Safari web browser. For the projects I have implemented a full featured

3D graphics engine in Javascript using the HTML5 Canvas element. At the time of their release these

games represented a previously unknown category (3D vector graphics) in terms of graphics on these

devices. The applications proved to be very popular and they are listed on Apple s website and a number

of other technology blogs and portals on the Internet. Rubik s Cube 3D (http://www.vagvolgyi.com/

rubikscube/) was Staff Pick, Featured and Most Popular Web Application for a week on http://

www.apple.com/webapps/ . Stunt Car 3D ( http://www.vagvolgyi.com/stuntcar/ ) also won the

recommendation ( Staff Pick ) award. Each Tap-a-Brick 3D (http://www.vagvolgyi.com/tab3d/), Stunt

Car 3D, and Rubik s Cube 3D got praising reviews on technology and gaming websites all around the

world. As of March 2010, approximately 500,000 people have tried these games online.

2/2003-2/2006 Image Quality Systems Engineer

General Electric Healthcare, Budapest, Hungary

GE Healthcare (GEHC, http://www.gehealthcare.com) is the global market leader of medical imaging

systems. As a Systems Engineer in the Image Quality (IQ) System team in the digital vascular X-ray

imaging modality my task was to simulate, analyze, evaluate, and improve the image quality of the most

sophisticated fully digital medical imaging systems (Innova 2000, 2100, 3100, 4100). This work

incorporates the extensive knowledge of medical imaging equipments and advanced digital image

processing algorithms. The engineering work is carried out under strict quality regulations using the Six

Sigma methodology.

Reference: Francois De Gaulmyn, Vascular Systems Manager (********.*********@***.**.***)

PROJECTS

Vascular X-ray Imaging Pipeline Simulator

(Key words: Cross-Platform Development, C++, FLTK GUI, Image Processing)

The imaging pipeline of GE Healthcare s at-panel vascular X-ray systems consists of a series of

image processing algorithms to correct detector irregularities and to enhance the visibility of X-ray

images. During the IQ veri cation process the imaging algorithms need to be tested and evaluated

for FDA approval. One of my main responsibilities at GEHC was to maintain and further develop

the cross-platform Image Processing Simulator software used for veri cation. The software was

thoroughly tested and development process was fully documented in order to be auditable.

page 3 of 8

Vascular X-ray Image Quality Veri cation

(Key words: FDA Compliance, System Design Veri cation, Testing)

As an integral part of the FDA compliant development process, the products have to be veri ed at

the end of each development milestone. Being member of the IQ team, I participated in the IQ

veri cation process of several product. The process involved the evaluation of the automatic X-ray

exposure management algorithms and imaging pipelines.

Vascular X-ray Image Optimization Management [17]

(Key words: Calibration, Statistical Analysis, X-ray Exposure Management)

The Image Processing Pipeline of the X-ray systems is optimized by an automatic logic that is

customized for each hardware con guration. In order to provide the best possible image quality, the

system status and exposure parameters are constantly monitored and the logic continuously ne-

tunes the imaging pipeline. For new hardware con gurations the optimization logic needs to be

modi ed based on image quality measurements and statistical analysis. I was responsible for

performing the IP optimization process for several product revisions.

On-site Engineering Support for Installed Base and Pilot Systems

(Key words: Meeting Customers and Clinical Specialists, Troubleshooting, System Evaluation)

During my years at GEHC, I have provided on-site support in multiple instances when IQ expertise

was required for troubleshooting. In one occasion I represented the system design team during the

installation and the rst clinical tests of an Innova 2100 pilot system in a hospital in Virginia, USA.

9/1998-2/2003 Systems/Research & Development Engineer

Tateyama Laboratory Hungary, Budapest, Hungary

Tateyama System Laboratory, Toyama, Japan

Tateyama Laboratory Hungary Ltd. is a member of the Tateyama Kagaku Group based in Toyama, Japan

(http://www.tateyama.jp/eng/index.html). The group is formed by 10 companies in Japan, Malaysia,

Hungary, and Thailand. The Hungarian division was responsible for research and development on the

elds of Automated Manufacture of Industrial Robots, Of ce Automation and Video Surveillance

Systems. I joined the R&D team as an intern graduate student in 1998 and was hired of cially in 2001 as

a full-time employee and team leader. I was responsible for managing the development team, carrying

out image processing (IP) development for surveillance systems and research unique IP algorithms for

future products. During my years at Tateyama, I spent months at the Japanese operations (mostly in

Toyama) in several occasions for direct collaboration in various projects. Moreover I am the co-author of

three Japanese patents on panoramic image processing and on motion detection on video.

Reference: Tsuneo Morita, Managing Director (******@********.**)

PROJECTS

Digit-Eye Video Surveillance System

(Key words: Image Processing, Motion Detection, Multi-Channel Video, Video Database Indexing, Web

Server Development, C++, Windows, MFC GUI, DirectShow, SourceSafe, HTML, HDML, WAP)

The Digit-Eye video surveillance system had two versions. One of them was a designed as a home

surveillance and security solution running on any Windows personal home computers. It featured

multiple USB camera video sources, automatic motion detection, video recording of suspicious

activities, video log Playback interface, mobile phone noti cations, and a Web based detection log

with screen captures and event descriptions. The other version aimed to provide a reliable,

distributed surveillance solution for small enterprises by supporting at most 16 professional quality

cameras and frame grabbers. Its feature set included all of the home version s with some additional

enterprise features, for example multiple layers of permissions for system administration. My

responsibilities in the small development team were extensive from designing the imaging pipeline

and image processing algorithms, through the user interface design, to actual C++ coding.

page 4 of 8

Panoramic Image Manipulation [25][26]

(Key words: Optical Calibration, Image Recti cation, Image Processing, C++, Java, Windows, PalmOS)

The company was continuously developing novel applications for its patented Panoramic Annular

Lens and for it s PalmOS software business. As the leader of the R&D team I was involved in several

feasibility studies and product developments, most of them related to computer vision/image

processing. Such as optical calibration, image recti cation, low-bandwidth video streaming, mobile

imaging, and panoramic video processing.

E D U C AT I O N

Computer Science (IT/Image Processing) Master s Degree, 2000

PhD studies [18][19][20] (till 2001, not completed)

University of Pannonia (formerly University of Veszpr m), Veszpr m, Hungary

Advisor: Tam s Szir nyi, Professor (http://www.sztaki.hu/~sziranyi/)

TEACHING EXPERIENCE

Image Processing/Computer Vision course, lectures, 2000-2001

Image Processing/Computer Vision course, laboratory exercises, 1999-2001

A WA R D S

University Scholarship for Excellence, 1999-2000

First prize and the special prize of the Hungarian Chamber of Trade and Commerce, 1999

On the Conference of the Science Ring of Students for Real-time motion analysis on panoramic

video streams .

COURSES

Surgery for Engineers 2008

Johns Hopkins Hospital, Minimally Invasive Surgical Training Center, Baltimore, MD

Six Sigma DMAIC and Design for Six Sigma Green Belt Certi cation, 2004

General Electric Healthcare, Budapest, Hungary

Medical Ionizing Radiation (X-ray) Safety Training 2004

General Electric Healthcare, Budapest, Hungary

MEMBERSHIPS

IEEE (Institute of Electrical and Electronics Engineers) Member since 2006

IEEE Computer Society Member since 2009

SKILLS

SKILL NAME SKILL LEVEL LAST USED EXPERIENCE

Systems Engineering Experienced Currently used 8 years

Six Sigma Green Belt 5 year ago 3 years

Image Processing, Computer Vision Experienced Currently used 11 years

Medical Imaging Experienced Currently used 8 years

ANSI C/C++ Experienced Currently used 11 years

page 5 of 8

SKILL NAME SKILL LEVEL LAST USED EXPERIENCE

POSIX Intermediate Currently used 5 years

Win32, MFC Experienced Currently used 12 years

Mac OS X, Objective-C Beginner Currently used

Assembly (x86, MMX) Intermediate 6 year ago 5 years

TCP/IP (Winsock32, Berkeley Sockets) Intermediate Currently used 7 years

Multithreading (Win32, POSIX) Experienced Currently used 8 years

OpenCV Intermediate Currently used 5 years

DirectShow Experienced Currently used 10 years

DC1394 & Video4Linux2 libraries Intermediate Currently used 5 years

libPNG, libJPEG, libZ Intermediate Currently used 2 years

Matrox Imaging Library Intermediate Currently used 4 years

GLUT, OpenGL Intermediate 3 years ago 2 years

CMake Intermediate Currently used 5 years

VTK Beginner Currently used 4 year

CVS, Subversion Experienced Currently used 6 years

Matlab Beginner Currently used 5 years

Latex Intermediate Currently used 5 years

HTML, CSS, Javascript, DOM Intermediate Currently used 10 years

J O U R NA L A RT I C L E S

[1] Balazs Vagvolgyi, Carol Reiley, Greg Hager, Russell Taylor, Li-Ming Su. Augmented Reality Using

Registration of 3D Computed Tomography to Stereoscopic Video of Laparoscopic Renal Surgery, The

Journal of Urology, April 2008 (Vol. 179, Issue 4, Supplement, Pages 241-242)

[2] Li-Ming Su, Balazs Vagvolgyi, Rahul Agarwal, Carol E. Reiley, Russell Taylor, Gregory Hager.

Augmented Reality During Robot-assisted Laparoscopic Partial Nephrectomy: Toward Real-Time 3D-CT

to Stereoscopic Video Registration, Urology, April 2009 (Vol. 73, Issue 4, Pages 896-900)

C O N F E R E N C E P R E S E N TAT I O N S & P O S T E R S

[3] Rajesh Kumar, Amod Jog, Balazs Vagvolgyi, Tom Tantillo, G.C.C. Chen, Hiep Nguyen, Gregory Hager,

David Yuh. Operational Skill Assessment for Robotic Surgery Training, 11th Society of American

Gastrointestinal and Endoscopic Surgeons (SAGES), 2011, San Antonio, TX

[4] Kelleher Guerin, Balazs Vagvolgyi, G.C.C. Chen, David Yuh, Rajesh Kumar. ReachIN: A Modular

Vision Based Interface for Teleoperation, International Workshop on Systems and Architectures for

Computer Assisted Interventions at MICCAI (Medical Image Computing and Computer-Assisted

Intervention) 2010, Beijing, China

[5] Peter Kazanzides, Simon DiMaio, Anton Deguet, Balazs Vagvolgyi, Marcin Balicki, Caitlin Schneider,

Rajesh Kumar, Amod Jog, Brandon Itkowitz, Christopher Hasser, Russell Taylor. The Surgical Assistant

Workstation (SAW) in Minimally Invasive Surgery and Microsurgery, International Workshop on

Systems and Architectures for Computer Assisted Interventions at MICCAI (Medical Image Computing

and Computer-Assisted Intervention) 2010, Beijing, China

page 6 of 8

[6] Tomonori Yamamoto, Balazs Vagvolgyi, Kamini Balaji, Louis L. Whitcomb, Allison M. Okamura. Tissue

property estimation and graphical display for teleoperated robot-assisted surgery, IEEE ICRA

(International Conference on Robotics and Automation) 2009: 4239-4245

[7] James Gwilliam, Mohsen Mahvash, Balazs Vagvolgyi, Alexander Vacharat, David Yuh, Allison

Okamura. Effects of haptic and graphical force feedback on teleoperated palpation, IEEE ICRA 2009:

677-682

[8] Balazs Vagvolgyi, Simon DiMaio, Anton Deguet, Peter Kazanzides, Rajesh Kumar, Christopher Hasser,

and Russell Taylor. The Surgical Assistant Workstation, Workshop on Systems and Architectures for

Computer Assisted Interventions at MICCAI (Medical Image Computing and Computer-Assisted

Intervention) 2008, New York, NY

[9] Balazs Vagvolgyi, Gregory Hager, Russell Taylor, and Li-Ming Su MD. Video to CT Registration for

Image Overlay on Solid Organs, Workshop on Augmented environments for Medical Imaging including

Augmented Reality in Computer-aided Surgery at MICCAI 2008, New York, NY

[10] Balazs Vagvolgyi, Gregory Hager, Russell Taylor, and Li-Ming Su MD. Augmented Reality Using

Registration of 3D Computed Tomography to Stereoscopic Video of Laparoscopic Renal Surgery, AUA

(Annual Meeting of the American Urological Association) 2008, Orlando, FL

[11] Balazs Vagvolgyi, Gregory Hager, Russell Taylor, and Li-Ming Su MD. Augmented Reality Using

Registration Using 3D CT to Stereoscopic Video of Laparoscopic Renal Surgery, EUS (Annual Meeting

of the Engineering & Urological Society) 2008, Orlando, FL

[12] Mohsen Mahvash, James Gwilliam, Rahul Agarwal, Balazs Vagvolgyi, Allison Okamura, David Yuh

MD, and Li-Ming Su MD. Force-Feedback Surgical Teleoperator: Controller Design and Palpation

Experiments, IEEE Haptics Symposium 2008, Reno, NV

[13] Ioana Fleming, Sandrine Voros, Balazs Vagvolgyi, Zach Pezzementi, James Handa MD, Russell Taylor,

and Gregory Hager. Intraoperative Visualization of Anatomical Targets in Retinal Surgery, IEEE WACV

(Workshop on Applications of Computer Vision) 2008, Copper Mountain, CO

[14] Carol Reiley, Henry Lin, Balakrishnan Varadarajan, Balazs Vagvolgyi, Sanjeev Khudanpur, David

Yuh, and Gregory Hager. Automatic Recognition of Surgical Motions Using Statistical Modeling for

Capturing Variability, MMVR (Medicine Meets Virtual Reality) 2008, Long Beach, CA

[15] Balazs Vagvolgyi, Carol Reiley, Gregory Hager, Russell Taylor, Adam Levinson MD, and Li-Ming Su

MD. Toward Direct Registration of Video to Computer Tomography for Intraoperative Surgical Planning

During Laparoscopic Partial Nephrectomy, WCE (World Congress of Endourology) 2007, Cancun,

Mexico

[16] Gregory Hager, Balazs Vagvolgyi, and David Yuh. Stereoscopic Video Overlay with Deformable

Registration, MMVR (Medicine Meets Virtual Reality) 2007, Long Beach, CA

[17] Berry Belanger, Farid Betraoui, Paritosh Dhawale, Priya Gopinath, Pal Tegzes, Balazs Vagvolgyi.

Development of next generation digital at panel catheterization system: design principles and

validation methodology, SPIE 2006, San Diego, CA

[18] Laszlo Czuni, Balazs Vagvolgyi, Tamas Sziranyi, Tamas Greguss. A Compact Panoramic Visual Sensor

for Intelligent Applications, ACCV (Asian Conference on Computer Vision) 2000, Taipei, Taiwan

[19] Ivan Kopilovic, Balazs Vagvolgyi, Tamas Sziranyi. Application of Panoramic Annular Lens for Motion

Analysis Tasks: Surveillance and Smoke Detection, ICPR (International Conference on Pattern

Recognition) 2000, Barcelona, Spain

page 7 of 8

[20] Ivan Kopilovic, Balazs Vagvolgyi, Tamas Sziranyi. Smoke-detection and Motion Sensitive Video

Surveillance Using Panoramic Annular Lens, KEPAF 2000, Noszvaj, Hungary

PAT E N T S AS CO-INVENTOR

[21] US PATENT #2010/020649 (pending):

A System for Registration and Information Overlay on Deformable Surfaces from Video Data

[22] US PATENT #2010/044596 (pending):

Programmable Multispectral Illumination System for Surgery and Visualization of Light-Sensitive

Tissues

[23] US PATENT #2009/061297 (pending):

Environment Property Estimation and Graphical Display

[24] US PATENT #200******** (pending):

Interactive User Interfaces for Robotic Minimally Invasive Surgical Systems

[25] JAPANESE PATENT #200-***-****:

Method and Device for Vertical Distortion Correction in Expansion of Annular Image into Panoramic

Image

[26] JAPANESE PATENT #200-***-****:

Method and Apparatus for Detecting Reference Position of Annular Image by Omnidirectional

Imaging

O T H E R P U B L I C AT I O N S

[27] Series of articles on Digital Image Processing in the Hungarian video professional magazine

Videopraktika, 2003-2005

Latest update: Saturday, April 30, 2011

page 8 of 8

Contact this candidate