Haptic Technology In Surgical Simulation and Graphical Training - ppt

 Haptic Technology In Surgical Simulation and Graphical Training

ABSTRACT

Engineering as it finds its wide range of application in every field not an exception even the medical field. One of the technologies which aid the surgeons to perform even the most complicated surgeries successfully is Virtual Reality.

Even though virtual reality is employed to carry out operations the surgeon’s attention is one of the most important parameter. If he commits any mistakes it may lead to a dangerous end. So, one may think of a technology that reduces the burdens of a surgeon by providing an efficient interaction to the surgeon than VR. Now our dream came to reality by means of a technology called “HAPTIC TECHNOLOGY”.

Haptic is the “science of applying tactile sensation to human interaction with computers”.

In our paper we have discussed the basic concepts behind haptic along with the haptic devices and how these devices are interacted to produce sense of touch and force feedback mechanisms. Also the implementation of this mechanism by means of haptic rendering and contact detection were discussed.

We mainly focus on ‘Application of Haptic Technology in Surgical Simulation and Medical Training’. Further we explained the storage and retrieval of haptic data while working with haptic devices. Also the necessity of haptic data compression is illustrated.

Introduction:

Haptic, is the term derived from the Greek word, haptesthai, which means ‘to touch’. Haptic is defined as the “science of applying tactile sensation to human interaction with computers”.  It enables a manual interaction with real, virtual and remote environment. Haptic permits users to sense (“feel”) and manipulate three-dimensional virtual objects with respect to such features as shape, weight, surface textures, and temperature.

A Haptic Device is one that involves physical contact between the computer and the user. By using Haptic devices, the user can not only feed information to the computer but can receive information from the computer in the form of a felt sensation on some part of the body. This is referred to as a Haptic interface.

In our paper we explain the basic concepts of ‘Haptic Technology and its Application in Surgical Simulation and Medical Training’.

Haptic Devices:

Force feedback is the area of haptics that deals with devices that interact with the muscles and tendons that give the human a sensation of a force being applied—hardware and software that stimulates humans’ sense of touch and feel through tactile vibrations or force feedback.

These devices mainly consist of robotic manipulators that push back against a user with the forces that correspond to the environment that the virtual effector’s is in.  Tactile feedback makes use of devices that interact with the nerve endings in the skin to indicate heat, pressure, and texture.  These devices typically have been used to indicate whether or not the user is in contact with a virtual object.  Other tactile feedback devices have been used to stimulate the texture of a virtual object. 

PHANToM:

A small robot arm with three revolute joints each connected to a computer-controlled electric DC motor. The tip of the device is attached to a stylus that is held by the user. By sending appropriate voltages to the motors, it is possible to exert up to 1.5 pounds of force at the tip of the stylus, in any direction. The PHANTOM interface from Sensible Technologies was one of the first haptic systems to be sold commercially. Its success lies in its simplicity. Instead of trying to display information from many different points, this haptic device simulates touching at a single point of contact. It achieves this through a stylus which is connected to a lamp-like arm. Three small motors give force feedback to the user by exerting pressure on the stylus. So, a user can feel the elasticity of a virtual balloon or the solidity of a brick wall. He or she can also feel texture, temperature and weight.           

                                                    

CYBER GRASP:

The CyberGlove is a lightweight glove with flexible sensors that accurately measure the position and movement of the fingers and wrist. The CyberGrasp, from Immersion Corporation, is an exoskeleton device that fits over a 22 DOF CyberGlove, providing force feedback. The CyberGrasp is used in conjunction with a position tracker to measure the position and orientation of the fore arm in three-dimensional space.

                        The Cyber Grasp system, another commercially available haptic interface from Immersion Corporation, takes a different approach. This device fits over the user's entire hand like an exoskeleton and adds resistive force feedback to each finger. Five actuators produce the forces, which are transmitted along tendons that connect the fingertips to the exoskeleton. With the Cyber Grasp system, users are able to feel the size and shape of virtual objects that only exist in a computer-generated world. To make sure a user's fingers don't penetrate or crush a virtual solid object, the actuators can be individually programmed to match the object's physical properties.

Haptics Continuum:

As a field of study, haptics has closely paralleled the rise and evolution of automation. Before the industrial revolution, scientists focused on how living things experienced touch. Biologists learned that even simple organisms, such as jellyfish and worms, possessed sophisticated touch responses. In the early part of the 20th century, psychologists and medical researchers actively studied how humans experience touch. Appropriately so, this branch of science became known as human haptics, and it revealed that the human hand, the primary structure associated with the sense of touch, was extraordinarily complex.

With 27 bones and 40 muscles, including muscles located in the forearm, the hand offers tremendous dexterity. Scientists quantify this dexterity using a concept known as degrees of freedom. A degree of freedom is movement afforded by a single joint. Because the human hand contains 22 joints, it allows movement with 22 degrees of freedom. The skin covering the hand is also rich with receptors and nerves, components of the nervous system that communicate touch sensations to the brain and spinal cord.

                                                                                                    

                                                                                                                          

Haptic Rendering:

           

It is a process of applying forces to the user through a force-feedback device. Using haptic rendering, we can enable a user to touch, feel and manipulate virtual objects. Enhance a user’s experience in virtual environment. Haptic rendering is process of displaying synthetically generated 2D/3D haptic stimuli to the user. The haptic interface acts as a two-port system terminated on one side by the human operator and on the other side by the virtual environment.

  

Contact Detection:

A fundamental problem in haptics is to detect contact between the virtual objects and the haptic device (a PHANToM, a glove, etc.). Once this contact is reliably detected, a force corresponding to the interaction physics is generated and rendered using the probe. This process usually runs in a tight servo loop within a haptic rendering system.

            Another technique for contact detection is to generate the surface contact point (SCP), which is the closest point on the surface to the actual tip of the probe. The force generation can then happen as though the probe were physically at this location rather than within the object. Existing methods in the literature generate the SCP by using the notion of a god-object, which forces the SCP to lie on the surface of the virtual object.

Application of Haptic technology           Haptic Technology as it finds it wide range of Applications some among them were mentioned below:

1.      Surgical simulation & Medical training.

2.      Physical rehabilitation.

3.      Training and education.

4.      Museum display.

5.      Painting, sculpting and CAD

6.      Scientific Visualization.

7.      Military application.

8.      Entertainment.

The role of Haptic Technology in“Surgical Simulation and Medical  Training” is discussed in detail below.

Surgical simulation and medical training:

         

Haptic is usually classified as:-

Human haptics: human touch perception and manipulation.

Machine haptics: concerned with robot arms and hands.

Computer haptics: concerned with computer mediated.

                                

A primary application area for haptics has been in surgical simulation and medical training. Haptic rendering algorithms detect collisions between surgical instruments and virtual organs and render organ-force responses to users through haptic interface devices. For the purpose of haptic rendering, we’ve conceptually divided minimally invasive surgical tools into two generic groups based on their functions.

1. Long, thin, straight probes for palpating or puncturing the tissue and for injection (puncture and injection needles and palpation probes) 

2.  Articulated tools for pulling, clamping, gripping, and cutting soft tissues (such as biopsy and punch forceps, hook scissors, and grasping forceps).

A 3D computer model of an instrument from each group (a probe from the first group and a forceps from the second) and their behavior in a virtual environment is shown. During real-time simulations, the 3D surface models of the probe and forceps is used to provide the user with realistic visual cues. For the purposes of haptic rendering of tool–tissue interactions, a ray-based rendering, in which the probe and forceps are modeled as connected line segments. ‘Modeling haptic interactions between a probe and objects using this line-object collision detection and response has several advantages over existing point based techniques, in which only the tip point of a haptic device is considered for touch interactions’.

                                           

·         Users feel torques if a proper haptic device is used. For example, the user can feel the coupling moments generated by the contact forces at the instrument tip and forces at the trocar pivot point.

·         Users can detect side collisions between the simulated tool and 3D models of organs.

·         Users can feel multiple layers of tissue if the ray representing the simulated surgical probe is virtually extended to detect collisions with an organ’s internal layers. This is especially useful because soft tissues are typically layered, each layer has different material properties, and the forces/torques reflected to the user depends on the laparoscopic tool’s orientation.

·         Users can touch and feel multiple objects simultaneously. Because laparoscopic instruments are typically long slender structures and interact with multiple objects (organs, blood vessels, surrounding tissue, and so on) during a MIS (Minimally Invasive Surgery), ray-based rendering provides a more natural way than a purely point-based rendering of tool-tissue interactions. To simulate haptic interactions between surgical material held by a laparoscopic tool (for example, a catheter, needle, or suture) and a deformable body (such as an organ or vessel), a combination of point- and ray-based haptic rendering methods are used. 

In case of a catheter insertion task shown above, the surgical tools using line segments and the catheter using a set of points uniformly distributed along the catheter’s center line and connected with springs and dampers. Using our point based haptic rendering method; the collisions between the flexible catheter and the inner surface of a flexible vessel are detected to compute interaction forces.

The concept of distributed particles can be used in haptic rendering of organ–organ interactions  (whereas a single point is insufficient for simulating organ–organ interactions, a group of points, distributed  around the contact region, can be used) and other minimally invasive procedures, such as bronchoscope and colonoscopy, involving the insertion of a flexible material into a tubular body .

Deformable objects:

One of the most important components of computer based surgical simulation and training systems is the development of realistic organ-force models. A good organ-force model must reflect stable forces to a user, display smooth deformations, handle various boundary conditions and constraints, and show physics-based realistic behavior in real time. Although the computer graphics community has developed sophisticated models for real-time simulation of deformable objects, integrating tissue properties into these models has been difficult. Developing real-time and realistic organ-force models is challenging because of viscoelasticity, anisotropy, nonlinearity, rate, and time dependence in material properties of organs. In addition, soft organ tissues are layered and nonhomogeneous.

Tool–tissue interactions generate dynamical effects and cause nonlinear contact interactions of one organ with the others, which are quite difficult to simulate in real  time. Furthermore, simulating surgical operations such as cutting and coagulation requires frequently updating the organ geometric database and can cause force singularities in the physics-based model at the boundaries. There are currently two main approaches for developing force-reflecting organ models:

1.      Particle-based methods.

2.       Finite-element methods (FEM).

In particle-based models, an organ’s nodes are connected to each other with springs and dampers. Each node (or particle) is represented by its own position, velocity, and acceleration and moves under the influence of forces applied by the surgical instrument.

In finite-element modeling, the geometric model of an organ is divided into surface or volumetric elements, properties of each element are formulated, and the elements are assembled together to compute the deformation states of the organ for the forces applied by the surgical instruments.

Capture, Storage, and Retrieval of Haptic Data:

The newest area in haptic is the search for optimal methods for the description, storage, and retrieval of moving-sensor data of the type generated by haptic devices. This

techniques captures the hand or finger movement of an expert performing a skilled movement and “play it back,” so that a novice can retrace the expert’s path, with realistic touch sensation; The INSITE system is capable of providing instantaneous comparison of two users with respect to duration, speed,  acceleration, and thumb and finger forces.

Techniques for recording and playing back raw haptic data have been developed for the PHANToM and CyberGrasp. Captured data include movement in three dimensions, orientation, and force (contact between the probe and objects in the virtual environment).

Issues in the usage of Haptics Technology

In the last 15 years or so in Web environment, we moved from static web pages to dynamic web pages and then to Database driven web pages. With the arrival of Haptics Technology, all this will end.  The servers would automatically talk to each other with the automated programs that are wholly dependent upon the state of the system. Various technical and social issues would crop up when Haptics is put to widespread usage in near future.

ü  Hard Real time Operating Systems would be needed.

ü  Backward integration software would be required.

ü  Network delay compensation in tele-haptic applications

ü  Social changes will also be major issue.

However, after this technology gains ground and is applied, this will usher in a new world order in which a lot of logical things would happen with the use of technical equipments.

Collaborative Hapto Virtual Environments would open new ways for human interaction, collaboration and learning. Thus, many miracles can be expected with the use of this technology in near future.

Applications of haptics Technology

Gaming is one of the first applications of haptics that is being realized. Many video games controllers such as force-feedback steering wheels and joystick already contain simple Haptics device to enable virtual rally drivers and pilots to feel the bumps of artificial roads or the rumble of machine guns.

The medical arena is the other popular area for applying the Haptics systems in making virtual, keyhole surgery and needle insertion simulators that provide a realistic pop as the needle enters the virtual vein.

Mobile phones of future would employ Haptics for recreating touch and texture through artificial stimuli. While the aviation industry, which is already using flight simulators to train their commercial pilots before handling actual aeroplanes with real passengers, can use the technology to make these trainings more worthwhile and useful.

Some of the other important application areas of Haptics are:

ü  Databases

ü  Networking

ü  Graphics

ü  Tele-collaboration

ü  E-commerce, etc

As the trend in processor power has been showing an upward swing, haptics technology will also receive a big boost in near future. Concisely speaking, the Haptics systems are going to provide:

ü  Sensing and manipulating objects

ü  A feel of being in control remotely

ü  Computer generated reality

ü  A new level of fidelity & convenience

ü  Means of exploring the new world of haptic interaction

ü  Convenience in GUI, gaming and training.

Haptic Data Compression:

           

Haptic data compression and evaluation of the perceptual impact of lossy compression of haptic data are further examples of uncharted waters in haptics research.

Data about the user's interaction with objects in the virtual environment must be continually refreshed if they are manipulated or deformed by user input. If data are too bulky relative to available bandwidth and the coding of an arbitrary trajectory in three-dimensional space. computational resources, there will be improper registration between what the user sees on screen and what he “feels.”

On analyzing data obtained experimentally from the PHANToM and the CyberGrasp, exploring compression techniques, starting with simple approaches (similar to those used in speech coding) and continuing with methods that are more specific to the haptic data. One of two lossy methods to compress the data may be employed:

One approach is to use a lower sampling rate; the other is to note small changes during movement. For example, for certain grasp motions not all of the fingers are involved.

Further, during the approaching and departing phase tracker data may be more useful than the CyberGrasp data. Vector coding may prove to be more appropriate to encode the time evolution of a multi-featured set of data such as that provided by the CyberGrasp. For cases where the user employs the haptic device to manipulate a static object, compression techniques that rely on knowledge of the object may be more useful than

 CONCLUSION:

We finally conclude that Haptic Technology is the only solution, which provides high range of interaction that cannot be provided by BMI or virtual reality. Whatever the technology we can employ, touch access is important till now. But, haptic technology has totally changed this trend. We are sure that this technology will make the future world as a sensible one.