ChainMail based neural dynamics modeling of soft tissue deformation for surgical simulation

1.Introduction

Modeling and simulation of soft tissue deformation is a fundamental research issue in the development of a surgical simulator. Surgical simulation requires realistic modeling of soft tissue deformation under tool-tissue interactions and real-time visual and haptic feedbacks [1], but it is challenging to satisfy these two conflicting requirements. Currently, the existing methods for soft tissue deformation can be grouped into two main categories. One is devoted to the computational performance such as mass-spring model (MSM) [2] for real-time soft tissue deformation, whereas the other is devoted to the physical realism such as finite element method (FEM) [3] for accurate soft tissue deformation. The former is easy in implementation and computationally efficient, but it cannot accurately reproduce the material properties of soft tissues. The latter allows accurate prediction of soft tissue deformation, but it is highly expensive in computation. There have been various techniques studied for the improvement of the runtime computation of FEM. Matrix condensation [4] improves the runtime computation by reducing the full computation of a volumetric object to the surface nodes only; however, with this simplification the deformation accuracy is significantly compromised. Pre-computation reduces the computational load by calculating the solutions [5] or spatial derivatives [6] prior to the simulation; however, the pre-computation does not permit changes on model topology during the simulation. Total Lagrangian explicit dynamics finite element algorithm [6] reduces the computation through eliminating the iterative process of solving the matrix equation; however, due to the use of explicit time integration, the simulation time step is restricted to a small value for the solution to maintain stable. The boundary element method (BEM) [7] reduces the computation by formulating the weak form of virtual work into a surface integral form, where only a discretization of object’s boundary is required. However, it cannot handle the anisotropic and heterogeneous characteristics of soft tissues due to the homogeneous material assumption. Meshless method [8], despite of being able to avoid the connectivity constraints from using mesh, requires more efforts in computer programming, and it cannot directly model the object’s surface. Although the GPU (Graphics Processing Unit) can be employed to facilitate the runtime computation from the GPU parallel computing [9], this technique is hardware-dependent and it does not fundamentally address the computational problem.

As an alternative to the aforementioned methods, ChainMail is a modeling method for real-time modeling and simulation of soft tissue deformation. It was first proposed by Gibson in 1997, named the 3D ChainMail [10]. In this method, each chain element (mass point) enforces a geometric region setting the free-moving distances of each of its neighbouring chain elements. The ChainMail has advantages in computation and is stable in numerical iteration. Although various studies have been reported for improvement of the ChainMail method [11, 12], it is still limited to linear deformation of soft tissues.

Neural network has also received attention for simulation of soft tissue deformation, given its fast computational advantage, which would be able to achieve the real-time computational performance required by surgical simulation. Zhong et al. reported a cellular neural network model [13] and a Hopfield neural network model [14] for modeling of soft tissue deformation; however, these neural networks are constructed based on the physical heat conduction process, rather than continuum mechanics for deformation.

In this paper, a new ChainMail based neural dynamics approach for modeling of soft tissue deformation is presented. It combines the ChainMail mechanism with nonlinear neural dynamics for the soft tissues’ nonlinear deformation and typical mechanical behaviors. It endows the principle of continuum mechanics to the neural network for soft tissue simulation by formulating the local connectivity of cells in the cellular neural network as the local position adjustments of ChainMail. It also improves ChainMail linear deformation with nonlinear neural dynamics. Experiments, simulations and comparisons have been performed to comprehensively evaluate the performance of the proposed method.

2.Model design

The proposed method employs the cellular neural network (CNN), which is a local-interconnected array-computing structure [15]. The neuron in the CNN is called cell, which is a nonlinear dynamic processing unit consisting of capacitors, resistors and current sources of linear and nonlinear types. Cells are locally connected and interact only with their nearest neighbors [16]; cells that are not directly connected affect each other indirectly via the global propagation effect of CNN [17].

Figure 1.

A CNN on an irregular grid: the spatial positions of 𝐏i and 𝐏j are occupied by cells C⁢(i) and C⁢(j).

The CNN can be applied to any type of geometric grid of any dimension. Consider a geometric grid shown in Fig. 1, points i and j at positions 𝐏i and 𝐏j are occupied by cells C⁢(i) and C⁢(j), respectively.

To describe the interaction range between cells, the neighborhood Nr⁢(i) of cell C⁢(i) is firstly defined by

where r is a positive integer number denoting the number of edges between cells C⁢(i) and C⁢(j).

The dynamic behaviors of cell C⁢(i) are governed by the following equations

where C is the cell capacitance, which can be set to 1 for simplicity; Rx is the cell resistance; Ii is the current source; r is 1 in our case; A⁢(i;j) is the feedback template which defines the interactions between neighboring cells, whereas B⁢(i;j) is the control template which characterizes the influence of input on the cell; vu⁢i⁢(t), vx⁢i⁢(t) and vy⁢i⁢(t) are the cell input, state and output at time t; vy⁢i⁢(t) is a nonlinear sigmoid function of vx⁢i⁢(t), and it is bounded by a constant K.

Without cell input vu⁢i, Eq. (2) is reduced to an autonomous CNN [18] whose governing equation is given by

3.Model construction

3.1ChainMail formulation of local connectivity of cells

The CNN and ChainMail share common characteristics. Under the given initial conditions and external inputs, the dynamic behaviors of the proposed CNN are described by the local connectivity of cells. Similarly, the behaviors of ChainMail are also described by the local position adjustments under the same conditions. Further, similar to the CNN output, which is bounded by the constant K, the movement of a chain element is bounded by the maximum extension and minimum compression lengths. The position of a chain element will be adjusted locally only if the two lengths are violated, to keep the chain link between the two chain elements within the geometric bounding region. Therefore, in this paper, the local connectivity of cells in the CNN is formulated according to the local position adjustments of ChainMail. From the perspective of continuum mechanics of elasticity, the ChainMail can be viewed as a spring system in which a spring’s length is bounded by its minimum compression and maximum extension lengths. Accordingly, the formulation of CNN’s local connectivity as the ChainMail local position adjustments enables the CNN dynamics to follow the principle of continuum mechanics for soft tissue deformation.

Traditional ChainMail method [10, 11] defines the compression and extension lengths with respect to (w.r.t) the coordinate axes x, y and z individually to regulate the movement of chain elements. However, unlike the traditional ChainMail method, it is straightforward to set the minimum and maximum limiting lengths w.r.t the length of the chain link connecting the two chain elements at the rest state, which is similar to a spring system. Further, a material parameter α is introduced for setting the minimum and maximum limiting lengths, and its value is set according to the spring stiffness k.

Define the initial length of the chain link connecting chain elements iand j to be li⁢j0. The minimum compression length li⁢jm⁢i⁢n and maximum extension length li⁢jm⁢a⁢x can be expressed as

(5)

li⁢jm⁢i⁢n=(1-α)⁢li⁢j0

li⁢jm⁢a⁢x=(1+α)⁢li⁢j0

Hence, the current length li⁢j between the current positions of chain elements i and j is bounded by

(6)

li⁢jm⁢i⁢n⩽li⁢j⩽li⁢jm⁢a⁢x

Figure 2.

Position adjustments Δ⁢𝐏i and Δ⁢𝐏j are applied to chain elements’ position 𝐏i and 𝐏j to adjust the current length between the two chain elements back to the (a) minimum compression length li⁢jm⁢i⁢n and (b) maximum extension length li⁢jm⁢a⁢x.

Define the position adjustments for chain elements i and j to be Δ⁢𝐏i and Δ⁢𝐏j. As illustrated in Fig. 2, position adjustments are applied to both chain elements to adjust the current length li⁢j back to li⁢jm⁢i⁢n or li⁢jm⁢a⁢x if the current length is less than the minimum compression length or larger than the maximum extension length. The adjustments of li⁢jm⁢a⁢x are expressed by Eq. (7). The adjustments of li⁢jm⁢i⁢n can be expressed similarly by substituting li⁢jm⁢a⁢x with li⁢jm⁢i⁢n into Eq. (7).

(7)

Δ⁢𝐏i=12⁢(||𝐏j-𝐏i||-li⁢jm⁢a⁢x)⁢𝐏j-𝐏i||𝐏j-𝐏i||

Δ⁢𝐏j=-12⁢(||𝐏j-𝐏i||-li⁢jm⁢a⁢x)⁢𝐏j-Pi||𝐏j-𝐏i||

For the sake of conciseness, consider a simple case where a chain element is connected with four neighbors as illustrated in Fig. 3.

Figure 3.

A grid with four local connections: chain element i at position 𝐏i is connected with four neighboring chain elements at positions 𝐏j1, 𝐏j2, 𝐏j3 and 𝐏j4 with distances in-between denoted by u1, u2, u3 and u4, respectively.

The net adjustment for chain element i can be expressed by the sum of the adjustments applied to 𝐏i

(8)

∑Δ⁢𝐏i=μ1⁢𝐏j1+μ2⁢𝐏j2+μ3⁢𝐏j3+μ4⁢𝐏j4-(μ1+μ2+μ3+μ4)⁢𝐏i

where

(9)

μn=12⁢((un-li⁢jn,m⁢a⁢x)un⁢δnm⁢a⁢x+(un-li⁢jn,m⁢i⁢n)un⁢δnm⁢i⁢n);n=1,2,3,4

where

(10)

un=||𝐏jn-𝐏i||;δnm⁢a⁢x={1;if(un>li⁢jn,m⁢a⁢x)0;if(un⩽li⁢jn,m⁢a⁢x);δnm⁢i⁢n={1;if(un<li⁢jn,m⁢i⁢n)0;if(un⩾li⁢jn,m⁢i⁢n)

By associating the cell state vx⁢i⁢(t) with the chain element’s position 𝐏i, the feedback template A of the proposed CNN can be expressed as

(11)

A=(0μ20μ31Rx-(μ1+μ2+μ3+μ4)μ10μ40)

In case that chain element i is connected to any number of neighboring chain elements, the net adjustment for chain element i can be expressed by the sum of the adjustments applied to 𝐏i, i.e.

(12)

∑Δ⁢𝐏i=∑𝐏j∈N⁢(𝐏i)μi⁢j⁢𝐏j-∑𝐏j∈N⁢(𝐏i)μi⁢j⁢𝐏i

where N⁢(𝐏𝐢) is the set of neighboring chain elements of chain element i, and μi⁢j is given by

(13)

μi⁢j=12⁢((ui⁢j-li⁢jm⁢a⁢x)ui⁢j⁢δi⁢jm⁢a⁢x+(ui⁢j-li⁢jm⁢i⁢n)ui⁢j⁢δi⁢jm⁢i⁢n)

where

(14)

ui⁢j=||𝐏j-𝐏i||;δi⁢jm⁢a⁢x={1;if(ui⁢j>li⁢jm⁢a⁢x)0;if(ui⁢j⩽li⁢jm⁢a⁢x);δi⁢jm⁢i⁢n={1;if(ui⁢j<li⁢jm⁢i⁢n)0;if(ui⁢j⩾li⁢jm⁢i⁢n)

Similar to Eq. (11), the feedback template A can be written as

(15)

A⁢(i;j)=μi⁢j

A⁢(i;i)=1Rx-∑C⁢(j)∈Nr⁢(i)μi⁢j

3.2CNN current source, initial and boundary conditions

When a soft tissue is deformed, there is a displacement experienced at the region of contact. Hence, the current source Ii is set to the input displacement at the contact point i, whereas its value is set to zero at other points. The initial condition for the CNN is the positions of chain elements at the rest state. The boundary condition in the proposed method is the Dirichlet boundary condition which enforces fixed positions to the related chain elements of the solution domain at all times, and it is achieved by employing fixed-state cells.

Figure 4.

Deformations of a plane by (a) the proposed CNN and (b) traditional ChainMail method: the deformation produced by the proposed CNN shows nonlinear behaviors whereas it shows only linear behaviors with the traditional ChainMail.

Figure 5.

(a) Nonlinear force-displacement, (b) hysteresis and (c) stress relaxation observed from the proposed method.

4.Performance evaluation

The proposed CNN has been implemented into a prototype surgical simulation system for soft tissue deformation. Trials have been conducted with the proposed method to evaluate its performance in terms of soft tissues’ nonlinear deformation and typical mechanical behaviors. Figure 4 compares the deformations of a rectangular plane (21 × 21 nodes) between the proposed CNN and traditional ChainMail method. It can be seen that the deformation produced by the proposed CNN in Fig. 4a behaves nonlinearly while the traditional ChainMail in Fig. 4b shows a linear deformation of pyramid shape.

Figure 6.

Interactive deformation of the human liver model: (a) the interactive simulation system; (b) the initial state of the volumetric liver model; (c) the deformation by the proposed CNN; (d) the deformation by the traditional ChainMail.

Trials have also been conducted to verify the proposed CNN against the soft tissues’ typical mechanical behaviors, such as the nonlinear force-displacement relationship, hysteresis and stress relaxation [19]. Nonlinear force-displacement relationship was examined using a compression test with a displacement occurred at the contact point. Figure 5a demonstrates the nonlinear force-displacement relationship. Hysteresis was examined using the same compression test for loading, while the unloading was achieved by resorting the chain links between chain elements to their initial lengths. It can be seen from Fig. 5b that the variations of force w.r.t displacement followed two distinct paths during loading and unloading, this behavior is similar to the hysteresis effect measured from living biological tissues [19]. Stress relaxation was also examined by maintaining a constant displacement at the contact point. As shown in Fig. 5c, the internal force decreased asymptotically towards a minimum value. This behavior is in agreement with the stress relaxation observed from real soft tissues [19].

The proposed CNN has been integrated into a prototype surgical simulation for interactive deformation of virtual human organs with haptic feedback. Figure 6 illustrates the prototype surgical simulation system with a comparison of deformations modeled by the proposed CNN and traditional ChainMail. The volumetric human liver model contains 5762 mass points and 20255 tetrahedrons and it is deformed via a virtual haptic probe. It can be seen that the proposed CNN generates a better deformation shape than the ChainMail method.

5.Conclusion and future work

In this paper, a new ChainMail based neural dynamics approach is presented for modeling and simulation of soft tissue deformation for surgical simulation. It models the nonlinear deformation of soft tissues via the nonlinear neural dynamics of CNN through the formulation of local connectivity of cells as the local position adjustments of ChainMail. Results demonstrate that the proposed method can produce soft tissues’ nonlinear deformation as well as the typical mechanical behaviors. Future research will be devoted to two aspects for enhancement of the proposed method. One is the scalability. Algorithms will be developed to map multiple chain elements to one single neural cell to expand the proposed method to accommodate the increase of chain elements. The other is material parameter determination. Optimization algorithms will be developed to determine optimal material parameters for the nonlinear properties of soft tissues to further improve the modeling realism.

Conflict of interest

None to report.