Control It

SCIENCE CHINA

Information Sciences

. RESEARCH PAPERS . doi: 10.1007/s11432-011-4428-z

C2G2FSnake: automatic tongue image segmentation

utilizing prior knowledge

SHI MiaoJing1, LI GuoZheng1 & LI FuFeng2

1The Key Laboratory of Embedded System and Service Computing, Ministry of Education,

Department of Control Science and Engineering, Tongji University,

Shanghai 201804, China;

2Laboratory of Information Access and Synthesis of TCM Four Diagnosis,

Shanghai University of Traditional Chinese Medicine,

Shanghai 201203, China

Received August 28, 2010; accepted March 22, 2011

Abstract Extraction of the tongue body from digital images is essential for automated tongue diagnoses in

traditional Chinese medicine. This paper presents a fully automated active contour initial method that utilizes

prior knowledge of the tongue shape and its location in tongue images. Then colorspace information is introduced

to control curve evolution. Combining the geometrical Snake model with the parameterized GVFSnake model,

a novel approach for automatic tongue segmentation: C2 G2 FSnake (color control-geometric & gradient ow

Snake) is proposed. This method increases the curve velocity but decreases the complexity. C2 G2 FSnake

greatly extends practical usage to tongue segmentation, at the same time increasing the precision. Compared

with other state-of-the-art works using di erent images of tongue color, C2 G2 FSnake realizes automatic tongue

segmentation with greatly improved accuracy.

Keywords tongue diagnosis, tongue segmentation, color space-control, Snake, parameterized Snake, geomet-

ric Snake

Shi M J, Li G Z, Li F F. C2 G2 FSnake: automatic tongue image segmentation utilizing prior knowledge.

Citation

Sci China Inf Sci, 2011, doi: 10.1007/s11432-011-4428-z

1 Introduction

Tongue diagnosis is an indispensable part of Traditional Chinese Medicine (TCM). It is highly valued in

that much physiological and pathological information can be obtained by observing and analyzing features

of the tongue body such as color, texture and, coating. Recently, utilizing image processing and pattern

recognition technology, researchers have tried to establish objective and quantitative tongue diagnoses to

cope with the large number of clinical cases. Objectifying TCM has its pathognomonic signi cances. In

a TCM system, the tongue body must be rst automatically segmented after patients tongue pictures

have been collected with digital apparatus (eg. a digital camera). The key lies in combining tongue body

segmentation, tongue picture gathering and the characteristics of the tongue. Challenges existing in this

eld at present are the following:

i) Blending of colors between the tongue body and the face(especially the lips).

Corresponding author (email: ****@******.***.**)

c Science China Press and Springer-Verlag Berlin Heidelberg 2011 info.scichina.com www.springerlink.com

Shi M J, et al. Sci China Inf Sci

2

ii) Di erent shooting angles, circumstances, and patients, together with his/her tongue shape, all of

which are subjective factors a ecting image segmentation.

iii) As a soft tissue, the edges of the tongue body do not show a distinctive gradient descent compared

with hard objects.

iv) From the perspective of pathology, features such as tongue color, texture, coating add additive

di culties to image segmentation.

Many studies have been proposed in recent years aimed at solving these problems [1], Examples of

these include region growing which takes photographic shadow characteristics into consideration [2] and

the shortest path [3] is introduced to compensate the shortage of region growing, and combining multi-

hyper spectral features to train support vector machines for segmentation [4]. All these algorithms can

perform tongue segmentation under certain conditions. The most widely used method in this eld is the

active contour model (ACM) or Snake, which evolves an energy minimizing curve, initialized manually

or automatically, and which propagates to the real boundary by both internal and external forces. Works

on Snake mainly concentrate on two topics: 1) how to obtain the initial contours [5,6], and 2) how to

evolve the curve quickly and accurately [7,8].

Regarding initialization, Zuo et al. [9] proposed a polar edge detector to obtain a rough contour as

initialization, but this method is sensitive to image noise. Watershed has also been applied to tongue

segmentation as initialization [5], but its parameter settings are complex. Colorspace, especially H-

channel, has been widely used in Snake initialization [8], however, it only works well on red tongues. On

the other hand, research on curve evolution has not revealed remarkable properties that can de nitely

solve any representative problems such as local minima, over-learning and weak gradients. Nevertheless

some advanced algorithms have been used to enhance velocity and gradient space, for example, GGVF

(general gradient vector ow) [7] and the double Snake [8].

Considering the challenges mentioned above in the eld of tongue body segmentation, existing works

usually focus on one of them in implementing their methods. But in practical application, success

remains a long way o . In this paper we propose a novel active contour model, C2 G2 FSnake (color

control-geometric & gradient ow Snake). An initial contour is rst obtained according to the speci c

shape of the tongue body and its location on the face, and then the whole image is divided into two parts

for separate evolution. The concrete evolution includes two steps: rough evolution and re ned evolution.

In rough evolution, a simpli ed geometric active contour model is adopted to drive the curve rapidly to

the neighborhood of the real boundary, and then a gradient vector ow model is used to force the curve

kiss-coating the real boundary. Color space information is introduced to control information correctly in

updating the parameters of the external force, which helps realize a seamless inter-grade between rough

and re ned evolution.

C2 G2 FSnake

2

The basic algorithm ow of C2 G2 FSnake is shown in Figure 1. First an initial contour combining tongue

features on the face is obtained and then the binary GVF eld for the upper part and color GVF for the

lower part are initialized for the geometric active contour model and the parameterized active contour

model respectively. Geometrical propagation is carried out rst to direct the curve evolution towards the

real boundary, with GVF applied afterwards to obtain the nal result.

Initialization of C2 G2 FSnake

2.1

The initial contour of C2 G2 FSnake is acquired by utilizing features of the tongue location on the face,

which is more robust than any other existing methods. C2 G2 FSnake separates the image region into two

propagation elds for the di erent modes. The lower half contour is a circumcircle that is constructed

based on the three special tongue points (two angular points around the tongue root and the tongue

tip). The upper half contour, however, is replaced by a straight line, to ensure that it remains within the

tongue body.

Shi M J, et al. Sci China Inf Sci 3

Flow chart of C2 G2 Fsnake.

Figure 1

mrow

Pt.3

Pt.2

Left Right Pt.4

0.7*row Pt.1

Pt.1

(a) (b) (c) (d) (e) (f)

Figure 2 (a) Original image; (b) edge detection image after Gaussian ltering, the red line is the local arc around tongue

tip that satis es the statistical character mentioned in subsection 2.1.1; (c) position of tongue tip (Pt.1), in red frame;

(d) binary image using Otsu s method; mrow is the minimum row of intensity value; (e) column number of tongue tip

(Pt.1); (f) searching the rst dark point (Pt.4) along the column of tip point (Pt.1) in the binary image.

2.1.1 Tongue tip point

Detection of the tongue tip is based on its statistical character as described below. First, apply the

CANNY edge detector to the image of Gaussian ltering with a small smoothing term (Figure 2(b)).

Then search the edge image for every edge point from bottom to top, tracking several points(For example

20 points on both the left and right sides for a 356 356 image resolution or adjusted comparatively, as

shown by the red line in Figure 2(b) . The radian character is determined as follows:

For each current point, its next proximate point is located either in the top left or top right.

Accumulate the points that satisfy the above metric, setting a threshold, e.g. 15, taking the current

point as the standard. Then search the left and right separately; if more than 15 points of the total 20

statistical points satisfy the metric, demarcate this standard point as the tip point (Pt.1) and exit the

loop (Figure 2(c)). Otherwise update the standard point.

2.1.2 Angular points

Harris angular detector [10] is used to obtain the angular points, considering that it is invariant to

rotation, scale, illumination and noise.

Regarding the tongue image, more constraints must be given to localize the accurate position of the

angular points. the detailed steps are: project the intensity image onto the y -axis; nd the darkest row

(mrow) of the whole image; and record row, col, respectively marking the row and column:

col

I (m, i) . (1)

mrow = min

m=1,2,3 row

i=1

Shi M J, et al. Sci China Inf Sci

4

Afterwards, Otsu s method is applied to the image to binarize the tongue image with two labels

depending on whether it is dark region between the root of tongue and the upper lip. Then, we add

a lower limit line (0.7*row, which is an alternative addition, and a rough estimation, since the angular

points always occur in the upper part of the image.) Utilizing Harris angular detector in the two graphical

regions (Left & Right) shown in Figure 2(d), two angular points are detected on the two sides (Pt.2, Pt.3)

with the following restrictions:

Pt left = max {ri + col ci }i=1,2 n, Pt right = max {ri + ci }i=1,2 n, (2)

where n is the number of angular points after running Harris s detector, and ri, ci represent the row and

column number of the ith point, respectively.

2.1.3 The fourth feature point

As shown in Figure 2(f), we search for the rst dark point along the column of the tip point (Pt.1) in

the binary image, and denote this point as Pt.4. This point is detected to keep the upper initial curve

within the tongue body, so that the upper lip cannot a ect the contour convergence. Since the slope of

the curve also increases in this way, the geometric movement of the GVFSnake contour is faster than

before.

2.1.4 Contour initialization

A circle is constructed from the above three points (Figure 3(a)). In practice, to make full use of the

traits of the tongue body, the fourth point needs to be included to reconstruct the initial contour of the

upper part as a straight line shown in Figure 3(b). Although the upper initial curve swells, the lower

part shrinks. There is another point that should be forwarded so that the model adopted in this paper

is highly adaptab to the initialization. The precision required in the initial contour is not rigid. For the

tip point, it only needs to lie underneath the tongue, while errors for the other three feature points can

be overcome afterwards through the model propagation.

2.1.5 Region division

To better satisfy the di erent evolution modes in subsections 2.2, 2.3 and 2.4, our algorithm divides the

whole region into two parts (Part I, Part II) corresponding to the row of angular points, with Part I

utilizing the binary image, and Part II the original image as shown in Figure 3(c).

2.2 GVFSnake (gradient vector ow snake)

We brie y introduce the original parametric active contour model and its GVF form in this subsection,

and then we present our modi cation.

2.2.1 Snake and GVFSnake

The parametric active contour model [11], also known as Snake, de nes a curve C (p, t), in which p

represents the curve parameter, and t is the timing term. General curve propagation is evolved as

follows:

C (p, t)

= (p, t)T (p, t) + (p, t)N (p, t) + Eext, (3)

t

where,, are parameters, T (p, t), N (p, t) represent the tangential force and normal force respectively,

Eext is the external force. Xu et al. [12] pointed out that the traditional parametric model depends

on the guidance of the gradient direction of the curve, which means that practical curve propagation is

sensitive to initialization, especially in the recess. It cannot converge but split as a result of the horizontal

gradient guidance. Thus, the real boundary can only be reached when the initial contour is close to it.

Subsequently, Chen proposed a Snake model based on the gradient vector ow (GVF). GVF is di erent

from the original binary gradient space and directly obtained from the continuous gradient space by solv-

Shi M J, et al. Sci China Inf Sci 5

Part I

Pt.4

Pt.3 Pt.3

Pt.2 Pt.2

Part II

Pt.1

Pt.1

(a) (b) (c)

Figure 3 (a) Circle constructed from three points (Pt.1, Pt.2, Pt.3); (b) initial active contour constructed from four

points (Pt.1, Pt.2, Pt.3, Pt.4); (c) initial color region for active contour model.

ing a set of vector equation of thermal expansion. In this way, the local gradient vector is constructed

not only by its own gradient, but also by the neighborhood. This greatly expands the gradient space, and

due to the combination of the gradient and Laplace operator, GVF is no longer strictly perpendicular to

the true edge, so the curve will evolve into the recess.

GVF is de ned as follow [12]:

E (V ) = g ( f ) 2 V h( f )(V f ), (4)

where f is a pre-de ned edge map,with the general form:

f (x, y ) = E i ext (x, y )i=1,2,3,4 . (5)

To guarantee that f points to the image edge, V is a two-dimensional vector of GVF: V (p) =

(u(p), v (p)), where p = (x, y ). g ( f ) 2 V is the smoothing term based on the following objective

function. When f is close to zero, the area is nearly a constant mainly dominated by the partial

derivatives of the vector eld. On the other hand, when the curve is close to the real edge, f becomes

large. The second item dominates the curve evolution, driving it to the real boundary continuously until

V = f . The weighting function g is chosen as g ( f ) =, a constant, indicating that the whole

image is being smoothed. h is chosen as h( f f 2, which is related to the strength of the edge.

The greater the strength, the larger is h [13].

2.2.2 Modi cation in this paper

Since f (x, y ) is the key factor of GVF, its accuracy strongly determines nal convergence. However, the

gray information I (x, y ) re ecting the gradient information for the color image is not complete. To fully

re ect it, we re-de ned Eq. (4) as

E (V ) = g ( X ) 2 V h( X )(V X ), X = max{R, G, B }, g ( X ) =, h( f ) = X X 2 . (6)

The revised equation always nd the largest gradient s R, G, B values, thereby compensating to a certain

degree, the shortage of single-channel gray information. In addition, multiplying X strengthens the

stronger edge of the tongue body, while weakening the disturbing edge in the face skin. As the upper

part of the tongue body is often subject to over-learning [5,11,12], we skillfully utilize the binary image

for the upper part and the color image for lower part (Figure 3(c)). This solves the problem completely,

since the binary image is a constant in the dark shadow region on both sides of the tongue root, thus

ensuing accurate convergence without over-learning.

Based on the above techniques, we initialize the GVF of the whole image space with VB, VX representing

the upper and lower parts respectively.

VB (p(x, y ), t)

= 2 VB B 2 (VB B ), when : y

t (7)

VX (p(x, y ), t)

= 2 VX X X 2 (VX X ), when : y > sline,

t

where B is the binary image, y is number of rows in the image, and sline is the dividing line between the

two parts.

Shi M J, et al. Sci China Inf Sci

6

2.3 Combining geometric active contour model with parameterized model

Only the GVFSnake parameter model cannot successfully direct the original curve convergence to the real

boundary. In fact, although GVF greatly enlarges the range of the gradient vector space, the direction

is still uncertain for the initial curve, especially in the upper part, as the majority of the GVF points are

zero after binarization and thus it takes too long. Even if it does eventually converge through its internal

force, we can still not ignore the sensitivity of parameters and to the curve. On the other hand,

in the process of evolution in Part II, it is hoped that the curve will converge along the nearly normal

direction of the GVF, which best ts the shape of the tongue, although it is possible that this will be

a ected by facial wrinkles or dark spots that may lead the curve in the wrong direction.

Referring to our initialization curve, the directions of the two parts are already con rmed: that is,

the upper half tends outward while the lower half tends inward. Due to this favorable priori condition,

referring to both the geometric [14] and balloon [15] models, we can directly introduce the geometric

vector ow into the parametric model, which is de ned as follows:

C (p, t) = g (C ) N, (8)

where N represents the normal direction of the curve, N is a curvature vector, is an arbitrary constant

factor and g is an edge detector.

We design an evolution model like this in the beginning, which roughly propagates relatively close to

the real boundary using the geometric model and then switches to the GVF model for re ned convergence

(the switch is automatically realized by the color control model mentioned below). Because of the color

region information control, when and where to switch the model does not need to be considered. As

long as the propagation of the curve is approximately close to the edge, it can achieve the nal result we

desire. In practice, updating the region control parameters can replace the time switching term. In most

cases, we set the switching parameter to a large constant. Since the geometric model need only rough

evolution, Eq. (8) is simpli ed according to the initial features as follows:

C (p, t) = g (C )N . (9)

In the upper half (Part I), suppose Na is an outward expansion, ga (C ) = 1+1 2, with VB representing

VB

the binary space of GVF. In the lower half, due to the initialization s arc characteristic, suppose Nb is the

inward radian vector pointing to the circle center where gb (C ) = 1+1 2 and V x is the GVF in the color

VX

space. Let Gthresha, Gthreshb be small gradient thresholds, such that when VB is greater than Gthreshb,

ga is set to zero. gb (C ) = 1+1 2 when it exceeds this threshold, else gb (C ) is zero and the curve totally

VX

relies on the regional information to converge. This method reduces the complexity, and is proven to be

a better model in practice than the original one since it takes advantage of the initialization curve.

2.4 Color region control

The active contour model, based on the edge gradient, displays superior performance most of the time.

However, the unlimited direction of the gradient makes the model quite sensitive to image noise. Moreover,

it also seems to be helpless with weaker edges. Therefore, more attention has recently been paid to the

active contour based on region information. A mature and widely used model was proposed by Chan and

Vese [16]. Since it is based on global region information, the model only works well when it is adapted

to single-channel images. As a result, further researches on region based model [17,18], which attempt to

combine region information with edge information, has been carried out, thereby expanding the practical

usage and obtaining better results. Generally speaking, region based models achieve sound results in

resolving weak edges and overcoming image noise, although in practice they are still somewhat lacking

compared with gradient active contour models.

After a number of experiments we found that region information is incomplete and inaccurate when

used directly as the curve s evolution force. Gradient information is needed to deal with the problem of

weak edges, whereas it is su cient to guarantee that the curve converges to the real local boundary when

Shi M J, et al. Sci China Inf Sci 7

the local color information is applied to the control information limiting the curve direction. At some

weak edges, the color information turns out to be distinguishable. Although this is not as accurate as

the gradient model, as a complementary method, it overcomes the drawbacks of the gradient models and

ensures that the curve propagate roughly towards the real boundary. Re ned evolution still depends on

the gradient. To the best of our knowledge, this is the rst time that color region information has been

applied as control information for the active contour model.

2.4.1 Sampling

After initialization, curve sampling is rstly carried out. If this were was done, the local region calculated

by every single point on the curve would overlap, making the information redundant. On the other hand,

if the sampling interval is too big, information may be inadequate for control. Let the length of the curve

be L, and the sampling interval be w, and then the total number of sampling points is: Num= L/w. In

the local circular region (radius r, where r generally equals w) of a certain point m, we have [17]

m n T r

C (p(x, y ), t)

= (19)

(p, t)T (p, t) + (p, t)N (p, t) + b1 1+1 2 Nb, t T r

where T (p, t) and N (p, t) are the tangential force and normal force respectively, VB and VX in Equation

(7). Na and Nb represent the outward normal direction of Part I and inward radian direction Part II,

respectively. T r is the time switching point between the geometric model and the parameterized model,

which occurs after a certain pre-de ned time or the deformation under the geometric model is below

a certain thresh value. sline is the separation row between Part I and Part II, and is obtained by

Equation (18). a represents the updating parameter of Part I, because of the obvious di erence between

the tongue body and pre-de ned dark region.The value of a for di erent images is a relative larger

constant value, and a1 = a2, On the other hand b represents the updating parameter of Part II,

and varies for di erent tongue colors due to the discrepancy between di erent tongue color and the face

color. To overcome certain problems, such as the over-learning caused by weaker tongue edges, we need

to set b1 and b2 slightly di erently, so as to adjust to the di erent tongue images. Moreover, since

colored region control information and the geometric model are used, the parameterized model is no

longer sensitive to the curve s inward force. Usually we set, to 0.5, and only adjust to control the

curve s accuracy and velocity.

C2 G2 FSnake (color controlled-geometric & gradient ow Snake) is implemented in MATLAB 7.9, with

the code available at http://levis.tongji.edu.cn/gzli/code/c2g2fsnake.zip.

3 Experimental setting

3.1 Measurements

3.1.1 Boundary error metrics

Let A = {a1, a2, . . ., am } denote the automated segmentation results, and B = {b1, b2, . . ., bn } denotes

the manual contour. Here we de ne two boundary error metrics, the Hausdor distance and the mean

of the distance to the closest point (DCP). The DCP for ai to B is de ned as

d(ai, B ) = min(bj ai ) . (20)

j

The Hausdor distance (HD) between two curves A and B is de ned as

hd(A, B ) = max(max{d(ai, B )}, max{d(bj, A (21)

i j

Shi M J, et al. Sci China Inf Sci 9

and the mean DCP distance (MD) of two curves A and B is de ned as

1

md(A, B ) = d(ai, B ) + d(bj, A) . (22)

m+n i j

The normalized HD (norm.HD%) and MD (norm.MD%)are computed by dividing HD and MD, re-

spectively, by the number of pixels of the manual contours.

3.1.2 Area error metrics

the three area error metrics, false positive volume fraction (FP), false negative volume fraction (FN), and

true positive volume fraction (TP), are de ned as follows

FP = Ab Aa Ab Aa , FN = Aa Ab Ab Aa , TP = Aa Ab Aa , (23)

where Aa, Ab are the areas of the tongue body determined by manual delineations and the proposed

method, respectively.

3.2 Experimental images

A clinical tongue image database is available from the Shanghai University of Traditional Chinese

Medicine, categorized by tongue color: light white tongue (tinge, light white), red tongue (red, deep

red, deep purple), purple tongue (heliotrope, pompadour and heliotrope, purple, modena) and Carmoi-

sine Tongue. In total, there are four classes and ten branches with each branch selecting 3 images

and 30 images in total. The original images and manually marked standard images are available at

http://levis.tongji.edu.cn/gzli/data/tonguepic.zip.

3.3 Parameter settings

This paper compares C2 G2 FSnake with two state-of-the-arts works [6,8] using the clinical tongue image

database mentioned in subsection 3.2. C2 G2 FSnake does not require initial settings and thus initial

settings of works [6, 8] refer to the two systems used for comparison. However, on some test images the

initialization was not close enough to the real boundary, and manual initialization was adopted so as to

obtain a better result. The internal force was set as the default setting of the active contour, while the

external force was set according to the di erent test images.

4 Results and analysis

4.1 Experimental results

Results of the comparative tests on the tongue image database (referred to in subsection 3.2) using

C2 G2 FSnake and two state-of-the-art algorithms [6, 8] are shown in Figure 4: C2 G2 FSnake shows better

performance for every metric. Detailed test results for C2 G2 FSnake are shown in Table 1 for ten branches

of tongue color.

Zhai et al. [6] adopted the H channel as the initial contour for outside the Snake curve, which is close

to the real boundary in general. The evolution form is de ned as

Cinside (p, t)

= (p, t)T (p, t) + (p, t)N (p, t) + G (x, y ) I (x, y )

t

+ (Cinside (p, t) Coutside (p, t))2, (24)

where Cinside (p, t), Coutside (p, t) represent inside and outside the curve respectively. There exists a force

driving the inside curve onto the real boundary as long as it does not meet the outside curve. Since the

outside curve is close to the real boundary, it only needs to evolve according to the gradient(see equation

Shi M J, et al. Sci China Inf Sci

10

6 98.0

(a) 97.5 (b)

5

Percentage

97.0

4

Percentage

96.5

3

96.0

2 95.5

1 95.0

0 94.5

Norm.HD Norm.MD FN FP 94.0

Fu et al. 0.5 2.13 3.66 5.09

93.5

Zhai et al. 0.74 1.74 4.33 4.66 C2G2FSnake

Fu et al. Zhai et al.

C2G2FSnake 0.37 1.08 1.79 2.28 TP 94.91 95.34 97.6

Test results of C2 G2 FSnake using two of the latest methods. (a) The smaller the values of Norm.HD%,

Figure 4

Norm.MD%, FN%, and FP%, the better are the results; (b) the larger the TP%, the better are the results.

Detailed results of C2 G2 FSnake

Table 1

Tongue color Norm.HD Norm.MD FN FP TP Tinge 0.47 0.71 1.3 1.1 98.92

Light white 0.12 0.60 0.84 0.36 99.30

Red 0.18 0.63 1.59 0.35 99.65

Deep red 0.43 1.09 1.86 2.97 97.03

Deep purple 0.48 1.17 1.86 3.53 96.47

Heliotrope 1.11 2.19 1.71 4.6 95.40

Pompadour and heliotrope 0.27 1.1 1.69 2.1 97.89

Purple 0.39 0.85 2.28 1.06 98.93

Modena 0.42 1.39 1.83 3.54 96.46

Carmoisine 0.34 1.06 2.9 3.19 96.80

Average 0.37 1.08 1.79 2.28 97.60

(3)). Actually its main role is to act as a barrier ensuring that the inside curve can get close to the

real boundary without over learning. When it is close enough, the local gradient guides the inside curve

to the nal result. Whether the outside curve converges to the nal result is not important. Fu et

al. [8] introduced AdaBoost to locate the tongue body in the human face, and then to eliminate the

H-channel tongue color, which is the same as the lip and skin color. The initial contour is obtained

by combining polar edge detection [9] that is close to the real boundary, and thus, Snake curve nally

evolves. Based on prior knowledge and further testing afterwards, initialization of the colorspace or polar

edge is sensitive to the image color. Moreover, the performance of the active contour model is highly

depended on whether the initial contour is close enough to the real boundary [8] (this manly depends on

the outside curve initialization), it has been suggested in subsection 3.3 that some images may have to be

initialized manully to overcome the problems caused by their automated initial contour being far from the

real boundary. Even in this case, C2 G2 FSnake still shows a distinct superiority compared with the other

two methods. The reason is that neither of the other two methods can deal with the special problems

of tongue segmentation, such as over-learning at the root of the tongue and local minima caused by a

stronger skin gradient and weak tongue edge gradient, and thus even if the initial contour is close to the

tongue body edge it cannot stop the curve from over-learning. Besides neither the GVF nor the GGVF

(general gradient vector ow) [7] Snakes, which enlarges the gradient space the most, shows satisfactory

results.

4.2 Analysis of initialization settings

This section discusses the results of the curve initialization and evolution. Tests were conducted on

three tongue images with di erent characteristics, namely, indentation, exfoliative fur, and red tongue

as shown in Figures 5(a) (c). Figures 5(d) (f) show the initializations proposed in this paper, which

has stronger applicability and requires no parameter settings, compared with some other initialization

methods (colorspace [8],watershed [5]). The tests were successful on all clinical tongue images from the

Shi M J, et al. Sci China Inf Sci 11

(a) (b) (c)

(d) (e) (f)

(g) (h) (i)

(j) (k) (l)

(a) (c) Original images; (d) (f) results of the initial method in C2 G2 FSnake; (g) (i) C2 G2 FSnake evolution

Figure 5

results; (g) bpre = 0.005, baft = 0.005; (h) bpre = 0.01, baft = 0.01; (i) bpre = 0.01, baft = 0.01; (j) (l) nal

segmentation results of (g) (i).

database of the Shanghai University of Traditional Chinese Medicine. Two initial contours are given in

the lower part Part II of which the green one is the original, while the white one is obtained by moving the

tongue tip down a few pixels. This modi cation is done to provide enough space for the geometric model

propagation; on the other hand, it guarantees a discrepancy between the initial sampling and the evolving

sampling for updating the color control parameter. Figures 5(g) (i) show the results of C2 G2 FSnake, and

the nal tongue body segmentation (j) (l). Since it is feasible for a single image to sample every point

of the contour, and after introducing color control information, active contour is no longer sensitive to

internal parameters, we can obtain ideal results simply by adjusting the control parameter. Others are

set to: = 0.5, = 0.5, = 2.0, iter=100, conv=30, a = 0.5, where, represent internal parameters,

is an external parameter, iter is the total number of iterations, and conv is the number of iterations in

the geometric evolution. If there is no special declaration, these are the default settings. a signi es the

control parameter for Part I, bpre, baft represent concrete values of for the geometric and gradient

model of Part II. From Figure 5, we can see that our method is obviously robust and accurate, and for

detailed observation, C2 G2 FSnake can fully converge to the real boundary, whereas GVF and GGVF

leave some small intervals in most cases.

Analysis of parameters of C2 G2 FSnake

4.3

In the following, we discuss and analyze key parameters of C2 G2 FSnake, namely, sampling interval,

Shi M J, et al. Sci China Inf Sci

12

(c)

(a) (b)

(d) (e) (f)

(g) (h) (i)

(a) (c) Di erent sampling interval results, with the same bpre =0.01, baft =0.01; (d) (f) di erent baft values

Figure 6

with the same bpre =0.003, (d) baft =0; (e) baft =1; (f) baft =0.003; (g) (i) di erent internal parameter sets with the

same = 2.0, iter = 100, conv = 30, a = 0.5, bpre = 0.05, baft = 0.05; (g) = 0.5, = 0.5; (h) = 1.0, = 0.1; (i)

= 0.1, = 1.0.

control parameters, and internal parameters.

4.3.1 Sampling interval

In general, the size of a single tongue image is small (356 356 pixels), so the sampling interval is every

point. Of course for large amounts of data, according to control theory, precision decreases as the sampling

interval increases. Figures 6(a) (c) show the results of full sampling, sampling at every two points and

at every three points, respectively. In Table 2 Nos.a, b, c give the error metrics corresponding to Figures

6(a) (c). This no longer su cient information to control the optimization as the interval increases,

thus the results vary with di erent sampling intervals for the same control parameters bpre = 0.01,

baft = 0.01. To obtain the same results we have to modify the control parameter to be more accurate.

Besides, it also shows that simply using gradient information is not enough to guarantee the seamless

switch from the geometric model to the gradient model.

4.3.2 Control parameters

When the switch to the parameterized model from the geometric model has taken place, control infor-

mation needs to be applied all the same to obtain more acceptable results. Table 2 Nos.d, e, f show the

results by adopting a local region control model, with d performing the best. Figures 6(d) (f) show this

problem: once the curve is close to the real boundary after geometric propagation, over-learning still

occurs along the tongue body (e), as a result of strong gradient guidance by the neighbor of the tongue

Shi M J, et al. Sci China Inf Sci 13

Statistical results of metrics using C2 G2 FSnake on Figures 6(a) (i)

Table 2

No. Norm.HD Norm.MD FN FP TP a 0 0 0 0 100

b 3.07 3.02 7.49 0.61 99.39

c 3.07 3.20 7.96 0.59 99.41

d 0.43 0.70 3.42 2.42 97.58

e 0.29 1.03 1.97 6.59 93.41

f 0.43 0.69 2.98 4.65 95.36

g 0.11 0.66 3.31 0.92 99.08

h 0.11 0.66 3.34 0.95 99.05

i 0.11 0.66 3.31 0.96 99.04

body. Therefore, control information is applied to re ne the evolution. In most cases we set bpre = baft,

but in certain instances, as shown in Figure 6, they are no longer the same; baft needs to be small enough

to ensure the curve stops at the real boundary without over learning.

4.3.3 Non-sensitivity of internal parameters

2 2

C G FSnake rst adopts the geometric model for rough evolution, driving the curve close to the tongue

body, and then utilizing the parameterized model for re ned evolution. One of the obvious merits of the

geometric model is its non-sensitivity to parameters, meanwhile as control information is added to the

parameterized model, it is also no longer sensitive to the internal parameter, and the nal result can be

obtained only by adjusting the control parameter, see Figures 6(g) (i). According to Table 2 Nos.g, h, i,

the results of di erent internal parameters show almost the same performance. Commonly, can be

constants which do not a ect the shape of the curve. Although control information has been added to

C2 G2 FSnake, the time the whole propagation spends on the geometric model is still shorter than that of

the parameterized model, while the average velocity is almost equal to the GVF model and sometimes

even greater than it.

5 Conclusion

In this paper we proposed a new algorithm, C2 G2 FSnake, for tongue body segmentation in tongue

diagnoses for TCM. We also presented detailed experimental tests and comparisons with the latest works

showing that C2 G2 FSnake realizes accurate automated tongue segmentation. Five techniques have been

integrated in this algorithm. 1) A fully automated method for the initial contour is given according

to the speci c shape of the tongue and its location on face, which highly improves practicability. 2)

Color space information is introduced in a creative way as control information to update the parameter

of the external force, thus controlling the precision and velocity of the curve. This not only succeeds

in C2 G2 FSnake, but also maintains a signi cant direction for general image segmentation. 3) Based on

the characteristics of the curve propagation, the geometric model is combined with the parameterized

model as rough evolution and re ned evolution, respectively. Meanwhile, vector ow is initialized as two

parts according to the dark shadow region at the root of tongue which compensate each other maximally.

4) As a result of the introduction of control theory, the geometric model is switched to a parameterized

model seamlessly, and is no longer sensitive to the internal parameters. 5) Di erent control parameter

settings allow the curve to stop correctly at the weak gradient edge without over-learning.

Our future work will focus on how to extend color region control information to general image segmen-

tation as an external force.

Acknowledgements

The work was supported by the National Science Foundation of China (Grant Nos. 60873129, 61005006). The

clinical tongue images were provided by the Laboratory of Traditional Medical Syndromes, Shanghai University of

Shi M J, et al. Sci China Inf Sci

14

Traditional Chinese Medicine. Thanks also go to the Editor and anonymous reviewers whose valuable comments

have contributed to making the paper more clearer and more practical.

References

1 Li G Z, Shi M J, Li F F, et al. An empirical study on tongue image detection. J Shangdong Univ(Eng Sci)(in Chinese),

2010, 40: 87 95

2 Kim K H, Do J H, Ryu H, et al. Tongue diagnosis method for extraction of e ective region and classi cation of tongue

coating. In: Proceedings of the 1st workshop on Image Processing Theory, Tools & Applications. Sousse: IEEE Press,

2008. 1 7

3 Yang B S, Wei Y K, Li J P. Research and application of image segmentation algorithm based on the shortest path

in medical tongue processing. In: World Congress on Software Engineering. Xiamen: Computer Society Press, 2009.

239 243

4 Liu Z, Yan Y Q, Zhang D, et al. Automated tongue segmentation in hyperspectral images for medicine. Virt J Biomed

Opt, 2007, (46): 8328 8334

5 Jia W, Zhang Y H, Bai J. Tongue area extraction in tongue diagnosis of Traditional Chinese Medicine. In: Proceedings

of the 27th Annual International Conference on Engineering in Medicine and Biology Society. New York: IEEE Press,

2006. 4955 4957

6 Fu Z C, Li W, Li X Q, et al. Automatic tongue location and segmentation. In: Proceeding of the IEEE International

Conference on Audio, Language and Image Processing. Shanghai: IEEE Press, 2008. 1050 1055

7 Fu H G, Wu R Q, Wang W M. Active contour model based on dynamic extern force and gradient vector ow. In:

Proceedings of International Conference on Biomedical Engineering and Informatics. Sanya: IEEE Press, 2008. 863 867

8 Zhai X M, Lu H D, Zhang L Z. Application of image segmentation technique in tongue diagnosis. In: Proceedings of

the International Forum on Information Technology and Applications. Chengdu: IEEE Press, 2009. 768 771

9 Zuo W M, Wang K Q, Zhang D, et al. Combination of polar edge detection and active contour model for automated

tongue segmentation. In: 3rd International Conference on Image and Graphics (ICIG 04). Hongkong, 2004. 270 273

10 Brown B M, Richard S, Simon W. Multi-image matching using multi-scale oriented paths. In: Proceeding of the

International Conference on Computer Vision and Pattern Recognition. Washington DC: IEEE Press, 2005. 510 517

11 Srikrishnan V, Subhasis, Chaudhuri. Stabilization of parametric active contours using a tangential redistribution term.

IEEE Trans Image Process, 2009, 18: 1859 1872

12 Xu C Y, Prince J L. Generalized gradient vector ow external forces for active contours. Signal Process, 1998, (71):

131 139

13 Xu C Y, Prince J L. Gradient vector ow: a new external force for snakes. In: IEEE Computer Society Conference on

Computer Vision and Pattern Recognition (CVPR 97). San Juan: IEEE Computer Society Press, 1997. 337 348

14 Kichenassamy S, Kumar A, Olver P, et al. Gradient ows and geometric active contour models. In: Proceedings of the

5th International Conference on Computer Vision. Washington DC: IEEE Press, 2005. 810 815

15 Laurent D, Cohen, Cohen I. Finite-element methods for active contour models and balloons for 2-D and 3-D images.

IEEE Trans Pattern Anal, 1993, 15: 1131 1147

16 Chan T F, Vese L A. Active contours without edges. IEEE Trans Image Process, 2001, 10: 266 277

17 Darolti C, Mertins A, Bodensteiner C, et al. Local region descriptors for active contours evolution. IEEE Trans Image

Process, 2008, 17: 2275 2288

18 Krinidis S, Chatzis V. Fuzzy energy-based active contours. IEEE Trans Image Process, 2009, 18: 2747 2755

19 Goldenberg R, Kimmel R, Rivlin E, et al. Fast geodesic active contours. IEEE Trans Image Process, 2001, 10: 1467 1475

Contact this candidate