Improvement of Accuracy for Gaussian

Curvature Using Modi cation Neural Network

Yuji Iwahori1, Takashi Nakagawa1, Shinji Fukui2, Haruki Kawanaka3,

Robert J. Woodham4, and Yoshinori Adachi1

1

Faculty of Engineering, Chubu University

Matsumoto-cho 1200, Kasugai 487-8501, Japan

abpxtk@r.postjobfree.com, abpxtk@r.postjobfree.com,

abpxtk@r.postjobfree.com

http://www.cvl.cs.chubu.ac.jp/

2

Faculty of Education, Aichi University of Education

Hirosawa, Igaya-cho, Kariya 448-8542, Japan

abpxtk@r.postjobfree.com

3

Faculty of Information Science and Technology, Aichi Prefectural University

Nagakute-cho, Aichi-gun 480-1198, Japan

abpxtk@r.postjobfree.com

4

Department of Computer Science, University of British Columbia

Vancouver, B.C. Canada V6T 1Z4

abpxtk@r.postjobfree.com

Abstract. This paper proposes a new approach to recover the relative

magnitude of Gaussian curvature of the test object from four shading

images using modi ed neural network. The method is expanded to an

object with color texture using four shading images taken under the

di erent light source directions. Neural network mapps four image ir-

radiances on the test object onto a point on a sphere. The area value

surrounded by four mapped points onto a sphere gives an approximate

value of Gaussian curvature. To get more accurate Gaussian curvature,

the modi cation neural network is introduced and learned for the syn-

thesized 2-D basis function consisting of 2-D cosine function. It is shown

that learnt NN gives better accuracy for the relative magnitude of Gaus-

sian curvature of the test object.

Keywords: Neural Network, Gaussian Curvature, Shape Recovery,

Photometric Stereo.

1 Introduction

Surface gradient and curvature are the essential information for the shape repre-

sentation. Especially, the surface curvature is the invariant and e ective feature

for the viewing direction, and curvature feature can be used to many applications

such as the shape recovery, shape modeling, segmentation, the object recognition

and pose determination in the eld of computer vision.

Based on the physics-based vision approach, Woodham [1] developed a method

to get surface curvature using the values of the surface gradients. Using the LUT

B. Apolloni et al. (Eds.): KES 2007/ WIRN 2007, Part II, LNAI 4693, pp. 1013 1020, 2007.

c Springer-Verlag Berlin Heidelberg 2007

1014 Y. Iwahori et al.

(Look Up Table), the method obtains the local surface gradients by the empirical

photometric stereo using a calibration sphere.

Iwahori has pursued neural network implementations of photometric stereo.

In [2] [3], neural network implementation with the PCA (principal component

analysis) was proposed.

Angelopoulou and Wol [4], Okatani and Deguchi [5] proposed a method to

recover the local sign of the Gaussian curvature from three images taken under

di erent light source directions without using the values of the surface gradient

or the correct light source directions. These methods are applicable to the di use

re ectance.

While Iwahori et al. [6] proposed the method to classify local surface from

multiple shading images including the sign of Gaussian curvature using neural

network. As with both previous non-parametric, empirical implementations [1,2]

[3], no explicit assumptions need to be made either about light source directions

or about the functional model of surface re ectance.

Iwahori et al. [7] further extended the approach not only to recover the sign

but also the relative magnitude of local Gaussian curvature of the object. The

key idea is that the area value surrounded by the four mapped points onto a

sphere corresponds to the relative magnitude of Gaussian curvature.

This area value is an approximate value of the relative magnitude. This paper

extends the approach with higher con dence of relative magnitude of Gaussian

curvature. A new idea is developed by introducing the modi cation neural net-

work learned for the basis function with di erent curvatures.

The modi cation neural network improves the accuracy of local Gaussian

curvature for a test object. The advantage of the method is Gaussian curvature

can be directly extracted from shading images and the proposed approach can

be applied for non-Lambertian case with di erent albedo. The real phenomena

of the surface re ectance is directly treated and the e ect is evaluated through

experiments.

2 Principle

2.1 Empirical Constraint

The original purpose of photometric stereo uses three light source to deter-

mine the surface normal vector (n1, n2, n3 ) from the observed image irradiances

(E1, E2, E3 ) locally. While, this approach tries to get the local curvature infor-

mation directly from (E1, E2, E3, E4 ) at the local four points on the test object.

Let the image irradiances at (xobj, yobj ) on the test object be (E1obj, E2obj,

E3obj, E4obj ) and that at (xsph, ysph ) on the sphere be (E1sph, E2sph, E3sph, E4sph ).

If the surface material is the same for both of a test object and a sphere, the fol-

lowing constraint

E1obj (xobj, yobj ) = E1sph (xsph, ysph )

E2obj (xobj, yobj ) = E2sph (xsph, ysph )

(1)

E3obj (xobj, yobj ) = E3sph (xsph, ysph )

E4obj (xobj, yobj ) = E4sph (xsph, ysph )

Improvement of Accuracy for Gaussian Curvature 1015

is satis ed. This constraint for E1 to E4 leads to the condition that the corre-

sponding surface normal vector should be the same between a test object and

a sphere. This constraint is used to determine the relative magnitude of a test

object using neural network.

2.2 RBF Networks and OLS Learning

Neural networks are attractive for non-parametric functional approximation. As

an algorithm to learn the mapping of the nonlinear functions using neural net-

work, back propagation algorithm is popular and it can often used for this kind

of problem. As another candidate and choice except the back propagation, a

radial basis function (RBF) neural network [8] is one choice suitable for many

applications. In particular, it has been widely used for strict interpolation in

multidimensional spaces. In the case that we have many sampled points, RBF

neural network has an advantage that the learning is much faster and e ciently

done than the standard feed forward neural network with the back propagation

algorithm. This is based on the fact that he weights of network can be par-

tially determined for each of outputs and nonlinear mapping problem can be

e ectively solved using the orthogonal least squared algorithm [8] proposed by

Chen et al.

3 Improvement of Accuracy for Relative Magnitude of

Gaussian Curvature

3.1 Case of Non-uniform Albedo

Let j be the j -th light source, the image irradiance Ej is observed under the

illumination from each light source j . For four light source photometric stereo

under the orthographic projection, the following image irradiance equations hold.

E1 (x, y ) = (x, y )R1 (nx, ny, nz )

E2 (x, y ) = (x, y )R2 (nx, ny, nz )

(2)

E3 (x, y ) = (x, y )R3 (nx, ny, nz )

E4 (x, y ) = (x, y )R4 (nx, ny, nz )

where R is the re ectance map (image irradiance map) at the surface normal

vector (nx, ny, nz ). represents the albedo (i.e., re ectance factor) at the point

(x,y). Under this condition,

E1 + E2 + E3 + E4 = (R1 + R2 + R3 + R4 ) (3)

holds. The e ect of is eliminated as follows.

Ej

ej = (f or j = 1, 2, 3, 4) (4)

E1 + E2 + E3 + E4

The objective now is to determine the mapping of (e1, e2, e3, e4 ) to the corre-

sponding coordinate (x, y ) on a sphere.

1016 Y. Iwahori et al.

3.2 Mapping onto Sphere by Neural Network

A RBF neural network (NN) is used to do the mapping of the observed image

irradiances (e1 (xsph, ysph ), e2 (xsph, ysph ), e3 (xsph, ysph ), e4 (xsph, ysph )) to the

corresponding coordinate (xsph, ysph ) on a sphere.

With this learning procedure, NN is trained using input/output data from

a sphere. Many training vectors are available since data from the calibration

sphere are dense and include all possible visible surface normal.

The learning procedure builds an RBF neural network one neuron at a time.

Neurons are added to the network until the sum-squared error falls beneath

an error goal or a maximum number of neurons has been used. The resulting

network generalizes in that it predicts a position (xsph, ysph ) on the sphere, given

any input values, [e1, e2, e3, e4 ] of the test object.

3.3 Relative Magnitude of Gaussian Curvature

The kind of local surface curvature can be classi ed from multiple shading im-

ages [6]. However, even if the local surface is classi ed into the same class, the

magnitude of Gaussian curvature is the additional information.

The magnitude of Gaussian curvature consists of the absolute and the relative

one. Absolute value of Gaussian curvature is very small value in general, while

the relative magnitude can be used to recognize the feature points. The principle

to recover the relative magnitude is as follows. The coordinates (x, y ) of four

mapped points onto a sphere are used to recover the relative value of Gaussian

curvature. Calculating the area value that is surrounded by four mapped points

onto a sphere results in obtaining the relative magnitude of Gaussian curvature.

This area value corresponds to the relative magnitude of Gaussian curvature.

To recover the relative magnitude of Gaussian curvature, [7] shows the results

for test objects with uniform albedo. The key idea of [7] is that the area value

surrounded by the four mapped points onto a sphere corresponds to the relative

magnitude of Gaussian curvature.

Let z = f (x, y ) be the height distribution of the object, then Gaussian curva-

ture is de ned as

1 fxx fxy

G= (5)

2 + f 2 fyx fyy

1 + fx y

Here, fx = z/ x and fy = z/ y . For the smooth surface, the relation of

fxy = fyx holds. Suppose the four mapped points are located as shown in Fig.1-

(a), the surrounded area value corresponds to the relative magnitude of Gaus-

sian curvature. Consider the area value S calculated from the square with four

mapped points as shown in Fig.1-(b), then the square area S is given as

1 1 1

b sin = b = (4 x y )(fxy fxx fyy )

2

S= a a (6)

2 2 2

This area value corresponds to the relative magnitude of Gaussian curvature,

that is, the approximation of the relative value is obtained according to Eq.(6).

Improvement of Accuracy for Gaussian Curvature 1017

(a) Area Value (b) Calculation

Fig. 1. Area Value and Calculation

3.4 Modi cation Neural Network

Here, we introduce a modi cation neural network to improve the accuracy for

the relative magnitude of Gaussian curvature. The area value calculated from

four points is the approximate value of the magnitude. To improve the accuracy,

we introduce the mapping of area value S to the true value of Gaussian curvature

using the synthesized images of the following basis function f (x, y ).

f (x, y ) = (cos x cos y ) (

Contact this candidate