Документ взят из кэша поисковой машины. Адрес оригинального документа : http://acat02.sinp.msu.ru/presentations/shibzoukhov/acat2002_spnn.pdf
Дата изменения: Tue Jul 16 12:22:19 2002
Дата индексирования: Mon Oct 1 20:33:40 2012
Кодировка:

VIII International Workshop on Advanced Computing and Analysis Techniques in Physics Research ACAT '2002, June, 2428, 2002, Moscow

CONSTRUCTIVE METHOD FOR SUPERVISED LEARNING WITH COMPLEXITY MINIMIZATION OF ARTIFICIAL NEURAL NETWORKS OF ALGEBRAIC -NEURONS Shibzoukhov Z.M. szport@fromru.com RESEARCH INSTITUTE OF APPLIED MATHEMATICS AND AUTOMATION, KBSC RAS, NALCHIK

1

ALGEBRAIC -NEURON xn x1
.....

i1

..... . . .

w1

............

i

N

.....

wN
out

y
Let K be a ring without of denominators of zero. An algebraic -neuron implement composition of a polylinear function sp(x1, . . . , xn) and a scalar function out(s): spn(x1, . . . , xn) = out sp(x1, . . . , xn) ,
N

sp(x1, . . . , xn) = +
k =1

wk
iik

xi ,

and x1, . . . , x output, coefficient, ik

n are inputs from K, y K be an

K be a bias, wk K be a weight {1, ..., n} be a multiindex (i = ).
2

ZERO-ORDERED SEQUENCES OF SPARSE VECTORS Let X = {xk } be a sequence of sparse vectors from Kn, I = {ik } be a sequence of multiindices. Definition. X is zero-ordered with respect to I if for any pair of j < k there exists t = t(j, k ) ik such that xj t = 0 and xkt = 0. We say X is zero-ordered if ik = {1, . . . , n} for all k . Let for any x K 1, if x = 0, x= 0, if x = 0 and x = (x1, . . . , xn) for any x Kn. Let X = {xk }. Lemma. Any sequence X Kn can be zero-ordered iff X not contain identical elements. For any zero-ordered sequence X we denote by i(xk ) a multiindex {i : xki = 0}. Lemma. For any zero-ordered sequence X there exists a sequence I (ik i(xk ) for all k ) such that X is zeroordered with respect to I.

3

TRIANGULAR SEQUENCES OF PRODUCTS Let's denote by p(x; i) a following product p(x; i) =
ii

xi .

For any I let's define a sequence of products P(I) = {p(x; ik )}. Definition. P(I) is triangular on X if the following conditions are true: 1) for any pair of j < k the equality p(xj ; ik ) = 0 is true; 2) for all k the inequality p(xk ; ik ) = 0 is true too. We further suppose that ik i(xk ) for all k . Lemma. P(I) is triangular on X iff X is zero-ordered with respect to I. Let X be zero-ordered with respect to I. We say that that multiindex ik is minimal with respect to zero-ordering of X if exclusion of any i ik make broken this property of X. Also we say that a product p(x; ik ) is minimal with respect of triangularity P(I) on X if exclusion of any factor xi from p(x; ik ) make broken triangularity of P(I). Definition. We say that sequence I is minimal with respect to zero-ordering of X if any multiindex ik I
4

is minimal with respect to zero-ordering. We also say that P(I) is minimal with respect to triangularity on X if any product p(x; ik ) is minimal. So we obtain a statement. Lemma. P(I) is minimal with respect to triangularity on X iff I is minimal with respect to zero-ordering of X.

5

RECURRENT SEQUENCES OF -NEURONS Let SP = {spk (x)} be a sequence of forms. Definition. SP be a recurrent sequence of forms if for any k spk (x) = spk-1(x) + wk p(x; ik ), where wk K, ik be a mutiindex, sp0(x) be any form. Let's suppose that for any s K, 0 = p K, y D(out) there exists a solution w(s, p, y ) of equation out(s + pw) = y . Let 0, if out(sk ) = yk wk = w(sk , pk , yk ), if out(sk ) = yk , where sk = spk-1(xk ), pk = p(xk ; ik ). Let X be zero-ordered with respect to I. For any k an expression X[a, b] denotes a subsequence {xj : j = a, ..., b}. Under these suppositions the following statement is true. Proposition. For any k -th element of recurrent sequence SP the equivalence y (x) out(spk (x)) is true on X[1, k ].
6

CONSTRUCTIVE LEARNING OF SEQUENTIAL CHAIN OF LINKED -NEURONS xn x1
.....

1 2

.....

u1 u2

.....

...................................

L

.....

uL
x1
.....

x1
.....

xn NNp-1 u1 u
p-1

xn NNp-1 u1 NN
p

up-1
.....

up

Let's consider a following sequential chain SPN of linked -neurons: u1 = spn1(x), up = spnp(x, up-1),
7

where p = 2, . . . , L, L is length of the chain. For any p let's define a trasformation function SPNp(x): SPN1(x) = spn1(x), SPNp(x) = spnp(x; SPNp-1(x)). Let sp0(u) be a -form such that out(sp0(u)) u on D(out). Proposition. For any sequence 0 < N1 < · · · < NL = N there exists a chain SPN such that y (x) SPNp(x) on X[1, Np]. We learn -neurons spn1, . . . , spnL sequentially; p-th -neuron spnp learns using construction of the recurrent sequences of -forms spp0, spp1, . . ., where for any p > 1 we put spp,0(x; up-1) = sp0(up-1).

8

CONSTRUCTIVE LEARNING MULTY-LAYERED NETWORK WITH FIXED WIDTH OF -NEURONS Let's denote by spn(x) a tuple of -neurons: spn(x) = (spn1(x), . . . , spnm(x)). Let's consider a following multy-layered network SPNN of -neurons: u1 = spn1(x), up = spnp(x, up-1), where p = 2, . . . , L, L is length of the chain, up = (up1, . . . , upm). For any p let's define a trasformation function SPNNp(x):

SPNN1(x) = spn1(x), SPNNp(x) = spnp(x; SPNNp-1(x)). Let sp0(u) be a -form such that out(sp0(u)) u on D(out). Proposition. For any sequence 0 < N1 < · · · < NL = N there exists a multy-layered network SPNN such that y (x) SPNNp(x) on X[1, Np]. We learn layers sequentialy, -neurons spn1, . . . , sp sequentially; all -neurons of p-th tuple spnp learns using construction of the recurrent sequences
9

of -forms. For any p > 1 we for j -th -neuron spnpj we put sppj,0(x; up-1) = sp0j (up-1,j ).

10

CONSTRUCTIVE LEARNING OF SEQUENTIAL CHAIN OF LINKED -NEURONS WITH A FIXED DEPTH
x1
.....

xn

1 2

.....

y1 y2 yl

.....

................................... -1

l

.....

.....

yl yp-l+1

...................................

p-l+2

.....

yp-l+2 yp-1 yp

....................................................

p

.....

.......

......................................................................

yL-1
L ..... .......

y

L

11

Let's consider a following sequential chain SPN of linked -neurons: u1 = spn1(x), uq = spn1(x; u1, . . . , uq -1), up = spnp(x, up- , . . . , up-1), where q = 2, . . . , , p = + 1, . . . , L, L is length of the chain. For any p let's define a trasformation function SPNp(x): SPN1(x) = spn1(x), SPNq (x) = spnq (x; SPN1(x), . . . , SPNq -1(x)), SPNp(x) = spnp(x; SPNp- (x), . . . , SPNp-1(x)). Let sp0(u) be a -form such that out(sp0(u)) u on D(out). Proposition. For any sequence 0 < N1 < · · · < NL = N there exists a chain SPN such that y (x) SPNp(x) on X[1, Np]. We learn any -neuron spnp using construction of the recurrent sequences of -forms spp0, spp1, . . ., where for any p > 1 we put spp,0(x; up-1) = sp0(up-1).

12