Complexity of neural networks on Fibonacci-Cayley tree

This paper investigates the coloring problem on Fibonacci-Cayley tree, which is a Cayley graph whose vertex set is the Fibonacci sequence. More precisely, we elucidate the complexity of shifts of finite type defined on Fibonacci-Cayley tree via an invariant called entropy. We demonstrate that computing the entropy of a Fibonacci tree-shift of finite type is equivalent to studying a nonlinear recursive system and reveal an algorithm for the computation. What is more, the entropy of a Fibonacci tree-shift of finite type is the logarithm of the spectral radius of its corresponding matrix. We apply the result to neural networks defined on Fibonacci-Cayley tree, which reflect those neural systems with neuronal dysfunction. Aside from demonstrating a surprising phenomenon that there are only two possibilities of entropy for neural networks on Fibonacci-Cayley tree, we address the formula of the boundary in the parameter space. 2010 MSC: 37A35, 37B10, 92B20


Introduction
For the past few decades, neural networks have been developed to mimic brain behavior so that they are capable of exhibiting complex and various phenomena; they are widely applied in many disciplines such as signal propagation between neurons, deep learning, image processing, pattern recognition, and information technology [1].
While the overwhelming majority of neural network models adopt n-dimensional lattice as network topology, Gollo et al. [15][16][17] propose a neural network with tree structure and show that the dynamic range attains large values.(See [2,18,19] for more references.)Recent literature also provides shreds of evidence that excitable media with a tree structure performed better than other network topologies [15,16].Mathematically speaking, a tree is a group/semigroup with finitely free generators, and group is one of the most ubiquitous structures found both in mathematics and in nature.A Cayley graph is a graph which represents the structure of a group via the notions of generators of the group.Remarkably, Pomi proposes a neural representation of mathematical group structures which store finite group through their Cayley graphs [20].
Since neural networks with tree structure come to researcher's attention, it is natural to ask how the complexity of a tree structure neural network can be measured.Alternatively, it is of interest to know how much information a neural network can store.This motivates the investigation of the notion of tree-shifts introduced by Aubrun and Béal [3,4].A tree-shift is a shift space defined on a Cayley tree, and a shift space is a dynamical system which consists of patterns avoiding a set of local patterns (such a set is called a forbidden set).Roughly speaking, a tree-shift consists of colored Cayley trees.
In [3], Aubrun and Béal indicate that tree-shifts constitute an intermediate class in between one-sided and multidimensional shifts.Ban and Chang propose an algorithm for the computation of the entropy of a class of tree-shifts called tree-shifts of finite type [8].They also demonstrate that the entropy of a tree-shift of finite type is the logarithm of a Perron number and vice versa [6].In other words, tree-shifts of finite type achieves rich phenomena.
Nowadays, we have known that the entropy of a tree structure neural network reflects its complexity.It is natural to consider the complex nature of a neural network representing disordered brain such as Alzheimer's disease, which is one of the most prevalent neurodegenerative disorders causing dementia and related severe public health concerns.It is well-known that Alzheimer's disease is an irreversible, progressive brain disorder that slowly destroys the patient's memory and thinking skills; although the greatest known risk factor is aging, Alzheimer's disease is not just a disease for elders.
Recent experimental investigations support the hypothesis that the concept of illness progression via neuron-to-neuron transmission and transsynaptic transport of pathogens from affected neurons to anatomically interconnected nerve cells is compatible with the inordinately drawn out prodromal phase and the uniform progression of the pathological process in Alzheimer's disease.Also, a neuron-to-neuron transfer and propagation via seeding offer the most straightforward explanation for both the predictable distribution pattern of the intraneuronal tau lesions in the brain as well as the prolonged rate of disease progression that characterize Alzheimer's disease neuropathologically.Readers who are interested in recent development about Alzheimer's disease are referred to [10,11,14] and the references therein for more details.
To investigate the complexity of a system with neuronal dysfunction, we propose neural networks whose topologies are Cayley graphs.The basic idea is that an infected neuron cannot transfer signal correctly.Hence we treat those inherit neurons as dead ones.In other words, if x is a node representing some infected neuron, then no node with prefix x is a node in the graph.To clarify the investigation, we study neural networks whose topology is Fibonacci-Cayley tree, which is a semigroup with generators {1, 2} and a relation 2 * 2 = 2. Namely, the nodes of a Fibonacci-Cayley tree come from the Fibonacci sequence.
In Section 2, we recall some definitions and notions about tree-shifts and extend the concept to define a Fibonacci tree-shift, which is a shift space whose topology is Fibonacci-Cayley tree.It follows from the definition that a tree-shift is a dynamical viewpoint of coloring problem on a Cayley tree under a given rule.
Section 3 provides the idea of the entropy of a Fibonacci tree-shift.We demonstrate therein that the computation of the entropy of a Fibonacci tree-shift of finite type is equivalent to studying a nonlinear recursive system and propose an algorithm for computing entropy.It is remarkable that, except for extending the methodology developed in [6] to Fibonacci tree-shifts of finite type, we can generalize such an algorithm to tree-shifts of finite type with more complex topology.Our result might help in the investigation of the multidimensional coloring problem.
Section 4 applies the results to elucidate the complexity of neural networks on Fibonacci-Cayley tree.The Fibonacci tree-shifts came from neural networks are constrained by the so-called separation property; after demonstrating there are only two possibilities of entropy for neural networks on Fibonacci-Cayley tree, we reveal the formula of the boundary in the parameter space.

Fibonacci-Cayley tree
We introduce below some basic notions of symbolic dynamics on Fibonacci-Cayley tree.The main difference between Fibonacci-Cayley tree and infinite Cayley trees is their topologies.We refer to [3,7] for an introduction to symbolic dynamics on infinite Cayley trees.

Cayley tree
Let Σ = {1, 2, . . ., d}, d ∈ N, and let Σ * = ⋃ n≥0 Σ n be the set of words over Σ, where Σ 0 = {ϵ} consists of the empty word ϵ.An infinite (Cayley) tree t over a finite alphabet A is a function from Σ * to A; a node of an infinite tree is a word of Σ * , and the empty word relates to the root of the tree.Suppose x is a node of a tree.Each node xi, i ∈ Σ, is known as a child of x while x is the parent of xi.A sequence of words Let t be a tree and let x be a node, we refer t x to t(x) for simplicity.A subtree of a tree t rooted at a node x is the tree t ′ satisfying t ′ y = t xy for all y ∈ Σ * , where xy = x 1 ⋯x m y 1 ⋯y n means the concatenation of x = x 1 ⋯x m and y 1 ⋯y n .Given two words x = x 1 x 2 . . .x i and y = y 1 y 2 . . .y j , we say that x is a prefix of y if and only if i ≤ j and x k = y k for 1 ≤ k ≤ i.A subset of words L ⊂ Σ * is called prefix-closed if each prefix of L belongs to L. A function u defined on a finite prefix-closed subset L with codomain A is called a pattern, and L is called the support of the pattern; a pattern is called an n-block if its support L = x∆ n−1 for some x ∈ Σ * , where be the set of all infinite trees over A. For i ∈ Σ, the shift transformations equipped with the shift transformations σ i is called the full tree-shift over A. Suppose w = w 1 ⋯w n ∈ Σ * .Define σ w = σ wn ○ σ wn−1 ○ ⋯ ○ σ w1 ; it follows immediately that (σ w t) x = t wx for all x ∈ Σ * .Suppose that u is a pattern and t is a tree.Let supp(u) denote the support of u.We say that u is accepted by t if there exists x ∈ Σ * such that u y = t xy for every node y ∈ supp(u).In this case, we say that u is a pattern of t rooted at the node x.A tree t is said to avoid u if u is not accepted by t; otherwise, u is called an allowed pattern of t.Given a collection of finite patterns F, let X F denote the set of trees avoiding any element of F. A subset X ⊆ A Σ * is called a tree-shift if X = X F for some F, and F is a forbidden set of X.
A tree-shift X F is called a tree-shift of finite type (TSFT) if the forbidden set F is finite; we say that X F is a Markov tree-shift if F consists of two-blocks.Suppose A 1 , A 2 , . . ., A d are binary matrices indexed by A, the vertex tree-shift X A1,A2,...,A d is defined as It follows immediately that each vertex tree-shift is a Markov tree-shift and each Markov tree-shift is a TSFT.In [7], the authors indicate that every TSFT is conjugated to a vertex tree-shift.

Fibonacci-Cayley tree
Let Σ = {1, 2} and let A = ( 1 1 1 0 ). Denote by Σ A the collection of finite walks of graph representation G (cf. Figure 1) of A together with the empty word ϵ.A Fibonacci-Cayley tree (Fibonacci tree) t over a finite alphabet A is a function from Σ A to A; in other words, a Fibonacci tree is a pattern whose support is the Fibonacci lattice Notably, there are two different supports for n-blocks, say It is seen that the shift transformation σ 2 is not well-defined in this case.We consider the shift Suppose A 1 and A 2 are binary matrices indexed by A, the vertex-Fibonacci tree-shift X A1,A2 is defined as See Figure 2 for the support of a Fibonacci tree.
Example 2.1.Suppose that the alphabet A = {R, G} consists of red and green two colors.Let A 1 and A 2 , the coloring rules on the left and right directions, respectively, be given as That is, we can not color green on any two consecutive nodes.

Complexity of colored Fibonacci-Cayley tree
This section investigates the complexity of Fibonacci tree-shifts of finite type.We use an invariant known as entropy introduced in [6, 8].

Entropy of Fibonacci tree-shifts
Let X be a Fibonacci tree-shift.For n ∈ N and w n (X) ∶ u w = c} and γ as follows.
Definition 3.1.Suppose X is a Fibonacci tree-shift.The entropy of X, denoted by h(X), is defined as where ln 2 = ln ○ ln.
Notably, the growth rate of the number of nodes of ∆ n ⋂ Σ A is exponential; this makes the speed of increase of γ n doubly exponential.Hence, the entropy h(X) measures the growth rate of feasible patterns concerning their height.
c ℓ ;n , and For w ∈ Σ A , the follower set of w is defined as Since each element w = w 1 w 2 ⋯w n ∈ Σ A is a finite walk in the Fibonacci graph G (cf. Figure 1), it is easily seen that This makes γ [1] i;n = γ i;n and γ [2] i for 1 ≤ i ≤ k and n ≥ 3 ∈ N. Substituting (4) with (5) derives Alternatively, computing the entropy of X is equivalent to the investigation of the following nonlinear recursive system.
Example 3.2.Suppose that A = {c 1 , c 2 } and Observe that In the next subsection, we introduce an algorithm for solving the growth rate of the nonlinear recursive system (6).

Computation of entropy
The previous subsection reveals that computing the entropy of a Fibonacci tree-shift of finite type is equivalent to studying a corresponding nonlinear recursive system.To address an algorithm for the computation of the entropy, we start with the following proposition.Proposition 3.3.Suppose that X = X A1,A2 is a vertex-Fibonacci tree-shift over A with respect to A 1 , A 2 .Then the limit (3) exists and Proof.From the Cauchy inequality It can be verified without difficulty that It is noteworthy that if there exist 1 ≤ i ≤ k and m ≥ 2 such that γ i;m ≥ γ j;m for all j ≠ i, then γ i;n ≥ γ j;n for all j ≠ i and n ≥ m ∈ N. Without loss of generality, we may assume that γ 1;n ≥ γ i;n for all n ≥ 2 ∈ N and 1 ≤ i ≤ k.The inequality Therefore, we conclude that This completes the proof.
A symbol c ∈ A is called essential (resp.inessential ) if γ c;n ≥ 2 for some n ∈ N (resp.γ c;n = 1 for all n ∈ N).Proposition 3.3 infers the alphabet A can be expressed as the disjoint union of two subsets A E and A I consisting of essential and inessential symbols, respectively.In addition, it is easily seen from ( 7) that For the simplicity, we assume that A = A E ; that is, each symbol is essential.Now we turn back to the elaboration of the nonlinear recursive system (6).First, we consider a subsystem arose from (6).Notably, for each i, the above subsystem contains only one term in the original system and the coefficient is 1.We call such a system simple.Generally speaking, a Fibonacci tree-shift has many simple recurrence representations.The following theorem reveals the entropy of a vertex-Fibonacci tree-shift with simple recurrence representation relates to the spectral radius of some integral matrix.
Theorem 3.4.Suppose that X = X A1,A2 is a Fibonacci tree-shift over A such that each symbol is essential.
If the recurrence representation of X is simple, then there exists an integral matrix M such that where ρ M is the spectral radius of M .
Proof.Since c i is essential for 1 ≤ i ≤ k, we may assume that γ i;2 ≥ 2 for simplicity.Proposition 3.3 demonstrates that Let be a 2k × 1 vector.The recursive system (5) suggests that there exists a 2k × 2k nonnegative integral matrix M such that On the other hand, This completes the proof.
We call the matrix M sketched in Theorem 3.4 the adjacency matrix of the simple recursive system (5).This makes Theorem 3.4 an extension of a classical result in symbolic dynamical systems.
Example 3.5.Suppose that the alphabet A and A 1 , A 2 are given, and the recurrence representation of X A1,A2 is Furthermore, each symbol in A is essential.Let and the spectral radius of is the golden mean.Theorem 3.4 asserts that the entropy of X A1,A2 is h(X A1,A2 ) = ln g.
Theorem 3.6.Suppose that X = X A1,A2 is a Fibonacci tree-shift over A such that each symbol is essential.Then The main idea of the proof of Theorem 3.6 can be examined via the following example.
Example 3.7.Suppose that A = {c 1 , c 2 } and are the same as considered in Example 3.2.Recall that the recurrence representation of X A1,A2 is

It can be verified without difficulty that
Therefore, we have two inequalities and Consider the nonlinear recursive system It is noteworthy that M is also the adjacency matrix of the simple recurrence representation where 1 4 = (1, 1, 1, 1) T .Therefore, we derive from Proposition 3.3 that is the golden mean.
Proof of Theorem 3.6.Since every symbol is essential, we may assume that, for 1 ≤ i ≤ k, γ i;2 ≥ 2 for simplicity.Set ̵ h = max{ln ρ M ∶ M is the adjacency matrix of a simple subsystem of X}.
Similar to the discussion in the proof of Theorem 3.4, there exists a 2k × 2k integral matrix M such that where 1 2k = (1, . . ., 1) T is a 2k × 1 vector.Combining the computation above with the fact Observe that ( 14) is a simple recurrence representation of X if we replace κ in ( 14) by 1, and M is the adjacency matrix of such a new system.Therefore, we conclude that h(X) ≤ ̵ h.
The proof is thus complete.
Remark 3.8.A key presumption in Theorem 3.6 is that every symbol is essential.If there are inessential symbols, the proof of Theorem 3.6 infers that we only need to replace M by M ′ , where M ′ is obtained by deleting all the rows and columns indexed by those inessential symbols.

Neural networks on Fibonacci-Cayley tree
In this section, we apply the algorithm developed in the previous section to neural networks with the Fibonacci-Cayley tree as their underlying space.For some reasoning, the overwhelming majority of considered underlying spaces of neural networks are Z n lattice for some n ∈ N; nevertheless, it is known that the characteristic shape of neurons is tree.Some medical experiments suggest that the signal transmission might be blocked by those diseased neurons, which motivate the investigation of neural networks on "incomplete" trees such as Fibonacci-Cayley tree.

Neural networks on Cayley tree
To clarify the discussion, we focus on the neural networks proposed by Chua and Yang [13], which is known as cellular neural network and is widely applied to many disciplines such as signal propagation between neurons, pattern recognition, and self-organization; the elaboration of other models such as Hopfield neural networks can be carried out analogously.Let Σ * be the set of nodes of Cayleys as described in the previous section.A cellular neural network on Cayley tree (CTCNN) is represented as for some finite set N ⊂ Σ * known as the neighborhood, v ∈ N , and t ≥ 0. The transformation x w x w1 x w2 a 1 a 2 a Figure 3.A cellular neural network with the nearest neighborhood defined on binary trees.In this case, the neighborhood N = {ϵ, 0, 1} and a = aϵ.
is called the output function or activation function, and z is called the threshold.The weighted parameters A = (a v ) v∈N is called the feedback template, and Figure 3 shows the connection of a binary CTCNN with the nearest neighborhood.
A mosaic solution x = (x w ) w∈Σ * of ( 15) is an equilibrium solution which satisfies |x w | > 1 for all w ∈ Σ * ; its corresponding pattern y = (y w ) w∈Σ * = (f (x w )) w∈Σ * is called a mosaic output pattern.Since the output function ( 16) is piecewise linear with f (s) = 1 (resp.−1) if s ≥ 1 (resp.s ≤ −1), the output of a mosaic solution x = (x w ) w∈Σ * must be an element in {−1, +1} Σ * , which is why we call it a pattern.Given a CTCNN, we refer to Y as the output solution space; namely, Y = {(y w ) w∈Σ * ∶ y w = f (x w ) and (x w ) w∈Σ * is a mosaic solution of (15)} .

Learning problem
Learning problem (also called the inverse problem) is one of the most investigated topics in a variety of disciplines.From the mathematical point of view, determining whether a given collection of output patterns can be exhibited by a CTCNN is essential for the study of learning problem.This subsection reveals the necessary and sufficient condition for the capability of exhibiting the output patterns of CTCNNs.For the compactness and consistency of this paper, we focus on cellular neural networks on the binary tree with the nearest neighborhood.The discussion of general cases is similar to the investigation in [5,9,12], thus it is omitted.
A CTCNN with the nearest neighborhood is realized as where a, a 1 , a 2 ∈ R and w ∈ Σ * .Considering the mosaic solution x = (x w ) w∈Σ * , the necessary and sufficient condition for Similarly, the necessary and sufficient condition for Let and let α = (a 1 , a 2 ) represent the feedback template without the self-feedback parameter a.The admissible local patterns with the +1 state in the parent neuron is denoted by where "⋅" is the inner product in Euclidean space.Similarly, the admissible local patterns with the −1 state in the parent neuron is denoted by Furthermore, the admissible local patterns induced by (A, z) can be denoted by where Namely, B(A, z) consists of two-blocks over A = {1, −1}.For simplicity, we omit the parameters (A, z) and refer to B(A, z) as B. Meanwhile, we denote the output space Y by X B to specify the set of local patterns B.
Suppose U is a subset of V n , where n ≥ 2 ∈ N. Let U c = V n ∖ U .We say that U satisfies the linear separation property if there exists a hyperplane H which separates U and U c .More precisely, U satisfies the separation property if and only if there exists a linear functional g(z 1 , z 2 , . . ., z n ) = c 1 z 1 +c 2 z 2 +⋯+c n z n such that g(v) > 0 for v ∈ U and g(v) < 0 for v ∈ U c . Figure 4 interprets those U ⊂ V 2 satisfying the linear separation property.Proposition 4.1 elucidates the necessary and sufficient condition for the learning problem of CTCNNs, which follows from straightforward examination.Thus the proof is omitted.Such a property holds for arbitrary N provided N is prefix-closed.Readers are referred to [5] for more details.We emphasize that Proposition 4.1 demonstrate the parameter space can be partitioned into finitely many equivalent regions.Indeed, whenever the parameters a 1 and a 2 are determined, (18) and (19) partition the a-z plane into 25 regions; the "order" (i.e., the relative position) of lines a − 1 + (−1) ℓ z = (−1) ℓ (a 1 y w1 + a 2 y w2 ), ℓ = 1, 2, can be uniquely determined by the following procedures: 1) The signs of a 1 , a 2 (i.e., the parameters are positive or negative). (-1,-1) (1,-1) 2) The magnitude of a 1 , a 2 (i.e., |a This partitions a-z plane into 8×25 = 200 regions.As a brief conclusion, the parameter space is partitioned into at most 200 equivalent regions.A careful but straightforward examination indicates that two symbols are both inessential if and only if the pair of parameters (a, z) is in the plane below the red W -shape line.

Neural networks on Fibonacci-Cayley tree
The definition of cellular neural networks on Fibonacci-Cayley tree is similar to the definition of Fibonacci-Cayley tree-shifts.Let A and Σ A be the same as defined in the previous section.A cellular neural network on Fibonacci-Cayley tree (FTCNN) with the nearest neighborhood is realized as where a, a 1 , a 2 ∈ R and w = w 1 ⋯w n ∈ Σ A .Suppose the parameters A = (a, a 1 , a 2 ) and z are given.Let B be the set of feasible local patterns corresponding to (A, z) which is studied in the previous subsection.It can be verified without difficulty that the output space of (23) is In other words, the output space Y F of ( 23) is a Markov-Fibonacci tree-shift.

Entropy of neural networks on Fibonacci tree
It is of interest that how much information a diseased neural network can store.Based on the algorithm developed in the previous section, we study the entropy of FTCNNs with the nearest neighborhood.
Suppose that the parameters a 1 , a 2 are given.A pair of parameters (a, z) is called critical if, for each r > 0, there exists (a ′ , z ′ ), (a ′′ , z ′′ ) such that h(X B ′ ) ≠ h(X B ′′ ) = 0, where B ′ = B(a ′ , a 1 , a 2 , z ′ ), B ′′ = B(a ′′ , a 1 , a 2 , z ′′ ).Notably, the a-z plane is partitioned into 25 equivalent regions whenever a 1 and a 2 are given and is indexed as [p, q] for 0 ≤ p, q ≤ 4; we use Y [p,q] instead of X B for simplicity.
Next, we show that (a, z) is critical if and only if (a, z) satisfies (24).Let C = {ℓ 1 a 1 + ℓ 2 a 2 ∶ ℓ 1 , ℓ 2 ∈ {−1, 1}}, and let be the largest and the second largest elements in C, respectively.A careful but straightforward verification asserts that (a, z) is critical if and only if (See Figure 5 for more information.)The desired result follows from the fact that This completes the proof.

Conclusion
This paper studies the entropy of Fibonacci-Cayley tree-shifts, which are shift spaces defined on Fibonacci-Cayley trees.Entropy is one of the most frequently used invariant that reveals the growth rate of patterns stored in a system.Followed by demonstrating that the computation of the entropy of a Fibonacci tree-shift of finite type is equivalent to studying a nonlinear recursive system, we propose an algorithm for computing entropy.It is seen that the so-called simple recursive system plays an important role.
As an application, we elucidate the complexity of neural networks whose topology is Fibonacci-Cayley tree.Such a model reflects a brain going with neuronal dysfunction such as Alzheimer's disease.A Fibonacci tree-shift came from a neural network is constrained by the so-called separation property.Aside from demonstrating a surprising phenomenon that there are only two possibilities of entropy for neural networks on Fibonacci-Cayley tree, we reveal the formula of the boundary in the parameter space.
A related interesting problem is "does the black-or-white entropy spectrum of neural networks still hold for other network topologies?"Further discussion is in preparation.

1 2 Figure 1 .
Figure 1.Each node of the Fibonacci-Cayley tree is a finite walk in the Fibonacci graph.

Proposition 4 . 1 .
A collection of patterns B = B + ⋃ B − can be realized in (17) if and only if either of the following conditions is satisfied: (Inv1) − B+ ⊆ B− and B− satisfies linear separation property; (Inv2) − B− ⊆ B+ and B+ satisfies linear separation property.Herein, B+ and B− are defined in (20) and (21), respectively.

Figure 4 .
Figure 4. Suppose U is a proper subset of V 2 = {−1, 1} 2 .There are only 12 choices of U that satisfies the linear separation property

Figure 5 .
Figure 5.For each chosen pair of parameters (a1, a2), the a-z plane is partitioned into 25 equivalent regions.

Theorem 4 . 3 .
Suppose that the parameters a 1 , a 2 are given.Let