Be Willing to Be Amazed

There was no way to plan what happened When I babbled to everyone I met Last night from bookseller to reader to writer — Except barista with downcast, red-rimmed eyes. I’d done this adventure for…

独家优惠奖金 100% 高达 1 BTC + 180 免费旋转

Evolving artificial neural networks using genetic algorithms

This blog is the culmination of a simple question asked a while ago: "Can we train and improve neural networks (NN) using the concepts of selection and evolution observable in nature?".

What to expect?

First, we’ll have a small overview of what evolution means and what are the elements necessary to be able to simulate such an environment.

Finally, I’ll present my own introductory exploration of this topic. The decisions which were made and the first set of small experiments.

In biology, evolution posits that plants, animals, and other organisms have their origin in preexisting types and that the distinguishable differences are due to changes wrought by successive generations.

Darwin’s theory states that evolution occurs through a series of small changes that occur over a long period of time. This evolutionary process is driven by the following mechanisms:

Providing the following conditions are met, evolution can happen without any oversight or planning:

Self-replication implies the instantiation and creation of the next generation. The origin of the first self-replicating organism is not entirely clear from a scientific perspective. Evolution is descent with modification and does not explain the origin of life. Every creature coming into existence must be built from a blueprint. The genome, stored as DNA or RNA, contains the information that is passed from generation to generation. Occasionally, the inherited genome that is passed to the offspring has to undergo mutations. Lastly, there needs to be a criterion that selects the fittest organisms. Only those will reproduce.

Firstly, how do we encode neural networks using an efficient genetic representation? Which is a similar question to how will agents/creatures/organisms/individuals be genetically represented?

The way we encode genetic information is the genotype; the way we represent it using neural networks is the phenotype. The genotype can thus be encoded directly or indirectly.

It is arguably the simplest to encode the genotype directly. The definition of a connection matrix can be achieved by a simple index or data structure approach using traditional bit string representation. The result is a clear and obvious relationship between genotype and phenotype.

Bit representation of Genetic Algorithm.

The 3 main disadvantages of this are:

Since bit string representations are not the most natural representation for networks, graph encoding is used to represent them more explicitly. Encodings are typically grid-based, as illustrated below:

The 2 main disadvantages are:

By indirect genotype encoding, the rules or parameters for creating an organism are specified. Although much smaller, bias within the search space can occur. An understanding of genetic and neural mechanisms is necessary for the proper use of indirect encodings so that the search does not focus on some suboptimal class of topologies.

NEAT uses a direct encoding approach where two lists of genes are created: one for the nodes and one for the connections. Node genes provide a list of inputs, hidden nodes, and outputs that can be connected. Each connection gene specifies the in-node, the out-node, the weight of the connection, whether or not the connection gene is expressed, and an innovation number (which will be explained below).

As shown below, there are 2 types of mutations:

Crossover is a genetic operator that determines how genetic information from parents is recombined to produce new offspring. In this paper, the genetic recombination of neural networks is discussed in light of two challenges.

The first is called competing conventions where simply crossing over genomes of two neural networks could result in “broken” networks. The researchers discovered that it has a greater likelihood of occurring in genomes representing the same solution but with different encodings.

In the neural network seen above, the three hidden neurons A, B, and C can produce the same result with 6 different permutations of the neural network. When one of these permutations crosses over with another, critical information is likely to be lost. For example, crossing [A, B, C] and [C, B, A] can result in [C, B, C] losing one-third of the representations.

The 2nd challenge tackled is concerning homology which is the alignment of chromosomes through gene matching for specific traits.

Matching up genomes for different network topologies using innovation numbers

First, a simulation is generated where we define:

Upon creation, an agent is randomly placed on the grid world with a randomly generated set of bit strings representing the DNA. This complete set of DNA, the genome, is then used to build an instructions-set as a brain.

The genome is a collection of genes that are made from DNA. Below, we can see that agents are instantiated with 2 genes. One of the assumptions made is that the more genes an agent has, the more it is capable of complex behavior.

As we can see, a gene’s DNA consists of 16 bits. The first generation of agents has a randomly generated binary sequence. DNA is the molecule that is the hereditary material in all living cells. As such, certain bit sequences will be passed on from a surviving (fit) agent to a newer agent, increasing its odds of survival.

From an agent’s genome, a brain is built with the NeuronFactory class. Once the neurons are created, they are connected into a directed, or not, graph.

Based on a current agent’s state, the brain then traverses this graph to return a decision that dictates the agent’s next action.

Before explaining how the NeuronFactory class works, let’s discuss the design decision on how to encode information in a 16-bit binary string.

First of all, a distinction was made in terms of Neuron types:

Secondly, the NeuronFactory decodes neural information with 2 helper classes: Source/Sink Decoding. Each class defines how a 16-bit string should be interpreted by the factory class. A source or hidden neuron occupies the first 8 bits, and a sink or hidden neuron occupies the last 8. The index neuron indicates whether a sink/source neuron or a hidden neuron will be created. Type indices will denote which type of sink/source neurons will be defined. Weight and bias indices define the neuron’s weight/bias.

Finally, the NeuronFactory will return a pair of connected neurons to be added to the brain’s graph. We could increase the bit length of the gene and/or the genome length in order to produce more complex behavior. This would allow our agent’s brain to have more neurons and/or more layers.

Assuming that each agent has only one 16-bit gene, we build a simulation where agents can successfully pass their genes on to the next generation if they are on the left half of the grid world at the end of the simulation. The bits of the genes are randomly instantiated, resulting in random behavior. As surviving agents pass their genes, we can observe that an increasing number start to migrate to the left half of the grid.

The second simulation will be similar, with the exception that the agents will pass their genes to the next generation if they end up in the upper right corner. Because a gene encodes a pair of source/sink neurons, we must instantiate our agents with two genes. A gene for the left movement and another for the up movement.

Despite barely scratching the surface, the following elements will be explored:

That’s about it! Hopefully, some of you will find this useful and I look forward to sharing my findings with you.

Be Willing to Be Amazed

Evolving artificial neural networks using genetic algorithms

Add a comment

Related posts:

Sui Name Service

Cathie Wood Explains The Future Of Crypto And NFTs

Perfume Bottles