AI

From MATLAB To Python

Posted on

MATLAB:

For many years MATLAB has been my primary tool for prototyping algorithms, because of its rich set of optimization functions and the AI tool box it has proven to be a valuable tool to have. However, It is not cheap and if you do not have a company to pay for license or attend a university that provides you with a license then you will have to find an alternative.

MATLAB is not a programming language rather its a tool that has as part of its framework a programming language called M language, this language has a lot of quirks and takes some getting use to, the other issue I found with MATLAB is that the functions while well documented do not seem to follow a standard in terms of parameters; on the whole while MATLAB is a good tool for prototyping and is used a lot in engineering and medical fields which are my core domain; However,I am forced to look for a cheaper/free alternative that will give me as much if not more tools than MATLAB now provides.

Python to the rescue:

Python is powerful… and fast;
plays well with others;
runs everywhere;
is friendly & easy to learn;
is Open

All these wonderful things make Python a big contender for my MATLAB replacement.

The first thing we want to do is install Python.
Next install my favurite Python IDE PyCharm
Create a new Python Project using PyCharm
Python project

How do we add packages to our project?
Python is nothing without its packages and two of my favourites are numpy and scipy. To add these packages simply download the Anaconda distribution and configure it to be your default python implementation.

pylibs

And here is my first piece of python code as taken from the python website 🙂

pyresult

All I need now is a good Python book and 2-4 months to delve into the language. Stay tuned for more posts on my Python journey. Happy coding!!

Using A Perceptron ANN With Genetic Algorithm Trainer To Solve Logical AND

Posted on Updated on

Neural nets (NN)  have always been a “black box” fascination for me until recently when i had my eureka moment. Neural Networks are mainly used for classification and prediction applications include stock market trends, feature recognition and some types of optimization techniques. A NN can be trained in two ways, supervised and unsupervised.

Supervised
The NN is exposed to a dataset with the training values and the correct outputs to those values. A Training algorithm such as back propagation is used to minimize the error between resulting outputs and the correct outputs.

Unsupervised
The NN is allowed to form its own conclusion based on the dataset provided. These types of networks are usually used for classification type problems, a very good example of this it the SOM (Self Organizing Map) network.

That said, if you have not already read my post on Genetic Algorithms (GA) please do so now, since we will be using GA’s as the means of optimizing the required weights for each input of the Perceptron.

Wow you are back already? Ok lets look at a simple NN to see how it works, the NN we will be reviewing is called the Perceptron this particular configuration has only two input neurons since we will be dealing with a binary operation and one output neuron.

Simple Perceptron

i = inputs
w = weights
o = output
b = bias

A bias is also associated with the network, this bias is simply a neuron with a constant multiplied by a weight, this bias is added to the activation function in order for the NN to recognize problems that are not linearly separable. Each input is multiplied by a corresponding weight, the weight is a small value which will be adjusted until the the output neuron is activated. Activation occurs when the activation function Σ(i1*w1+i2*w2)+b is greater than certain threshold. The output of the activation function is then sent to a transfer function which interprets that output as 1 or 0 depending on the inputs supplied to the NN.

 

Simple Genetic Algorithm To Evolve A String of Integers

Posted on Updated on

Genetic Algorithms are a means of optimization copied from the natural world. According to the theories of evolution, nature has a way of selecting the best (fittest) individuals to mate (Crossover) and reproduce; thus carrying on the best features of each selected individual. Even though traits from both parents are carried over into children there is still an element of randomness involved (Mutation) that gives their offspring the ability to explore their fitness landscape (adapt) to their environment. Genetic Algorithms tries to mimic this behavior with three common operators (1)Selection, (2)Crossover, (3)Mutation.

Selection:  
The selection operator determines how the individuals of a population are selected to mate, the most popular selection method is called elitism and this is the method that we will use in our genetic algorithm implementation.

Crossover:
The crossover operator determines how the parents are recombined to form offspring. We will be using single point crossover in this implementation.

Mutation:
Mutation inserts randomness into the genotype of each offspring giving it the ability to diversify from the features of its parents.

Note
Implementing genetic algorithms can be seen as somewhat of an art because almost all of the code is boiler plate except for the chromosome representation used and how the fitness of each individual is calculated. These two factors usually have the most impact on the accuracy and speed of the genetic algorithm and are the most difficult to represent. These factors along with the mutation rate, crossover rate and selection method have to be tinkered with until a viable configuration is reached.

This implementation will evolve the number sequence 123456789 in that specific order.

Interesting Functions

private static void CalculateFitness(sequence seq)
{
var ordered = new sequence{buffer = new List<string>() {"1", "2", "3", "4", "5", "6", "7", "8", "9"}};

int result=0;
for (int i = 0; i < 9; i++)
{
if (ordered.buffer[i] == seq.buffer[i]) result++;
}
seq.fitness = result;
}

 

private void Epoch(List<sequence> population)
        {
            //Elitism
            survivors.AddRange(population.Where(i => i.fitness >= survivorThreshold));

            Mutate(population.Where(i => i.fitness >= mutantThreshold));

            CrossOver(population.Where(i => i.fitness >= crossoverThreshold) as IEnumerable);

            population.Clear();
            population.AddRange(survivors);

            for (int i = 0; i < populationSize - survivors.Count; i++)
            {
                var temp = GenerateSequence();
                CalculateFitness(temp);
                population.Add(temp);
            }
            survivors.Clear();
        }

The full code listing can be found at github