Neuro-Genetic Hybrid Model

Introduction 

In this project we aim to solve customer churn prediction problem using different classification algorithms. Customer churn prediction means that we want to predict that whether a customer will stay with our network or leave our network. Our main objective is to check the efficacy of the Neuro-Genetic Hybrid system with traditional classification methods such as Logistic Regression, Neural Network and KNN. We evaluate the models based on different metrics. Our results show that Neuro-genetic model outperforms all traditional classification models. 

Data

The data set we are using for this study is South-East Asia Telecommunication data set of Pakistan. It consists of 14 different attributes of network usage for different networks such as Telenor, Ufone, Mobilink and Warid. However, not all attributes are related in explaining the target variable i.e. ‘Churned’. Hence, we used correlation analysis to select only the five most important attributes in the data using our own correlation threshold. The selected attributes are as following. 

1. Network Age 

2. Aggregate Total Revenue 

3. Aggregate Data Usage 

4. Aggregate Calls 

5. Aggregate On-net Revenue 

The correlation matrix for these five attributes is shown below. From this we can see that Aggregate calls is highly correlated with customer churn. The data set was then normalized so that each attribute has an equal weightage in the algorithm fitting process. We used standard normalization process for this. There were no missing values in the data set. For evaluation purpose, we split our data into training and test set using a 90:10 split ratio. 

Chromosome representation 

The chromosome for this system is a one-dimensional array consisting of weights of the neural network for both hidden layer and the output layer. Other hyper-parameters such as the number of hidden nodes, learning rate or number of hidden layers could be added. However, for simplicity’s sake we decided on making the model simple. We had five input and hidden nodes and 2 output nodes. So, our chromosome length is 35. 

Length of chromosome = (input nodes x hidden nodes) + (hidden nodes x output nodes) 

Fitness Function 

Neural network works by repeated forward pass and backward pass. After each forward pass the loss is obtained by using the cross-entropy loss function. Hence, the objective of the neural network is to minimize the loss after learning the weights. To calculate the fitness, we took inverse of this cross-entropy loss. Thus, the objective of our hybrid system is to maximize the fitness i.e. inverse of loss. 

Producing Offsprings 

For selecting the fittest individuals for mating from the population where the fittest individuals have greater proportion in the next generation. Afterwards, we applied crossover at selected points in the chromosome. For the weights of the hidden layer, we applied cross over at two points and for output layer weights at one point. Finally, we introduced some mutations with a pre-defined probability at random points in these crossed chromosomes to produce the children. These children then replace the previous population and the process is repeated until convergence. 

Learning in Standard Neural Network 

In a standard Neural network, the learning process is as following. 

1. Initialize weights randomly 

2. Forward pass

3. Calculate loss

4. Backward Pass

5. Update weights

6. Repeat 2-5 for given number of epochs

Learning in Neuro-Genetic Hybrid System 

In the hybrid system, we represent the weights of the neural network in the form of chromosomes which is one-dimensional array of weights. There is no backward pass or update of weights in this system. The main steps are as following. 

1. Initialize the population represented by chromosomes. 

2. Convert one-dimensional chromosomes to 2-dimensional weights. 

3. Do a forward pass using these weights and calculate the cross-entropy loss. 

4. Calculate fitness which is the inverse of the loss. 

5. Based on this fitness select the best individuals i.e. their chromosomes. 

6. Do crossover and mutation on these chromosomes and produce offsprings. 

7. Repeat steps 2-6 for given number of generations. 

The main objective is that, based on the Darwin’s theory of survival of the fittest, only the fittest individuals will be able to pass on their genes to the next generations. Towards the end, only the fittest individuals will be left. From the fitness plot, we can observe that the over the generations the fitness is increasing which means that fittest individuals are surviving. 

Limitations 

The results obtained are acceptable but not the best. However, this is due to the nature of the data set itself. There are not many telecommunication data sets available on customer churn prediction. Hence, the model’s accuracy is limited by it. Secondly, to further evaluate this proposed hybrid system, other classification data sets can be used in the future.