Genetic Algorithm – Pratical Example with Keras and Open.AI Challenge

In this article we will talk about the usage of a Genetic Algorithm approach to optimize Keras Neural Network that may use 2 types of Hidden Layers (Dense and/or Dropout) mixed.

To accomplish that goal, we work to optimize a Neural Network created in my previous article:

To learn and understand more about Genetic Algorithm, click here.

If you are anxious for the code, click >

I strongly recommend you to read the previous links about the Genetic Algorithm (in case you don’t have any idea about Genetic Algorithm) and the link to the CartPole-V1 challenge.

Otherwise, this article may be confuse for who don’t have the correct background.

OK, let’s dive into code

Genetic Algorithm Parameters

# Define Hyperparameters for NN
HIDDEN_LAYER_COUNT = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
HIDDEN_LAYER_NEURONS = [8, 16, 24, 32, 64, 128, 256, 512]
HIDDEN_LAYER_RATE = [0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9]
HIDDEN_LAYER_ACTIVATIONS = ['tanh', 'relu', 'sigmoid', 'linear', 'softmax']
HIDDEN_LAYER_TYPE = ['dense', 'dropout']
MODEL_OPTIMIZER = ['adam', 'rmsprop']

# Define Generic Algorithm Parameters
MAX_GENERATIONS = 50  # Max Number of Generations to Apply the Generic Algorithm
POPULATION_SIZE = 20  # Max Number of Individuals in Each Population
BEST_CANDIDATES_COUNT = 4  # Number of Best Candidates to Use
RANDOM_CANDIDATES_COUNT = 2  # Number of Random Candidates (From Entire Population of Generation) to Next Population
OPTIMIZER_MUTATION_PROBABILITY = 0.1  # 10% of Probability to Apply Mutation on Optimizer Parameter
HIDDEN_LAYER_MUTATION_PROBABILITY = 0.1  # 10% of Probability to Apply Mutation on Hidden Layer Quantity

First of all, let’s define our parameters range to use for you Neural Network.

The parameter range consist in range of all possible values for each parameters that we want to evaluate for the Neural Network topology. In this example case we defines the Hidden Layers Count, Layer Size, Layer Neuron Count and Activation Function (Only for Dense Layer), Layer Dropout rate (only for Dropout Layer) and whole model optimization strategy.

Then, we have all definitions for the Genetic Algorithm processing it self. That contains the Number of Generations to Run, Population Size, Candidates Selection Count (Best and Random) and Mutation Probabilities Rate.

Chromosome and Hidden Layer Layout

class LayerLayout:
    def __init__(self, layer_type):
        self.neurons = None
        self.activation = None
        self.rate = None
        self.layer_type = layer_type

class Chromosome:
    def __init__(self, layer_layout, optimizer, specie):
        self.layer_layout = layer_layout
        self.optimizer = optimizer
        self.result_worst = None
        self.result_best = None
        self.result_avg = None
        self.result_sum = None
        self.specie = specie
        self.ml_model = None

    def safe_get_hidden_layer_node(self, index=0):
        if len(self.layer_layout) > index:
            return self.layer_layout[index]
        return None

Using 2 new classes, one for Hidden Layer Specification (LayerLayout) and other one for the Chromosome (Chromosome).

This 2 classes holds all information that’s our algorithm needs to run, evaluate, evolve, mutate and run it again, again, again, and again over the Generations.

Main Process

def main():

    # Play Random (Initial) Games to create Test and Training Data
    df = play_random_games(games=10000)

    print("\n********** Reference Network Model **********")

    # Creates a Reference NN Model bases on Sample
    ml_model = generate_reference_ml(df)

    # Play Games with Reference NN Model
    print("[+] Playing Games with Reference NN Model \t>", end='', flush=True)
    ref_worst, ref_best, ref_avg, ref_sum = play_game(ml_model=ml_model, games=100)
    print(f"\tWorst Score:{ref_worst} | Average Score:{ref_avg} | Best Score:{ref_best} | Total Score:{ref_sum}")

    # >>>>>> Genetic Algorithm Section <<<<<<
    print("\n********** Genetic Algorithm **********")
    population = generate_first_population_randomly(

    # Run Each Generation
    for current_generation in range(MAX_GENERATIONS):
        print(f"[+] Generation {current_generation+1} of {MAX_GENERATIONS}")
        i = 0

        # >>>>>> Training Phase <<<<<<
        print(f"\tTraining Models: ", end='', flush=True)
        training_start = time.time()

        # Train all Models in Population
        for individual in population:
            generate_model_from_chromosome(df, individual)

        training_stop = time.time()
        print(f"Done > Takes {training_stop - training_start} sec")

        # >>>>>> Evaluation Phase <<<<<<
        print(f"\tEvaluating Population: ", end='', flush=True)
        evaluation_start = time.time()

        for individual in population:

            # Play the Games
            score_worst, score_best, score_avg, score_sum = play_game(

            # Update Chromosome Results
            individual.result_worst = score_worst
            individual.result_best = score_best
            individual.result_avg = score_avg
            individual.result_sum = score_sum

            # Update Indexer
            i += 1

        evaluation_stop = time.time()
        print(f"Done > Takes {evaluation_stop - evaluation_start} sec")

        # Sort Candidates by Sum of Results
        population.sort(key=lambda x: x.result_sum, reverse=True)

        # Compute Generation Metrics
        gen_score_avg = np.average([item.result_avg for item in population])
        gen_score_min = np.min([item.result_worst for item in population])
        gen_score_max = np.max([item.result_best for item in population])
        gen_score_sum = np.sum([item.result_sum for item in population])

        print(f"\tWorst Score:{gen_score_min} | Average Score:{gen_score_avg} | Best Score:{gen_score_max} | Total Score:{gen_score_sum}")

        # >>>>>> Genetic Selection, Children Creation and Mutation <<<<<<
        population = evolve_population(population)

Our main process consist into execute the Generic Algorithm steps:

  • Generate First Population
  • Evaluate Generations
    • Training Population
    • Evaluate Individuals
    • Select Parents
    • Select Random Individuals
    • Create Children;s
    • Mutate new Children’s

Generating the First Population

def generate_first_population_randomly(population_size=10):
    result = []

    for current in range(population_size):

        # Choose Hidden Layer Count
        hidden_layer_count = HIDDEN_LAYER_COUNT[random.randint(0, len(HIDDEN_LAYER_COUNT)-1)]
        hidden_layer_layout = []

        # Define Layer Structure
        for current_layer in range(hidden_layer_count):

        chromosome = Chromosome(
            optimizer=MODEL_OPTIMIZER[random.randint(0, len(MODEL_OPTIMIZER)-1)],
            specie=f"I {current}"


    return result

def create_random_layer():
    layer_layout = LayerLayout(
        layer_type=HIDDEN_LAYER_TYPE[random.randint(0, len(HIDDEN_LAYER_TYPE) - 1)]

    if layer_layout.layer_type == 'dense':
        layer_layout.neurons = HIDDEN_LAYER_NEURONS[random.randint(0, len(HIDDEN_LAYER_NEURONS) - 1)]
        layer_layout.activation = HIDDEN_LAYER_ACTIVATIONS[random.randint(0, len(HIDDEN_LAYER_ACTIVATIONS) - 1)]

    elif layer_layout.layer_type == 'dropout':
        layer_layout.rate = HIDDEN_LAYER_RATE[random.randint(0, len(HIDDEN_LAYER_RATE) - 1)]

    return layer_layout

Initially, we generate a completely random population. Each individual with a random number of Layers, random Optimizer and random configuration for each Hidden Layer.

Converting Layer Layout and Chromosome into a Neural Network Model

def generate_model_from_chromosome(df, chromosome):
    # Define Neural Network Topology
    m_model = Sequential()

    # Define Input Layer (Fixed)

    # Add Hidden Layers
    for layer in chromosome.layer_layout:

        if layer.layer_type == 'dense':
        elif layer.layer_type == 'dropout':

    # Define Output Layer (Fixed)
    m_model.add(Dense(2, activation='sigmoid'))

    # Compile Neural Network
    m_model.compile(optimizer=chromosome.optimizer, loss='categorical_crossentropy')

    # Fit Model with Data
        df[['cart_position', 'cart_velocity', 'pole_angle', 'pole_velocity_at_tip']],
        df[['action_to_left', 'action_to_right']],

    # Update Model into Chromosome
    chromosome.ml_model = m_model

Now, it’s time to convert our definition models (LayerLayout and Chromosome) into a valid Keras Neural Network Model. During this procedure, we build, compile and fit the NN model.

Children Generation

After each generation fitness and evaluation we need to generate a fresh new children from selected parents. Again, it is time to use some randomness in the process.

def generate_children(mother, father):

    # Layer Layout
    c_layer_layout = []
    layers_counts = len(mother.layer_layout) if random.randint(0, 1) == 0 else len(father.layer_layout)
    for ix in range(layers_counts):
            mother.safe_get_hidden_layer_node(ix) if random.randint(0, 1) == 0 else father.safe_get_hidden_layer_node(ix)

    # Remove all Nones on Layers Layout
    c_layer_layout = [item for item in c_layer_layout if item is not None]

    # Optimizer
    c_optimizer = mother.optimizer if random.randint(0, 1) == 0 else father.optimizer

    chromosome = Chromosome(

    return chromosome

Out algorithm select characteristics from one of the parents randomly.

Population Evolve Process

That’s are our primary process select the parents (The N Best and Y Random), create and evolve the next population of individuals based on selected parents and apply some mutations.

def evolve_population(population):

    # Clear Graphs from Keras e TensorFlow

    # Select N Best Candidates + Y Random Candidates. Kill the Rest of Chromosomes
    parents = []
    parents.extend(population[0:BEST_CANDIDATES_COUNT])  # N Best Candidates
    for rn in range(RANDOM_CANDIDATES_COUNT):
        parents.append(population[random.randint(0, POPULATION_SIZE - 1)])  # Y Random Candidate

    # Create New Population Through Crossover
    new_population = []

    # Fill Population with new Random Children with Mutation
    while len(new_population) < POPULATION_SIZE:
        parent_a = random.randint(0, len(parents) - 1)
        parent_b = random.randint(0, len(parents) - 1)
        while parent_a == parent_b:
            parent_b = random.randint(0, len(parents) - 1)


    return new_population


It’s time to apply mutation. That’s are very import into the process to ensure that we have some chance to evolve and keep variations from the parent chromosomes definition. Otherwise, our algorithm are unable to find new combinations and other alternatives to the problem they are trying to resolve.

def mutate_chromosome(chromosome):

    # Apply Mutation on Optimizer
    if random.random() <= OPTIMIZER_MUTATION_PROBABILITY:
        chromosome.optimizer = MODEL_OPTIMIZER[random.randint(0, len(MODEL_OPTIMIZER)-1)]

    # Apply Mutation on Hidden Layer Size
    if random.random() <= HIDDEN_LAYER_MUTATION_PROBABILITY:

        new_hl_size = HIDDEN_LAYER_COUNT[random.randint(0, len(HIDDEN_LAYER_COUNT)-1)]

        # Check if Need to Expand or Reduce Layer Count
        if new_hl_size > len(chromosome.layer_layout):

            # Increase Layer Count
            while len(chromosome.layer_layout) < new_hl_size:

        elif new_hl_size < len(chromosome.layer_layout):

            # Reduce Layers Count
            chromosome.layer_layout = chromosome.layer_layout[0: new_hl_size]

            pass  # Do not Change Layer Size

    return chromosome

Access the code at

I hope I explained in a simple way. Until the next article!

Leave a Reply

Your email address will not be published. Required fields are marked *