Creating Facial Recognition Software with Machine Learning “Using Keras and OpenCV”

 Although the idea of using Machine Learning to perform Facial Recognition has been known for a long time, it is still quite useful to create this software yourself.

Concept:

The goal of this article is to create a binary classification facial recognition network, to allow access to one person, and not to another.

I will be using open-cv to access the built-in camera in the computer and use the haar-cascade classifiers to detect faces in the image. After collecting data, I will use a convolutional neural network to train the model on the data.

After that, when OpenCV detects the person who was given access, it will draw a square around his/her face, with the words “access verified” on the square.

The Code:

from numpy import unique
from numpy import argmax
import os
import cv2
from PIL import Image
import numpy as np
from tensorflow.keras import Sequential
from tensorflow.keras import optimizers
from tensorflow.keras.layers import Dense
from tensorflow.keras.layers import Conv2D
from tensorflow.keras.layers import MaxPooling2D
from tensorflow.keras.layers import Flatten
from tensorflow.keras.layers import Dropout
from tensorflow.keras.layers import BatchNormalization
from tensorflow.keras.layers import Dropout

Apart from the standard numpy and os libraries for access to data and data manipulation, I am also using open-cv and PIL for image processing.

def data_path():
    file = 'XXXXXXXX'
    os.chdir(file)
    files = os.listdir()
    files.remove('.DS_Store')
    return file,files

This is the function to access the data in your computer. For this function to work, change the variable file, to the relevant directory that your data is in.

def data_setup(files,file):
    pixels = [0]*len(files)
    answers = list()
    print(len(files))
    for i in range(len(files)):
        image = Image.open(files[i])
        pixels[i]= np.asarray(image)
        pixels[i] = pixels[i].astype('float32')
        pixels[i] /= 210.0
        if files[i][0] == 'm':
            answers.append(1)
        elif files[i][0] == 'n':
            answers.append(0)
    dataset = np.array(pixels)
    for i in range(len(dataset)):
        dataset[i] = dataset[i].reshape(320,320)
    return np.asarray(dataset),np.asarray(answers)

This function accesses the data, and reshapes all photos into 320 by 320 pictures. It then returns the X and y values, that is the data and the true values.

def train(data,answers):
    x_train = data
    y_train = answers
    x_train = np.array(x_train)
    x_train = x_train.reshape((x_train.shape[0], x_train.shape[1], x_train.shape[2], 1))
    print(x_train.shape)
    in_shape = x_train.shape[1:]
    print(len(data),len(answers))
    model = Sequential()
    model.add(Conv2D(10, (3,3), activation='relu', kernel_initializer='he_uniform', input_shape=in_shape))
    model.add(MaxPooling2D((2, 2)))
    model.add(Conv2D(10, (3,3), activation='relu', kernel_initializer='he_uniform', input_shape=in_shape))
    model.add(MaxPooling2D((2, 2)))
    model.add(Flatten())
    model.add(Dense(1,activation ='sigmoid'))
    model.compile(optimizer='adam', loss='binary_crossentropy',metrics = ['accuracy'])
    model.fit(x_train, y_train, epochs=100, batch_size=100, verbose = 2, validation_split = 0.33)
    return model

This script creates, compiles and trains the convolutional network. The loss used is binary cross entropy and the metric is accuracy, as we want the network to predict the correct face with high accuracy.

def face_recognition(model):
    dirx = 'XXXXXXXXXXXXXXXXXXXXXXXXXX'
    os.chdir(dirx)
    face_cascade = cv2.CascadeClassifier('cascades/data/haarcascade_frontalface_alt.xml')
    cap = cv2.VideoCapture(0)
    while True:
        ret,frame = cap.read()
        gray = cv2.cvtColor(frame,cv2.COLOR_BGR2GRAY)
        faces = face_cascade.detectMultiScale(gray,scaleFactor = 1.05, minNeighbors = 5)
        for (x,y,w,h) in faces:
            print('Face Detected')
            roi_gray = gray[y:y+h,x:x+w]
            roi_color = frame[y:y+h,x:x+w]
            roi_gray = roi_gray.astype('float32')
            roi_gray /= 210.0
            classify = cv2.resize(roi_gray,(320,320))
            if classify.shape == (320,320):
                classify = classify.reshape((1, classify.shape[0], classify.shape[1], 1))
                color = (255,0,0)
                stroke = 2
                end_cord_x = x+w
                end_cord_y = y+h
                pred = model.predict(classify)
                print(pred)
                if pred == 1:
                    cv2.rectangle(frame,(x,y),(end_cord_x,end_cord_y),color,stroke)
                    cv2.putText(frame, 'Access', (x, y-10), cv2.FONT_HERSHEY_SIMPLEX, 0.9, (36,255,12), 2)
                elif pred == 0:
                    cv2.rectangle(frame,(x,y),(end_cord_x,end_cord_y),color,stroke)
                    cv2.putText(frame, 'Denied Access', (x, y-10), cv2.FONT_HERSHEY_SIMPLEX, 0.9, (36,255,12), 2)
        if cv2.waitKey(20) & 0xFF == ord('q'):
            break
        cv2.imshow('frame',frame)

This script applies the model and makes predictions in real time, creating and labelling the boxes in which there are faces. For this function to work, you must once again, identify which directory the haar-cascade is in. You might need to take the folder from an external location to move it to the right directory.

file,files=data_path()
data,answers = data_setup(files,file)
model = train(data,answers)
face_recognition(model)

This final part of the program operates all the functions and starts the real-time facial recognition.

How you can improve my program:

When I write programs, I always share a strong framework, where more complex features can be added. Here are some things you can do to increase the functionality of the program:

• Multi-class classification

Instead of binary classification, try training the model to predict who the person is, when given a balanced set of pictures of each person.

• Add more haar-cascade classifiers

The haar-cascade that I used was a frontal face cascade, that would only detect frontal face photos. You could add more classifiers, that would make it work, no matter the angle of the camera.

Thank you for reading my article!