Identifying the Handwritten digits using TensorFlow in Python

By ALBINUS B

It allows the user to draw a number on the screen in Python using Pygame and have the program take a guess of which digit it is from the trained model. An augmented MNIST dataset is used.

This my version of the Python program solving the MNIST Digit data set. It allows the user to draw a number on the screen using Pygame and have the program take a guess of which digit it is from the trained model. This uses a basic neural network model. I've augmented the data in the MINST dataset to give more accurate results.

 

I've done a simple yet powerful code to identify/guess the handwritten digit using TensorFlow. The model is trained based on the MNIST dataset. The MNIST database is a large database of handwritten digits that is commonly used for training various image processing systems. The database is also widely used for training and testing in the field of machine learning. It was created by "re-mixing" the samples from NIST's original datasets. There are three phases in this project.

 

Phase 1: Training the dataset

A lot of other models have been published with 97-99% accuracy but they still guess the number wrong even when the digit is written clearly. To correct this error I’ve added data augmentation to my code to improve the results. Although the training accuracy is less compared to the primitive models it can guess the number more effectively on the test set or any new set. the ImageDataGenerator() function from tensorflow.keras.preprocessing.image has been used for the augmentation purpose. I’ve trained the model for 60 epochs. (HO-OH.model created by modelcreator.py)

Phase 2: Testing the model

As said earlier the model predicted 100% correct on the test set. In addition, it predicts better than the primitive model on the new test set. This proves the effectiveness of the algorithm. (modeltest.py)

 

Phase 3: Prediction

I’ve made a kind of a scribble pad where the user can write the number to get the model to predict it. This uses the Pygame module for this purpose. Once the user has completed writing the number the user could press any key to predict the number. The program could also be tuned to auto predict after drawing without inputting a key. (scribble.py)

The user interface is designed very friendly (Pygame window):

The left mouse button is used to write and the right mouse button could be used as an eraser.

The process of predicting starts when the user completes the drawing or when the keystroke is given. For the sake of simplicity, the Pygame window is saved as a PNG image and then passed to the Guess function.

Modules used:

1. TensorFlow
2. Keras
3. Tkinter
4. Matplotlib
5. OpenCV

 

Future use:

My algorithm could be further developed to identify numbers from a sheet of paper in real time or even alphabets. This could be very useful in converting handwriting to digital letters of numbers. Like letters make words this file could repeated to identify larger numbers.