Check out the code on GitHub

Made with Python, OpenCV, Selenium

Description

This is a program that can analyze chess games on Chess.com in real-time. It can log in and play automatically. It can also be configured to only show its evaluation of the position and the best moves. It supports any UCI protocol chess engine, such as Stockfish.

Demo

The chess bot evaluates the three best moves and displays them with arrows. The opacity of the arrows is proportional to how good the move is. The score for the current player (in this case White) is displayed in the center of each arrow. Positive scores are good while negative scores are bad.

Those familiar with chess engines will recognize the score is in centipawns. For example, a score of 100 means that the current player will have an advantage equivalent to having an extra pawn.

The percentage chance of winning the game can be displayed in parentheses next to the score.

History

The beginning

This was the first major programming project I built. I initially was inspired by a browser extension that could analyze games on the fly on Chess.com. I had little knowledge about programming at the time and just knew the basic syntax of Python. The initial version was completed after weeks of Googling and hacky workarounds.

First, the browser extension would scrape the game's position and information from the current page. Then it would make an AJAX call to send the data to a free hosting site. I copied some PHP to create a PHP file on the server that would in turn relay the data to a Python script I had running. The script would then use a local chess engine to analyze the game's position and then send the results back to the browser extension.

Although it wasn't pretty, it still worked well so I was really excited and proud of what I had made.

A journey in computer vision

I wanted to clean up the mess I made and find a simpler way to fetch the current position and play the moves. What I ended up with may have been more complicated but it was a very interesting journey. I learned about OpenCV, a computer vision library, and wanted to use it to detect chess pieces. After reading a couple of guides on sites like pyimagesearch.com, I created a program that would

1. take a screenshot of the chessboard on Chess.com
2. slice the screenshot into images of squares
3. use OpenCV's template matching to detect similarities between each square image and manually labeled images of pieces
4. determine which piece the square image was most likely to be and reconstruct the board's position

I also used OpenCV's ability to show and edit images to show an image of the board's current position with arrows indicating the best moves. The initial version of my program was not very performant, requiring up to several seconds just to complete the template matching and reconstruct the board's position.

After profiling the code, I discovered most of the time was spent on template matching. I decided to employ some common optimizations such as shrinking the size of each square's image and short-circuiting the template matching if the similarity to the labeled image was above a threshold. Through these optimizations, I managed to reduce the time from over 10 seconds to around 0.3 seconds per move.

I later used Tensorflow to train a model to recognize chess pieces with even better accuracy than template matching.

Returning to the web

Although using OpenCV was very interesting and fun, I missed the ability to see the analysis in the browser itself. It was cumbersome to have to look at a different monitor to see the image with arrows generated by OpenCV.

I stumbled upon Selenium, which is an automated browser used for testing web applications. It integrated well with Python and allowed executing JavaScript on a page. I created a program to automatically go to Chess.com, sign me in, and then play games.

Using CSS and XPATH selectors in Selenium, I identified useful HTML elements and scraped data. It was a pain to parse some of the data, but once I got it working it was more reliable than OpenCV. By executing JavaScript on the page, I managed to draw arrows pointing out the best moves and other useful information such as the current position's evaluation.

Once again, I was unsatisfied with the performance, the program took up to 2 seconds to fetch the current board's position per move. Profiling the code again showed me a bottleneck with Selenium's functions for interacting with the page. After switching to executing JavaScript instead of using Selenium's functions, the speed improved from a couple of seconds per move to nearly 0.1 seconds per move.