Running Stockfish in the Cloud

tl;dr

I used Amazon Web Services (AWS) to run a server in the cloud to run the stockfish chess engine. On the largest and most expensive server, I saw performance of over 100 Million Nodes Per Second (MN/s) at a cost of $40/month with the limited use I have. More realistically, I could spend about $10/month for a server more powerful than I have, getting over 30 MN/s.

I may use this approach instead of having a dedicated chess engine server at my house.

Introduction

I use stockfish when I am analyzing my chess games AFTER I played them (I’m not a cheater!). I actually only use the engine for part of my analysis process, but that is a different topic.

I recently experimented with running a stockfish engine in the cloud using Amazon AWS. My thought was that instead of using my raspberry pi cluster or building my own chess computer, I will just start up a cloud server when I actually need it, and then shut it down when I am done. This will give me a high performing system that I will only pay for when I use it. Also, I do not have a chess server that is always on that is idle 95% of the day just waiting for me.

If you want something even more powerful, you can check out my experiment with a cloud cluster supercomputer that I got up to 700 MN/s without much work.

Use At Your Own Risk

I chose AWS because I had used it in the past. However, it is not the easiest to use. If you haven’t used AWS before, this might not be for you. The interface is powerful, but confusing, and the pricing is pretty complicated. But, if you are bold, you can do this.

The biggest risk here is that you will get charged more money than you expect. That has happened to me with AWS in the past for a different project. You may want to look into setting up spending alerts within AWS.

I’m not responsible if you rack up a huge AWS bill! You should be OK if you make sure you keep an eye on your spending each day. But be warned, the billing can be delayed by a day.

Getting Started: Launch A Cloud Server

I started in AWS with a EC2 image on a small instance type. My goal is not performance yet, it is just setting up a template I can use for future deployments.

I am going to use Amazon’s c5a server class which is aimed at computing-intensive work. They have many different “classes” of servers depending on the type of work you need the server to do. Each “class” then has many levels like large, xlarge, or 4xlarge.

Update Jan 2023 The latest generation of this server type are c7g which are ARM-based processors. If you are using this, when you go to build stockfish be sure to run make -j clean build ARCH=armv8

I used the Ubuntu image because that is what I know. You could try to do this with the standard AWS’s preferred “AWS Linux 2” instance type, which I think is based on CentOS, and it would work just as well but would need slightly different commands.

Just a couple of things to keep in mind as you are setting up the instance:

• You can use the defaults for almost all the options, but be sure to click through each page so you do not miss anything.
• I only used the minimum of 8 GB of EBS storage (like a virtual hard drive).
• Make sure to enable SSH access when setting up the security group
• Generate a keypair: I called it stockfish-keypair.pem. Be sure to save this on your computer someplace safe. I have mine under ~/.ssh/ but as long as it is secure and you know where it is you should be good.

Connect to the Server

Once everything was launched, I needed to connect to the running server. To find the IP address, look in the EC2 console and find the public IP address under the instance details. I’ll just use a fake address of 11.222.33.444 for everything here. Then you connect with:

ssh -i stockfish-keypair.pem ubuntu@11.222.33.444

The default username is ubuntu here because of the OS I started with. Also, on some systems you may need to change the permissions of the .pem file before you use it so it is more secure.

Setup the Server

Once I am on the server, I basically just need to build stockfish. I could have also done something easier like sudo apt-get install stockfish but I wanted the latest version.

Update Jan 2023 Another way to do this would be to use homebrew and then you can just call brew install stockfish.

Start with updating the server to get the latest packages:

sudo apt-get update
sudo apt-get upgrade

Next install the tools needed to compile/build Stockfish (or build anything for that matter):

sudo apt-get install build-essential

Now we can install Stockfish:

git clone https://github.com/official-stockfish/Stockfish.git
cd Stockfish/src
make net
make -j clean build ARCH=x86-64
sudo ln -s /home/ubuntu/Stockfish/src/stockfish /usr/bin/stockfish

Geeky Note: There are other ARCH settings you could try. The AWS c5a instance has an AMD Epyc chip so you could try things like x86-64-ssse3 or x86-64-avx2. I tried a few and the performance seemed about the same.

Running Stockfish remotely

At this point I have a server that I can connect to and run stockfish remotely:

ssh -i stockfish-keypair.pem ubuntu@11.222.33.444 stockfish

I am not sure what I expected, but I was honestly surprised when this worked.

Create an AMI

The next step was creating an AMI (Amazon Machine Image), which is like a template for all servers that I want to launch. Normally you would just launch a server, set it up however you want, and you are ready to go. But I wanted to deploy the same setup (a machine that has stockfish installed) to multiple servers. I wanted to do this so I could test it out on different types of hardware. I also did not have to worry about losing the changes if I terminated an instance (see below). But it is so easy to set up, that was not a huge concern.

To do this you create what Amazon calls an AMI. This can be done from the AWS console under your EC2 dashboard. Look for the menu for “images” to get started.

Starting / Stopping Instances

Now you can start/stop instances whenever you need them. You can do that with the AWS console. I also started writing a script to do this, which I’ll talk about in a future post.

You will pay for however long the server is up, so be sure to shut it down when you are done with it.

When you are done you can stop the instance and then it will be ready to start up quickly next time. It is like you just put your computer in “hibernate” mode.

You could also terminate the instance and then you will lose access to it. This is like throwing away your computer and then getting a brand new one next time you need it.

Launching new instances

Now I have an AMI (image / template) I can launch for any size server. I can launch a smaller one for less-intensive analysis. Or a massive one for GM-level analysis (that I likely won’t understand). You can launch a new instance from the AWS EC2 dashboard and just configure it to use the AMI you created earlier. Be sure to select the same security group that you created originally so you can use the same .pem file. You could also create a new security group each time if you wanted, but the downside is having to use a different .pem file each time.

Performance Results

Here are some of the server sizes I tried along with some performance results. To run the test I did something like this:

ssh -i stockfish-keypair.pem ubuntu@11.222.33.4443 "stockfish bench 64 16 20"

If you want to know more about the benchmark tool, I talk about it a bit in my post on cluster performance.

Here are the results in millions of nodes per second (MN/S):

There are other levels that I did not include here as well. There are also other server classes I could use, like the ARM-based chips.

I found a good middle ground to be the c5a.8xlarge with a hash of 128 and 96 threads.

And yes, the c5a.24xlarge can search over 100 million NPS. I’ve never used a server this large so I’m guessing I could tweak the benchmark settings to squeeze out a bit more performance. This type of server has 96 CPUs and 192 GB of RAM. It is also the most expensive…

Cost

There are a couple of costs associated here:

EBS

This is the “hard drive” for your server and you have to pay for it even when your server is stopped. It is pretty cheap, currently at $0.08 / GB / Month (pricing). For the minimum 8 GB I used that will be $0.64 per month. The good news is that this may fall into the AWS free tier. That “free tier” is a little tricky though, just be warned.

If you are going to need more storage, for example by wanting to have large Syzygy tablebases, then you will be paying more. If you are storing 100s of GBs of endgame tables, there are probably better storage options available in AWS.

EC2

This is where the bulk of the expense is, and will depend on how much you use the server. To find out how much you have spent, you can look at the AWS billing dashboard, though that seems to have a delay of about 1 day.

The cost is actually computed by how many seconds you use the server, with a minimum of 60 seconds.

For some reference here are some estimates for the costs of different size servers. Just for the sake of simplicity, I’ll say I will use the server for 30 minutes a day for 20 days in a month. That is the Cost/Month (Limited) column below. If I were to leave it on all the time, that is the Cost/Month (Always On):

Other Costs

You could opt for a static IP address or what AWS calls an elastic IP address for a small additional fee.

If you are transferring a huge amount of data, you may have to start paying data transfer costs. You get the first 1 GB free per month, which I think will be fine with stockfish, but I am not sure.

Summary

For simplicity, let’s say I can build myself a specialized chess computer for $1000 and it will get me around 30 MN/s performance. If I am only going to be using it for 30 minutes a day for 20 days a month (just as an example), then the server AWS costs would be about $12/month. It would take almost 7 years to reach that $1000 mark. The more you use the engine, the higher the costs, and the sooner you might be better off with your own server.

I’d be remiss if I did not mention that there are services like Chessify that provide these cloud-based engine services as well.

I’m providing this information so anyone can make their own cloud-based server. Maybe a bunch of us should go together in a co-op and share a server!

Next Steps

Once everything is setup, you can read about how I use cloud-based stockfish which includes a script to automatically start/stop the instance and how I use my SSHEngine proxy to connect to the remote server from something like Chessbase.

If you have any other thoughts or questions, let me know on twitter.