What’s up everyone! Today we are going to talk about the different factors of graphics cards for deep learning workstation builds. We will go through what to look for and what can be compromised on for your particular use cases. Obviously, the best graphics card is going to be one that’s more expensive, but I’ll be discussing the ones that are the best for the price so that average graphics cards consumers. Disclaimer, right now they are nowhere near affordable though.
So, Graphics cards: let’s start with why are they important? Well, the way in which they are built makes them extremely efficient for matrix operations which is essentially what most of deep learning is. Doing massive volumes of operations on matrices really fast. They are more ideal than CPUs since they can transfer large amounts of information from memory at a time, whereas CPUs transfer smaller amounts but are able to do it quickly. Still the difference is that GPUs transfer much more data per second. The GPUs also have many different parallel processes that can do these large transfers simultaneously which further increases their advantage. The number of parallel processors is referred to as CUDA cores. Nvidia chips usually have several thousand of these on a single chip whereas a CPU might have 4-12 cores. So, they have vastly more cores which are slower but the bandwidth, or information that can be carried with each process is much higher than a CPU core. So, graphics cards end up being vastly more capable of handling the operations that are required for training neural networks.
Lots of graphics cards have huge amounts of dedicated VRAM as well. You need massive amounts of dedicated ram if you are training gigantic models. If you don’t think the models themselves that you’ll be training are going to exceed 10GB in size, then I would stick with my recommendation. If you need more than that, you’ll need to start looking at more advanced cards. A good rule of thumb is that if you don’t KNOW you’ll need more than 10GB of RAM for your GPU, then you will do just fine with 10GB of RAM.
As far as PCIe lanes, it’s pretty widely accepted that a single GPU is going to run best on 16 lanes, and this appears to be the standard with most cards that you see for sale.
The perfect graphics card for you will come down to exactly what you want to do with it. Some of the main factors to consider are dedicated VRAM, CUDA cores, and PCIe express lanes. Taking a look at all of these factors, the best cards available for the price are unfortunately probably the most in-demand since they are widely considered to be the best cards available. I don’t think anyone is going to be shocked by my recommendation. I would recommend Nvidia’s 3070 for someone starting out but knows they want to train some serious neural networks. The 3070 has 8GB of dedicated memory with 5888 CUDA cores. Even though this is the entry-level card in the 3000 series, it’s a beast. Check it out on Amazon here.
For more advanced machine learning, I’d go for the 3090 if you don’t want to fully commit to a two-card build. Anything more intense than that, you’ll want to start looking at doubling up on cards. Check out the 3090 on Amazon here.
The 3090 is much beefier but will break the bank quite a bit. It is more cost effective than buying two 3070s, however. With one 3090, you get 24GB of RAM which is three times the amount of RAM with 10496 CUDA cores. The prices that I was seeing online were roughly double the price going from the 3070 to 3090.
I would add though, that all of these recommendations are pretty serious business. My old GTX 1070 works fine for some of the exploratory model training that I was looking at. I can participate in Kaggle competitions and train on some medium sized data – it does help that I have 32 GB of RAM in my workstation. But the 1070 is not terrible, it can handle a lot, but can take a long time to train stuff relative to the cutting edge. I’m saying all this because, if you’re literally just starting in machine learning, I don’t think you need to go as hardcore as these recommendations, I’d check out something like Google Co-lab to get your feet wet to see if you even enjoy training models before diving in and buying a $3000 graphics card.
Unfortunately, right now, it is probably one of the hardest times in history to buy one of these latest graphics cards – people want to purchase them for everything: gaming, machine learning, mining cryptocurrency. The latest ones like the RTX 3000 series are sold out just about everywhere, however I have been seeing some trickle in on Amazon.
I hope that this guide gives you a little more background into what the right graphics card might be for you! Happy deep learning 🙂
Disclosure: AI Buzz Media is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to amazon.com