autodl

Official website address of autodl, autodl computing cloud GPT rental platform, Stable Diffusion unlimited rendering

What is autodl computing cloud?

Personal AI computing cloud: Flexible, easy to use, cost-effective, larger and more comprehensive professional AI computing cluster, start computing rental immediately

autodl official website:https://www.autodl.com/

autodl computing cloud pricing

"AutoDL"An excellent domesticGPU rental platformproviding professional GPU rental services, instant billing, stable and easy to use, 7x24-hour service. The corresponding GPU's price is as follows:

autodl computing cloud advantages

Its advantages are

• billing by the secondTurn on the computer when using, turn it off when not using, and there is no charge when it is turned off
• Simple operationUser-friendly for beginners, equipped with one-click dummy start package, ready to use after installation
• Configurable as neededConfigure GPU according to your own needs, and even the strongest GPU can be affordable
• Multi-platform operationNo longer local operation, as long as there is a browser, you can use it on mobile phones, computers, pads, smart TVs, and any other entry points

GPU selection

How to troubleshoot performance bottlenecks, refer to the document. In addition, please note that cards with Ampere architecture such as 3060, 3090, 3080Ti, A4000, A5000, A40, A100, A5000 requirecuda11.1or higher to use (TitanXp, 1080Ti, 2080Ti, P40, V100 have no requirements), please use a higher version of the framework.

The mechanism for AutoDL platform to allocate GPU, CPU, and memory is: proportionally allocate CPU and memory according to the number of rented GPUs, and the CPU and memory displayed in the computing power market are the CPU and memory allocated per GPU. If you rent two GPUs, then CPU and memory are x2. In addition, GPUs are not shared, and each instance has exclusive access to the GPU.

I. Select CPU

CPU is very important! Although the CPU does not directly participate in deep learning model calculations, the CPU needs to provide data processing capabilities greater than the model training throughput. For example, the throughput of a 8-card NVIDIA V100 DGX server training ResNet-50 ImageNet image classification reaches 8000 images/second, while the DGX2 server with 16-card V100 does not reach twice the throughput, indicating that the CPU of this DGX2 server has become a performance bottleneck.

We usually allocate a fixed number of CPU logical cores per GPU. Ideally, the model calculation throughput should increase linearly with the number of GPUs, and the allocation of a reasonable number of CPU logical cores for a single GPU can be directly linearly extended to multiple GPUs. The computing power instances provided by the AutoDL platform offer various CPU allocation specifications. Each GPU should be equipped with at least 4~8-core CPU to meet the asynchronous data reading of multi-threading. Allocating more cores usually does not bring significant benefits, and the data reading bottleneck at this time usually originates from the Python multi-process switching and data communication overhead (such as using PyTorch DataLoader). So how to save money and overcome the data reading bottleneck? Why not try the NVIDIA DALI data reading acceleration library written in C++ and CUDA on the AutoDL platform? In our tests, the data reading capability of a single-core CPU instance exceeded that of an 8-core instance based on Python, truly ensuring the safety of model training.

High-performance CPU choices in AutoDL include:

1. Inner Mongolia A Area A5000 / 3090 / A40 using AMD EPYC 7543 CPU
2. Inner Mongolia A Area A100 using AMD EPYC 7763 CPU
3. Beijing A Area 3090 using Intel(R) Xeon(R) Gold 6330 or AMD EPYC 7642 CPU
4. Shenzhen A Area 3090 using Intel(R) Xeon(R) Gold 6330

The CPU of the server is generally not as high as the desktop CPU's frequency, but the core count is more. Therefore, after switching from desktop CPU to server CPU, you need to fully utilize the performance of multi-core, otherwise you cannot give full play to the performance of the server CPU. How to utilizePlease click

II. Select GPU

There are many types of GPUs provided on the AutoDL platform. We roughly divide them into five categories according to GPU architecture:

1. NVIDIA Pascal architecture GPU, such as TitanXp, GTX 10 series, etc. This type of GPU lacks low-precision hardware acceleration capabilities but has moderate single-precision computing power. Due to its low price, it is suitable for practicing training small models (such as Cifar10) or debugging model code.
2. NVIDIA Volta/Turing architecture GPU, such as GTX 20 series, Tesla V100, etc. This type of GPU is equipped with TensorCore specifically designed for low-precision (int8/float16) computing acceleration, but the single-precision computing power is not significantly improved compared to the previous generation. We recommend enabling mixed-precision training of the deep learning framework on the instance to accelerate model calculation. Compared to single-precision training, mixed-precision training usually provides more than 2 times the training acceleration.
3. NVIDIA Ampere architecture GPU, such as GTX 30 series, Tesla A40/A100, etc. This type of GPU is equipped with the third-generation TensorCore. Compared to the previous generation, it supports TensorFloat32 format and can directly accelerate single-precision training (PyTorch has already enabled it by default). However, we still recommend using ultra-high-performance float16 half-precision training models to achieve more significant performance improvement than the previous generation of GPUs.
4. Cambricon MLU 200 series acceleration card. Currently not supported for model training. Model inference using this series of acceleration cards requires quantization to int8 for calculation. And it is necessary to install the deep learning framework compatible with Cambricon MLU.
5. Huawei Ascend series acceleration card. Supports model training and inference. However, it is necessary to install the MindSpore framework for calculation.

The selection of GPU models is not difficult. For commonly used deep learning models, the performance of GPU training models can be roughly calculated according to the computing power of the GPU corresponding to the precision. The AutoDL platform labels and ranks the computing power of each type of GPU model, making it convenient for everyone to choose the suitable GPU.

The number of GPU choices is related to the training task. Generally, we believe that the training of a model should be completed within 24 hours, so that the improved model can be trained the next day. The following are some suggestions for choosing multiple GPUs:

• 1 GPU. Suitable for some small dataset training tasks, such as Pascal VOC.
• 2 GPUs. The same as a single GPU, but you can run two sets of parameters at the same time or increase the Batchsize.
• 4 GPUs. Suitable for some medium-sized dataset training tasks, such as MS COCO.
• 8 GPUs. The classic configuration that has always been passed down! Suitable for various training tasks and very convenient to replicate paper results.
• I need more! Used for training large parameter models, large-scale parameter adjustment, or completing model training as quickly as possible.

III. Select memory

Generally, sufficient memory does not affect performance, but due to the stricter upper limit restrictions on memory usage of AutoDL instances compared to local computers (local computers with insufficient memory will use hard disk virtual memory, affecting speed), for example, if the allocated memory of the rented instance is 64GB, and the program needs to use 64.1GB during training, the process exceeding the limit at this moment will be killed by the system, causing the program to interrupt. Therefore, if there is a high requirement for memory capacity, please choose a host with more allocated memory or rent multiple GPU instances. If you are unsure about memory usage, you can observe the memory usage status in the instance monitoring.