Running DeepSeek R1 on Hyperbolic GPU

Running large-scale models like DeepSeek R1 requires substantial GPU power, but AWS isn’t your only option. With Hyperbolic, you can rent high-performance GPUs on demand for deep learning and AI inference. In this guide, I’ll walk you through the setup process, including installing Ollama, and how to run DeepSeek efficiently.

Step 1: Renting a GPU Instance on Hyperbolic

1. Sign Up and Log In

Create an account at Hyperbolic and log in to your dashboard. Have at least $5 in your account.

2. Set Your SSH Key

Navigate to Settings and add your SSH key. This will allow you to securely connect to the machine once it boots.

3. Rent a GPU

• Go to the Rent GPU tab and choose a GPU that fits your workload. For DeepSeek, a minimum of an NVIDIA A100 or RTX 3090 is recommended.
• Select the OS as nvidia/cuda:12.4.1-cudnn-devel-ubuntu22.04, which comes with CUDA pre-installed.

4. Launch the Instance

• Confirm your configuration and launch the instance. It takes about 3-4 minutes to boot up.
• Once ready, you’ll be given an SSH command to connect to your instance, like:

ssh ubuntu@inquisitive-thyme-ladybug.1.cricket.hyperbolic.xyz -p 31095

Copy this command and run it in your terminal to connect.

Step 2: Setting Up the Environment

Since this is a bare machine, we need to install a few dependencies before running Ollama and DeepSeek.

1. Install curl and systemctl:
   Update the package list and install the necessary utilities:

   sudo apt update  
   sudo apt install -y curl systemctl  
   

2. Install Ollama:
   Install Ollama using the official script:

   curl -fsSL https://ollama.com/install.sh | sh  
   

3. Run the DeepSeek Model:
   Start the DeepSeek model using Ollama:

   ollama run deepseek-r1:7b  
   

   This will initiate the DeepSeek model, allowing you to interact directly through your terminal.

Note: The size of the DeepSeek model affects the amount of RAM needed

• deepseek-r1:70b requires ~35 GB of RAM
• deepseek-r1:671b requires ~412 GB of RAM

Step 3: Using DeepSeek for AI Inference

Once the model is running, you can start using it for real-time AI inference or data analysis directly via the terminal. Ollama will handle the rest, including model optimization and runtime configuration.

Why Choose Hyperbolic for Deep Learning?

Hyperbolic offers:

• Cost-effective GPU rentals compared to major cloud providers
• High-performance GPUs like NVIDIA A100 and RTX 3090
• Flexible pay-as-you-go pricing
• Pre-installed CUDA environments to simplify setup

By leveraging Hyperbolic GPUs, you can save on costs while scaling your AI experiments and deep learning models efficiently.

Conclusion

With Hyperbolic’s GPU rental service and Ollama’s easy-to-use interface, running DeepSeek R1 becomes straightforward and efficient. Follow this setup to maximize performance, avoid cloud overhead, and dive into high-powered AI computing.

Happy computing!