Picking the right GPU for an AI workload is the difference between paying for idle silicon and shipping a model on time — here is how the RTX 4090, A100 and H100 actually compare when you rent them.
Renting a GPU server means you get datacenter-grade compute without the capital cost, the power bills, or the 40-week lead times on datacenter cards. The hard part is matching the card to the job.
Buy too small and your model will not fit in VRAM. Buy too big and you are paying H100 rates to run a 7B chatbot that a 4090 would serve fine. This guide breaks down where each card wins.
The three cards at a glance
RTX 4090
Consumer Ada Lovelace, 24GB GDDR6X. The price/performance king for inference and small fine-tunes. No NVLink, no ECC.
A100
Datacenter Ampere, 40GB or 80GB HBM2e. NVLink, MIG partitioning and ECC. The proven workhorse for training.
H100
Datacenter Hopper, 80GB HBM3. A Transformer Engine with native FP8 and huge memory bandwidth. Built for large-model training and high-throughput LLM serving.
VRAM is the first question, not the last
Before you look at raw speed, ask whether your model even fits. If the weights, activations and KV cache do not fit in VRAM, no amount of compute helps — you are stuck offloading to system RAM at a brutal penalty.
- 24GB (RTX 4090): comfortable for 7B-13B inference, quantized 30B models, and LoRA fine-tuning of smaller models.
- 40GB (A100): full-precision 13B, quantized 70B inference, and mid-size training runs.
- 80GB (A100/H100): 70B inference in reduced precision, larger fine-tunes, and multi-GPU training as a single node.
- Multi-GPU: frontier training and 70B+ full fine-tuning need several cards linked over NVLink or a fast fabric.
Rule of thumb: budget roughly 2GB of VRAM per billion parameters at FP16 for inference, plus headroom for the KV cache. Quantization (INT8/INT4) cuts that substantially but can cost accuracy.
Precision: where the H100 pulls ahead
The RTX 4090 and H100 both support FP8, but the H100's Transformer Engine manages FP8 scaling automatically across a model's layers. That is a real throughput advantage for transformer training and serving, not a spec-sheet gimmick.
The A100 predates FP8 and tops out at TF32 and BF16. It is still excellent for training — just expect the H100 to finish the same run meaningfully faster on transformer architectures.
Match the card to the job
Inference / serving
RTX 4090. Best tokens-per-dollar for models up to ~13B. Scale horizontally with more 4090s rather than one giant card.
Fine-tuning (LoRA / small)
RTX 4090 or A100 40GB. LoRA on a 7B-13B model runs happily on a single 4090.
Full fine-tuning / mid training
A100 80GB. ECC and NVLink matter once runs take hours and multi-GPU scaling starts.
Large-model training
H100. FP8 and bandwidth cut wall-clock time on transformer training, which is where rental hours add up fastest.
High-throughput LLM API
H100. When you are serving thousands of concurrent requests, the memory bandwidth and FP8 throughput pay for themselves.
Batch / offline jobs
RTX 4090. If latency does not matter, cheap consumer cards run overnight for a fraction of the cost.
The most common mistake: renting an H100 to run inference on a 7B model. You will use a fraction of its capability and pay several times the 4090 rate for it.
Cost math that actually matters
Hourly rate is the headline, but throughput-per-dollar is what you should optimize. A card that is twice as expensive but three times faster on your workload is cheaper per finished job.
- 1
Benchmark on your workload
Run your real model and batch size, not a generic benchmark. Measure tokens/sec or samples/sec, not TFLOPS.
- 2
Divide by hourly rate
Compute throughput-per-dollar for each card option. This reorders the ranking more often than people expect.
- 3
Account for idle time
If your GPU sits idle between jobs, a cheaper card or an hourly top-up model beats a fast card you are not saturating.
- 4
Factor egress and storage
Moving datasets and checkpoints in and out has a cost. Unmetered bandwidth removes that variable entirely.
At ChainVPS the GPU fleet is prepaid from a crypto top-up balance with unmetered bandwidth, so shifting a 200GB dataset onto the box costs nothing extra — you pay for the GPU hours, not the transfer. If you are sizing a build, the /gpu-server page lists the current RTX 4090, A100 and H100 tiers and locations.
Privacy considerations for AI workloads
Training data and model weights are sensitive assets. Where they physically sit, and who can subpoena the host, is part of the engineering decision — not an afterthought.
- Choose a jurisdiction deliberately. Privacy-tier locations (NL, CH, RO, IS, MD, LU) sit outside the most aggressive data-request regimes.
- Prefer hosts that do not require identity documents to spin up compute — no KYC means no dataset of your projects tied to a legal name.
- Pay from a prepaid balance rather than a recurring card charge, so there is no payment trail linking the workload to you.
- Keep checkpoints encrypted at rest and pull them off the box when a run finishes.
ChainVPS GPU servers require no identity verification and are paid from a prepaid balance topped up with any of 21 coins including Monero, which keeps your research separate from your identity. See the offshore GPU options on the /gpu-server page.
A quick decision shortcut
Is an RTX 4090 really usable for serious AI work?
Yes — for inference and small fine-tuning it offers the best tokens-per-dollar of the three. Its limits are the 24GB VRAM ceiling and the lack of NVLink and ECC, which matter for large multi-GPU training runs but not for serving a 7B-13B model.
When is the H100 actually worth its premium?
When your workload is transformer training at scale or high-throughput LLM serving. Its FP8 Transformer Engine and memory bandwidth cut wall-clock time enough that the higher hourly rate produces a lower cost per finished job. For light inference it is overkill.
Can I rent GPU servers offshore and pay in crypto?
Yes. ChainVPS GPU tiers require no KYC and are paid from a prepaid balance funded with 21 coins including Monero. You top up, deploy, and your workload is never tied to a legal identity.
Do I need NVLink for my project?
Only if you are training a model too large for a single card, where GPUs must exchange gradients at high speed. Single-GPU inference and LoRA fine-tuning do not use it, which is why the 4090's lack of NVLink rarely matters for those jobs.
