Cooling Cube

Should you use tensor parallel for your inference setup?
How many GPUs? Does DFlash break scaling?
Get the answer before you commit hardware.

01
Install
One pip command. No dependencies. Runs locally.
02
Pass your setup
GPU names, model size, draft length if using DFlash or MTP.
03
Get the verdict
Predicted speedup, recommendation, optimal weight split.
install
pip install coolingcube-tp
usage
from coolingcube_tp import analyse_tp result = analyse_tp( gpus=["rtx5090", "rtx_pro_6000"], model_params_b=32, dflash_draft=8, # 0 if not using DFlash or MTP interconnect="pcie" # or "nvlink", "roce" ) print(result["speedup"]) # 0.74x print(result["verdict"]) print(result["recommendation"]) print(result["proportional_split"])
quick reference — Qwen3.6 27B
2x RTX4090 no DFlash → 1.81x productive 2x RTX4090 DFlash 8 → 1.02x marginal 2x RTX5090 DFlash 8 → 0.74x structural waste 4x RTX4090 no DFlash → 3.29x productive 2x H100 NVLink DFlash 8 → 1.27x marginal 4x H100 NVLink no DFlash → 3.55x productive
TP Waste Benchmark → supported GPUs
rtx5090rtx4090rtx4080super rtx4080rtx4070tirtx3090 rtx3090tirtx3080tirtx_pro_6000 rtx6000adartxa6000h100_sxm h100_pciea100_sxma100_pcie a40v100_sxm
interconnect
pciedefault — consumer and cloud GPU setups
nvlinkH100 SXM, A100 SXM, workstation NVLink pairs
rocemulti-node RoCE fabric

Is your quantized model safe to deploy for coding, math, reasoning or chat?
Paste your KLD and top-1 measurements and get a per-task verdict.

01
Measure your quant
Run eval/model_diff.py from ExLlamaV3 to get KLD and top-1 vs BF16 base.
02
Pass the numbers
KLD, top-1 agreement, target task, and optionally bitwidth and model size.
03
Get the verdict
Feasible, marginal, or below quality floor — per task, with bitwidth recommendation.
install
pip install coolingcube-q
usage
from coolingcube_q import analyse_quant result = analyse_quant( kld=0.0056, # from eval/model_diff.py top1_agreement=0.9594, task="coding", # coding / math / reasoning / chat / general bpw=5.05, model_params_b=27 ) print(result["verdict"]) # Feasible — safe to deploy print(result["recommendation"])
check all tasks at once
from coolingcube_q import analyse_quant, list_tasks for task in list_tasks(): r = analyse_quant(kld=0.0056, top1_agreement=0.9594, task=task) print(task, r["verdict"])
example — Step3.7 27B at 5.05bpw
codingFeasible — safe to deploy
mathMarginal — test before deploying
reasoningMarginal — test before deploying
chatFeasible — safe to deploy
generalFeasible — safe to deploy
get measurements from ExLlamaV3
# KLD and top-1 vs BF16 base python eval/model_diff.py -ma ./quantized_model -mb ./bf16_model -r 5 # Perplexity across bitrates python eval/compare_q.py --help
tasks
codingcode generation, agentic coding — most demanding
mathmathematical reasoning, step-by-step problems
reasoningthinking models, chain of thought
chatconversation, instruction following — most forgiving
generalmixed use — balanced threshold

Cooling Cube computes start-time offsets for DDP training.
Paste your per-worker timing logs and receive an optimized schedule.
Hardware faults are flagged automatically.

01
Drop your data
Paste per-worker timing logs as JSON. Keys are rank numbers, values are step times in microseconds.
02
We optimize
Computes start-time offsets and classifies the problem — scheduling issue or hardware fault.
03
You get the schedule
Optimized offsets as JSON. Hardware faults identified by rank.
timing logs (json)
Result will appear here.
pip package
pip install coolingcube
collect timing data from a PyTorch DDP job
import time, json import torch.distributed as dist # add inside your training loop, after backward() t = int(time.perf_counter() * 1_000_000) all_times = [None] * dist.get_world_size() dist.all_gather_object(all_times, t) if dist.get_rank() == 0: t0 = min(all_times) print(json.dumps( {str(i): all_times[i] - t0 for i in range(len(all_times))} ))