OVHcloud GPU benchmark - LlaVa

Go back to list

eval_count_mean eval_rate_mean
model llava llava:13b llava:13b-v1.6-vicuna-q8_0 llava:34b llava:34b-v1.6-q8_0 llava:7b-v1.6-mistral-q8_0 llava llava:13b llava:13b-v1.6-vicuna-q8_0 llava:34b llava:34b-v1.6-q8_0 llava:7b-v1.6-mistral-q8_0
provider_name flavor_name flavor__gpu_model
Amazon Web Services g5.xlarge NVIDIA A10G 72.000 74.000 92.667 95.917 92.000 78.468 47.888 30.177 21.006 51.518
p3.2xlarge Tesla V100-SXM2-16GB 73.000 63.125 87.667 85.167 102.687 67.589 20.319 71.446
Google Cloud a2-highgpu-1g NVIDIA A100-SXM4-40GB 80.667 82.667 97.000 117.567 81.016 38.250
a2-ultragpu-1g NVIDIA A100-SXM4-80GB 82.261 83.792 94.750 126.470 87.103 42.525
g2-standard-16 NVIDIA L4 87.333 101.333 97.667 49.961 27.921 11.715
n1-highmem-8 Intel Skylake Tesla V100 Tesla V100-SXM2-16GB 87.333 91.000 84.786 82.923 104.618 67.999 22.047 72.743
Microsoft Azure Standard_NC40ads_H100_v5 NVIDIA H100 NVL 108.000 128.000 99.692 120.000 75.667 79.000 188.615 129.355 103.871 65.704 45.540 158.181
Standard_NC6s_v3 Tesla V100-PCIE-16GB 87.333 91.000 51.853 38.179
OVHcloud H100-380 NVIDIA H100 PCIe 90.225 78.947 82.333 120.966 85.046 55.378
L4-90 NVIDIA L4 87.333 101.333 96.833 97.667 95.667 51.931 28.691 16.663 12.058 30.220
L40S-90 NVIDIA L40S 94.105 106.261 98.857 80.091 85.857 86.923 122.442 71.789 43.420 31.316 17.934 75.705
T1-45 Tesla V100-PCIE-16GB 87.667 85.286 20.808 71.191
T1-LE-45 Tesla V100-PCIE-16GB 87.333 91.000 99.917 65.651