Cookbook

Benchmarking FP8 with vLLM

EEthan Petersen
Aug 29, 2024
L40SLlama3vLLM

80 tokens/s with DBRX in 15 minutes

EEthan Petersen
Jul 19, 2024
L40SDBRXvLLM

Serving Llama3 at 3,500 tokens/s

EEthan Petersen
Jul 16, 2024
L40SLlama3Beginner