What LLM quantization works best for you? Q4_K_S or Q4_K_M

Quantization methods for LLM, including Q3_K_S, Q4_K_M, Q4_0, and Q8_0, are discussed. The K_M models are recommended for their balance between size and perplexity. Implementation details of llama.cpp for quantization are provided.