Can Councils do it, or is this a power restricted to central government? I haven’t heard this before! (View Post)

[cut] I need to cite this. (View Post)

Llama-3.1-8B-Instruct IIRC. The Q6_K quant is 6.2GB, so fits into a 8GB GPU with just enough space for a small KV cache & your display memory etc. I really wasn’t expecting it to write working Ha… (View Post)

Apologies! Llama-3.1 is the model - from Meta (Facebook). 8B is the size of the model - 8 Billion weights. Instruct means it’s been instruction-tuned after the initial training, so it interacts like … (View Post)

Or it’s simply too expensive for Apple to run at a profit right now. All the current LLM companies are being heavily subsidised by investors in one way or another. Microsoft paid for OpenAI’s NVidia … (View Post)