LOCAL LLM GATE

Cause: quantization too low (Q2/Q1) or the context was artificially squeezed; the default temperature may also be too high.

Fix: switch to q4_K_M/q5_K_M; set num_ctx (OLLAMA_CONTEXT_LENGTH) to 8192+; drop temperature to 0.6–0.8; for code tasks keep top_p near 0.9. Still broken? Try that model's original F16 once to determine whether the damage is “quantization” or “the data was always weak”.

Cause: it never actually landed on the card; driver or model layer was not recognized as GPU-capable.

Fix: check whether PROCESSOR in ollama ps says GPU; on Windows install the matching CUDA driver and use a supported backend; for llama.cpp check the -ngl layer count and offload in the logs. Don’t be fooled by “it started” — seeing is believing.

Cause: you are downloading the original FP16 rather than a quantization; or the context is pushed to the ceiling and KV blows up.

Fix: find a …-GGUF repository and pick Q4_K_M; start at -c 4096 and creep upward; if that is still not enough consider KV-cache quantization or --flash-attn. To do it in one shot, pick the model size that matches your VRAM tier.

Cause: the service isn't listening on 0.0.0.0; or traffic still goes through another proxy; or the browser blocked cross-origin.

Fix: confirm the endpoint answers with curl http://localhost:11434/v1/models; use the official web UI or enable the CORS switch; try another browser hitting 127.0.0.1 directly. Kill clash / system proxy and retry — that alone usually fixes it instantly.

Cause: weights + KV + runtime exceed physical VRAM, so the system falls back to RAM/disk thrashing.

Fix: drop one model tier or bit width; squeeze -c context; enable flash attention & KV quantization; close browser/AI processes competing for VRAM. If it still crashes, this model simply is not for this machine — downsizing is not shameful.

Cause: training corpora skew English-heavy; small parameter counts under-cover other languages.

Fix: for non-English work prefer the Qwen / GLM / InternLM / DeepSeek families and language-boosted variants; watch for chat/…zh in the model name; don't expect “full-strength academic-grade” output from a small model — when it is time to go 8B→14B, don't scrimp.

Cause: direct connections to huggingface.co are slow from some networks.

Fix: use a mirror via HF_ENDPOINT=https://hf-mirror.com; Ollama uses its own distribution channel; if needed run hf download with hf_transfer for multithreading; pull only the quantization file you need rather than a whole snapshot.

Cause: no concurrency or batching was configured, so every request monopolizes KV and VRAM.

Fix: add Continuous batching at the serving layer (Ollama / OpenWebUI have queues), cap max model len, spread across tensor-parallel (multi-GPU) or data-parallel (multiple instances); don't hand everyone a 128k context.

Origins: founded in California in 2015 by Musk, Altman, Sutskever, Brockman and others, positioned as “non-profit, caring about all humanity, open-source friendly”; the intent was to hedge against “for-profit giants locking AGI inside a few companies”. Named OpenAI, yet early weights were deliberately withheld on safety grounds (GPT-2’s “too dangerous to publish fully” is one footnote to its later controversies).

Milestones: in 2019 Microsoft’s multi-billion backing turned it into a “capped-profit” structure; in 2020 GPT-3 set off the “big doesn’t mean dumb” scaling law; late 2022 ChatGPT brought conversational AI to the public and is regarded as the gate to the golden age from 2023 on; in Nov 2023 the board abruptly fired CEO Altman and asked him back within four days — a public clash with the “AI safety” line; the following years ran GPT-4 / o-series reasoning / various flagships on a “closed high performance + paid API” main line, while in 2024–25 it released the locally deployable, open-weight gpt-oss-20b/120b (MIT/Apache — community-runnable), widely read as “forced to respond by the open-source camp”.

Neutral verdict: strengths — long the reference choice for “engineering integration + productisation + ecosystem”, with system prompts/tooling/evals that are de facto baselines; limits and criticism — transparency, data provenance and copyright, moderation boundaries, and military use plus safety-guardrail disputes have all made headlines; it still treats closed lock-in as its monetisation core, with open weights as a “fill-in”. Both sides have public sources; avoid single-sided emotion.

Live (2026-11 view): GPT- and o-series flagships remain API-only; what you can actually deploy locally is still the open-weight gpt-oss-20b · gpt-oss-120b (MoE; the 20B-class quant suits workstations) — as of 2026-11-01 OpenAI has shipped no new open weights (there is no gpt-oss-2), plus gpt-oss-safeguard-20b/120b for local safety classification. Sources and channels: openai.com; Hugging Face repo `openai/gpt-oss*`; Ollama `gpt-oss`: 20b / 120b.

Origins: founded in 2021 by former OpenAI executives including the Amodei siblings, with a stance clear from day one — putting interpretability and alignment first. The team uses “Constitutional AI”, constraining the model’s self-training with a set of values; that is its biggest differentiator from the mainstream benchmark-chasing labs.

Milestones: Claude iterated through Opus/Sonnet/Haiku tiers (a naming lineage still in use). It prioritised very long context and “code / research workflows” early, becoming a daily driver for many developers; by 2025–26 Claude also entered coding agents and some high-frequency government and military use, with the resulting “guardrails vs authorisation boundaries” debate appearing repeatedly in public reporting.

Neutral verdict: strengths — an alignment narrative plus a strong real-world reputation on code and long documents, and product language naturally close to developers; limits — entirely closed (no local weights), dependent on API/subscription monetisation, and “how tight is the safety limit, and is it applied correctly” has always been the same knife that cuts both ways.

Live (2026-11 view): API / app via Claude subscriptions; cannot be self-hosted locally (a fundamental dividing line from the open-source camp — settle this before choosing). Channels: anthropic.com / api.anthropic.com.

Origins: Google’s AI research long split between an “academic conference school” (Google Brain) and DeepMind’s deep-reinforcement-learning eccentrics; the two merged into Google DeepMind in 2023, wrapping the LLM product line as Gemini. Having wowed the public with AlphaGo (2016) and carrying the Transformer paper’s team DNA, it is arguably the “original shareholder” of the LLM era.

Milestones: the Gemini line spread via multimodality and a “rebuild every product” push (Workspace/search/agent tools such as Deep Research), iterating for several generations through 2024–2026 and holding the top of evaluations; it also shipped the open Gemma family (small and mid-size weights you can run locally) — a compromise footnote to the “open about technology but not fully open flagship” route.

Neutral verdict: strengths — deep platform integration, strong multimodal foundations, and compute plus a big-client ecosystem; limits — the Gemini flagship is closed and its services live mostly in its own ecosystem, making it “tweakable but uncontrollable” for third-party developers, and what it opens (Gemma, aimed at edge and small-mid tier) is far from flagship scale.

Live (2026-11 view): the flagship runs via Gemini API; what you can run locally is Gemma 4 (E2B/E4B/12B/26B-A4B/31B, all Apache-2.0, with the E tiers taking audio input), shipped as official QAT GGUF. Channels: deepmind.google / aistudio.google.com / Ollama “gemma”.

Origins: Meta (formerly Facebook) was not an obvious “model lab”, yet one Llama move in 2023 made it the engine of open-source LLMs. What it has long wanted to accumulate is “community + ecosystem + AI infrastructure say” — trading weights for long-term developer stickiness and a standard-setting position, the exact opposite of the “sell APIs” camp.

Milestones: Llama (research) → Llama 2 (2023, commerce relaxed) → Llama 3/3.x (2024–25, many sizes, ecosystem grind) → later iterations through 2026 continuing a hybrid stance of “open, while its own Meta AI stays closed”. If anyone rivals Google at turning open source into a traffic funnel, it is Meta.

Neutral verdict: strengths — relatively permissive licensing and the community/inference stack (llama.cpp/Ollama supported it earliest and most completely), making it the local crowd’s natural default; limits and controversy — licence terms have tightened version by version (commercial and MAU thresholds), the early “open ≠ freely commercial or attribution-free” boundary was pulled back and forth by the community, and Chinese ability usually trails Chinese-native models like Qwen/GLM.

Live (2026-11 view): after Llama 4 (2025-04) Meta’s open-weight line moved to the Muse brand — Muse-Glimmer-30B (Apache-2.0, natively multimodal) and Muse-Spark-Open-816B-A42B; there are no open Llama 5 weights. Channels: huggingface.co/meta-models.

Origins: Musk founded xAI in 2023; its flagship Grok was bound to the X (formerly Twitter) account ecosystem from birth, training while borrowing social data and traffic. The stated mission is “to understand the true nature of the universe”, though in practice it appears more often under the banner of “fewer limits + big data + competitive compute”.

Milestones: the first Grok got attention via “sharp tongue and fast access to X”, then shifted into a serious LLM arms race (including reasoning and multimodal mainlines), and has headlined both for mouthy outputs backfiring and for a reported military contract. By 2026 the commercial closed-source tone remains dominant, with little official local weight.

Neutral verdict: strengths — X ecosystem data and distribution plus heavy compute investment, with a differentiated, down-to-earth style; limits — maturity plus multimodal and long-document ability still trail the leaders, and “saying whatever it wants” is both a selling point and a liability for commercial compliance; local self-hosting is essentially unavailable, a hard constraint for many enterprise buyers.

Live (2026-11 view): used mainly through X / API; no official open local weights is the normal state — check the official page before planning a deployment. Channel: x.ai

Origins: it grew out of a quant-trading-backed research outfit and is seen by the industry as “the value-for-money efficiency camp of open source”: it first made “spend less compute, score high” a mainstream play with the V/R series, and in early 2025 R1’s open reasoning plus V3’s MoE architecture made “a Chinese lab can open weights and still top the charts” a headline.

Milestones: DeepSeek-R1 (2025, open reasoning line) and V3 (late 2024, MoE 671B-A37B) → the V3.1 family’s “thinking / direct answer hybrid” → to the V4 line (V4-Pro: 1.6T total / 49B active; V4-Flash: MoE, 284B total / 13B active; V4.1-Flash: 552B / 8–16B active, 1M-token context), continuing to prove that “sparse activation × long context” can fit large capacity into efficient local and edge deployments.

Neutral verdict: strengths — MIT-style (for most weights) open-source freedom, a strong reputation in Chinese general, long-document and reasoning work, and a stance of generously releasing research output that developers appreciate; limits — training and launch resources are tight, international compliance and some modality depth are less developed, and some “hit waves” see occasional queues. Check the official documentation for licence and stability before commercial use.

Live (2026-11 view): quantized tiers of the R1/V3.2 series run directly on a single machine or workstation; the V4 line (Pro 1.6T-A49B / Flash 284B-A13B / V4.1-Flash 552B-A8~16B, all MIT with 1M context) makes a million-token window the default, but Ollama ships :cloud tags only — running it locally means self-hosting vLLM; to try the V4 previews, look for official weights and quantized copies first (locally it needs a lot of RAM/VRAM). Channels: deepseek.com / github.com/deepseek-ai / the official HF repos (mind the version date stamps).

Origins: from Alibaba Cloud’s research line, the Qwen family achieved extremely high adoption by “open weights at almost every size from small to large + multilingual + strong native Chinese”; it is one of the most common defaults seen among local users in Chinese-speaking regions.

Milestones: Qwen (1.5/2.5, full coverage from 0.x to 72B) → Qwen-Coder (coding-specific, multiple sizes) → Qwen3 (0.6B–235B, including MoE, adding thinking tiers and tools) → continuing into 2026 with Qwen3.5/3.8 (3.8-27B etc., plus smaller, multimodal and “flash-next” forward-looking tiers); its cadence makes it one of the fastest-updating open lines for deployment.

Neutral verdict: strengths — full size coverage, generous licensing (mostly Apache-2.0 or its own permissive terms), mature tool calling and Chinese agent ecosystem, and the broadest compatibility stack (a familiar face in Ollama/LM Studio); limits — data and review follow domestic delivery policy (check first for some overseas or authority-sensitive deployments), and the strongest flagship versions are often not fully opened, so self-hosting requires weighing capability against licence.

Live (2026-11 view): locally prefer the 27B–35B tiers of Qwen3.5 / 3.6 / 3.8 (Apache-2.0, they fit a single 24 GB card), with 0.8B–9B covering edge and entry use and 122b-A10B and up reserved for multi-GPU rigs; note that the 2.4T tier of Qwen3.8 switched to the custom Qwen3.8-Max licence (weights public); channels: qwenlm.github.io / github.com/QwenLM / Ollama “qwen*”; licence and review terms follow its release announcements.

Origins: spun out of Tsinghua-linked research accumulation (the Zhipu/GLM line has a long academic lineage), it leans on a natural shape “strong in Chinese and in coding agents”. It moved from ChatGLM to the GLM line, and after 2025 began shipping on a dual track of “open core + a stronger commercial edition”.

Milestones: the GLM-4.x era already had Chinese agent and multimodal layouts; by 2026 the GLM-5.3 family (including GLM-5.3-Flash: roughly 320B total / 18B active, multimodal, 1M-token context, weights already open) pushed “long-horizon code and tool agents + vision in the coding loop” as selling points; community licensing (weights or MIT, see its repos) makes it a new locally-runnable face.

Neutral verdict: strengths — strong Chinese-native ability, a developer-community reputation for agents and tool calling, and a Flash tier whose “small activation, high throughput” fits personal workstations; limits — some top tiers are “commercial closed or licensed”, the long-tail ecosystem and international plugins still trail Qwen/Llama, and deployment likewise requires reading licence and review terms.

Live (2026-11 view): locally you can get new weights from the official repo or HF “zai-org/GLM-*”; the GLM-5.x line is around 753B with 1M context, but 5.3 moved to a custom licence (weights still public) — the MIT exits that stay local-friendly are GLM-4.7-Flash (30B-A3B, Q4 ≈ 19 GB) and GLM-5.3-Flash (320B-A18B, natively multimodal); channels: z.ai / github.com/zai-org / huggingface.