You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
@1bcMax assume that there are solutions that can be solved with a 3-4B model, or 7-8B model, or 12-16B mode, or 22-32B model etc... https://github.com/Pavelevich/llm-checker
I am kinda trying to find if there are linear attention models <12B that would be good for local quick-work, and then something like GPT-OSS or Nemotron or Ring-Mini-Linear or Kimi-Linear that can do something longer length. Maybe Qwen3-Next for the hard-to-reach items (but maybe REAM compressed models with MTP for a little speedup).
Cross-check examples for accelerating local models to match OpenRouter and other places (reducing cost and dependency) waybarrios/vllm-mlx#45 (comment)
@1bcMax I think ollama is still useful, and whats keeping me from using this project. I like the idea of being able to use local models or cloud models. Their cloud models include basically all of the non-foundational models in the model list you support, but they dont process queries in china.
Its also pretty hard to fill up their $20/month membership using heavy models.
@1bcMax : Absolutely - Ollama support gets us selfhosted. As the smaller models get better, even being able to offload say the "SIMPLE" and "MEDIUM" tiers could help. Or go in reverse -- use Cloud for the first (3) tiers, and run the expensive REASONING tier on a local model.
Activity
TomLucidor commentedon Feb 16, 2026
@1bcMax assume that there are solutions that can be solved with a 3-4B model, or 7-8B model, or 12-16B mode, or 22-32B model etc... https://github.com/Pavelevich/llm-checker
I am kinda trying to find if there are linear attention models <12B that would be good for local quick-work, and then something like GPT-OSS or Nemotron or Ring-Mini-Linear or Kimi-Linear that can do something longer length. Maybe Qwen3-Next for the hard-to-reach items (but maybe REAM compressed models with MTP for a little speedup).
Cross-check examples for accelerating local models to match OpenRouter and other places (reducing cost and dependency) waybarrios/vllm-mlx#45 (comment)
DoWhileGeek commentedon Feb 22, 2026
@1bcMax I think ollama is still useful, and whats keeping me from using this project. I like the idea of being able to use local models or cloud models. Their cloud models include basically all of the non-foundational models in the model list you support, but they dont process queries in china.
Its also pretty hard to fill up their $20/month membership using heavy models.
FirebirdRender commentedon Apr 2, 2026
@1bcMax : Absolutely - Ollama support gets us selfhosted. As the smaller models get better, even being able to offload say the "SIMPLE" and "MEDIUM" tiers could help. Or go in reverse -- use Cloud for the first (3) tiers, and run the expensive REASONING tier on a local model.
+1 for this request.