— Open-source large models are moving from 'usable' to 'runnable at home'—today, even the 2.8T-parameter Kimi K3 can be inferred on 8GB of RAM.
Open-source large models are being released at a rapid pace: Thinking Machines Lab unveiled Inkling-Small with 276B parameters, AMD released Instella-MoE with 16B parameters, and NVIDIA open-sourced the RL framework Molt. Community enthusiasts are pushing hardware to its limits, successfully running Kimi K3 on a CPU with 8GB of RAM and adding MTP support for DeepSeek-V4-Flash. Additionally, an interactive tutorial for Go 1.27 went live, and the Rust 2026 All-Hands meeting recap was published.
Thinking Machines Lab released Inkling-Small, an open-weight MoE model with 276B total parameters and 12B active parameters, roughly a quarter the size of the original Inkling. Trained on NVIDIA GB300 NVL72 systems, it natively supports text, image, and audio inference with a 1M-token context window, and weights are released under the Apache 2.0 license. Why it matters: its NVFP4 quantized version requires only 180GB of VRAM to run on a single B300 GPU, significantly lowering the deployment barrier for cutting-edge multimodal models—a major win for local inference and small-to-mid-sized teams.
A developer wrote a dedicated inference engine in C99, leveraging the MoE model's property of activating only a few experts per token to deploy the 1.56TB Kimi K3 on a single CPU with just 8GB of RAM, streaming expert parameters on demand from NVMe storage. Meanwhile, another open-source project, Mference, achieved running the DeepSeek-V4-Flash 284B model on 5.3GB of RAM. Why it matters: this proves the feasibility of inferring ultra-large MoE models on consumer hardware, shattering the conventional wisdom that 'large models require large VRAM' through extreme expert offloading and streaming strategies.
The community is generally amazed by this extreme optimization, but some point out it's more of a technical demo, with potentially very slow inference speeds and limited practical value.
NVIDIA's NeMo team open-sourced Molt, a PyTorch-native framework designed for agent reinforcement learning research. Its core RL code is only about 8.6K lines, building an asynchronous training loop by combining Ray, vLLM, and NeMo AutoModel, aiming to let researchers easily modify algorithms without diving into distributed backend details. Why it matters: agent RL research often suffers from slow iteration due to framework complexity; Molt's minimalist design could significantly reduce experimentation costs and accelerate algorithmic innovation in agent intelligence.
The VictoriaMetrics team released an interactive tutorial for Go 1.27, continuing the community tradition of showcasing new language feature changes through runnable examples. The tutorial is based on official release notes and Go source code. Why it matters: Go 1.27 introduces further generics expansion, a major shift for a developer community accustomed to concise syntax; the interactive tutorial helps developers quickly get up to speed and understand the real behavior of new features.
The community is broadly focused on Go 1.27's generics expansion, finding the syntax hard to read and adding cognitive load, though some see it as a long-awaited key feature.
ウェブは全体像、購読者にはあなた専用を——興味に合わせた AI 精選、プライベート RSS の統合、コミュニティの見解付きで毎朝配信。ずっと無料。
23 号配信 · 毎日150件超から読む価値ある30件に厳選
AMD released Instella-MoE-16B-A3B, a fully open MoE model with 16B total and 2.8B active parameters, using Gated MLA and FarSkip-Collective techniques, with weights released under the ResearchRAIL license.
llama.cpp has added MTP / DSpark support for DeepSeek V4 Flash, further boosting local inference performance.
Community reminds DeepSeek-V4-Flash-0731 users: avoid inserting system role messages mid-conversation, as this breaks the prompt cache with no real benefit.
The paper proposes AskChem, a claim-centric infrastructure that refines chemical paper retrieval units from whole papers to atomic claims with citations.
Syncular released an offline-first SQL sync tool, using local SQLite on the client with optimistic outbound and ordered commit logs for server-authoritative data sync.
The Fuse language was released, a statically typed pure functional language compiling to LLVM native code, supporting higher-kinded types and ad-hoc polymorphism.
The Kakehashi project attempts to run macOS binaries on Linux ARM, loading Mach-O and translating BSD syscalls via a user-space translation layer.
Commenters generally view the project's prospects favorably, seeing it as viable long-term, though some think the current approach is still early-stage and needs clarification on code independence from the Darling project.
The NixOS-DGX-Spark project was released, allowing Nix or full NixOS on DGX Spark, with USB images and system modules.
Bor v0.8.0 was released, an open-source system for centralized policy management on Linux desktops, adding Thunderbird, Edge for Business, and Firewalld policy support.
The Rust 2026 All-Hands recap was published, with 166 contributors discussing the language's future; the community is optimistic about Rust's prospects but concerned about compile speed and project governance.
Commenters generally view Rust's future applications favorably, but some believe compile speed could become a bottleneck, along with project maintenance and governance issues.
The F* language sparked discussion; the community recognizes its value in formal verification and C code migration but criticizes its documentation for lacking code examples and having a high entry barrier.
Commenters generally recognize F*'s value in migrating C code and formal verification, but some think its documentation lacks code examples and has a high entry barrier.
A user showcased a cluster built from 16 DGX Spark units, planned for locally running cutting-edge open models like DeepSeek v4 pro and Kimi K3.
Hey y'all. I'll be concise. TL;DR: DS V4-Flash-0731 @ UD-IQ2_M running fully in VRAM on 3xMI50s (90.9 GB model, 96 GB VRAM). Actual speed on llama-server is: - Text Generation: ~15-16 tokens/second stable. Never dipped below 14 tokens/second, even when the model was spitting out a 30K token long reply. - Prompt Processing: ~105-110 tokens/second or so. Dipped down on prompt processing of smaller token-length prompts, which is pretty typical of course. llama-server CLI logs, for those interested:
Ten advances in mathematics and theoretical computer science A few days ago it was Anthropic discovering cryptographic weaknesses with Claude using Mythos Preview, spending $100,000 on tokens and with prompts that included "again we are not looking for low hanging fruit, we want proper research to find genuinly hard findings." Now it's OpenAI's turn to flex. They set "an internal version of Astra, our next major model" on finding solutions to ten mathematical problems that "have seen no progress
Hi r/selfhosted , it's been a while! I thought I might share this release here as it contained a bunch of things that people have been asking about for, for a while! Before we get to 0.33, I want to remind people what Karakeep is. Karakeep ( ), is a selfhosted bookmark manager that's meant to be the place where you throw things in that you might need in the future, and it aims to help you with retrieving those things easily in the future. It has a blazingly fast full text search and opt-in LLM b
On the latest episode of Equity, we discuss why Sam Altman has calling on the industry to "pace the rate of AI development."
Here's my attempt at summarizing the various "open letters" about AI development that have been doing the rounds over the past few weeks simonwillison.net/2026/Aug/2/o...
New EU rules stipulate that people must be told when they’re interacting with AI or looking at AI-generated or -edited content, leading to fear of “disclosure fatigue.”
Open letters about AI development I wrote this summary of the past few weeks of open letters as a section of my sponsors-only newsletter but I've decided to share it here as well. Open Weights and American AI Leadership was shepherded by Microsoft, dated July 24th, and signed by 235 AI-adjacent companies including NVIDIA (see Jensen's first ever tweet ), Amazon, Y Combinator, The Linux Foundation, and (a later signer) OpenAI. It's clearly an argument designed to counter any instincts by the curr
In this tutorial, we build an advanced end-to-end time-series forecasting workflow with TimesFM 2.5. We begin by configuring the runtime, installing the required dependencies, detecting available hardware, and generating a realistic multi-store retail dataset with trend, seasonality, pricing, promotions, holidays, temperature effects, and random variation. We then load and compile the TimesFM 2.5 model, examine […] The post End-to-End Forecasting with TimesFM 2.5: Backtesting, Covariates, Anomal
Recent game world models can generate visually realistic and interactive environments conditioned on player actions. However, games are not defined by pixels alone; they are governed by explicit mechanics, namely state-dependent rules that control health reduction, skill activation, and game termination. These mechanics depend on precise internal states, such as health points, skill meters, and timers, which are tightly coupled with visual observations and determine how gameplay evolves. Without
I let Gemma4-31b run on my laptop for like almost a day using a heavily altered pi to do a deep dive on our beloved Llama tangentially related Subreddit, and this was the conclusion. Feels pretty accurate. Kind funny to let a small LLM loose and see what happens. Next target I'm trying to let it steal some benchmark answers from Huggingface, wish me luck.
Build Voice AI agents that actually speak India Discussion | Link
A 16.5-trillion-parameter model that contains nothing. This model is just a ████ you to the labs and companies who say that "haha I have the biggest model out there!". We the people with shitty laptops want to get a record. And I now have a record for a temporary amount of time of about 16.5 trillion parameters and use for them so its completly useless. What it demonstrates Hugging Face computes a repository's parameter count from safetensors headers alone — it sums prod(shape) per tensor and ne
According to rumors (spread by myself) Le Chaton FAT will be 26T-a3b and I AM READY for it. Let's be real, I can't afford that many 5060Ti, so I got 12x Gen 4 3.2 TB (two per card). This gives me about 60GBs bandwidth on 30TB. Added 256gb ddr4 just for kv cache, but I can also write KV-cache to the disks, these are high endurance drives. Are you ready for the next era of local inference? Jokes aside, this is what I use for my HF_HOME - model and dataset storage. I'm also setting up a few contain