2026-08-03

Hey y'all. I'll be concise. TL;DR: DS V4-Flash-0731 @ UD-IQ2_M running fully in VRAM on 3xMI50s (90.9 GB model, 96 GB VRAM). Actual speed on llama-server is: - Text Generation: ~15-16 tokens/second stable. Never dipped below 14 tokens/second, even when the model was spitting out a 30K token long reply. - Prompt Processing: ~105-110 tokens/second or so. Dipped down on prompt processing of smaller token-length prompts, which is pretty typical of course. llama-server CLI logs, for those interested:

Ten advances in mathematics and theoretical computer science A few days ago it was Anthropic discovering cryptographic weaknesses with Claude using Mythos Preview, spending $100,000 on tokens and with prompts that included "again we are not looking for low hanging fruit, we want proper research to find genuinly hard findings." Now it's OpenAI's turn to flex. They set "an internal version of Astra, our next major model" on finding solutions to ten mathematical problems that "have seen no progress

Hi r/selfhosted , it's been a while! I thought I might share this release here as it contained a bunch of things that people have been asking about for, for a while! Before we get to 0.33, I want to remind people what Karakeep is. Karakeep ( ), is a selfhosted bookmark manager that's meant to be the place where you throw things in that you might need in the future, and it aims to help you with retrieving those things easily in the future. It has a blazingly fast full text search and opt-in LLM b

On the latest episode of Equity, we discuss why Sam Altman has calling on the industry to "pace the rate of AI development."

Here's my attempt at summarizing the various "open letters" about AI development that have been doing the rounds over the past few weeks simonwillison.net/2026/Aug/2/o...

New EU rules stipulate that people must be told when they’re interacting with AI or looking at AI-generated or -edited content, leading to fear of “disclosure fatigue.”

Open letters about AI development I wrote this summary of the past few weeks of open letters as a section of my sponsors-only newsletter but I've decided to share it here as well. Open Weights and American AI Leadership was shepherded by Microsoft, dated July 24th, and signed by 235 AI-adjacent companies including NVIDIA (see Jensen's first ever tweet ), Amazon, Y Combinator, The Linux Foundation, and (a later signer) OpenAI. It's clearly an argument designed to counter any instincts by the curr

In this tutorial, we build an advanced end-to-end time-series forecasting workflow with TimesFM 2.5. We begin by configuring the runtime, installing the required dependencies, detecting available hardware, and generating a realistic multi-store retail dataset with trend, seasonality, pricing, promotions, holidays, temperature effects, and random variation. We then load and compile the TimesFM 2.5 model, examine […] The post End-to-End Forecasting with TimesFM 2.5: Backtesting, Covariates, Anomal

Recent game world models can generate visually realistic and interactive environments conditioned on player actions. However, games are not defined by pixels alone; they are governed by explicit mechanics, namely state-dependent rules that control health reduction, skill activation, and game termination. These mechanics depend on precise internal states, such as health points, skill meters, and timers, which are tightly coupled with visual observations and determine how gameplay evolves. Without

I let Gemma4-31b run on my laptop for like almost a day using a heavily altered pi to do a deep dive on our beloved Llama tangentially related Subreddit, and this was the conclusion. Feels pretty accurate. Kind funny to let a small LLM loose and see what happens. Next target I'm trying to let it steal some benchmark answers from Huggingface, wish me luck.

Build Voice AI agents that actually speak India Discussion | Link

A 16.5-trillion-parameter model that contains nothing. This model is just a ████ you to the labs and companies who say that "haha I have the biggest model out there!". We the people with shitty laptops want to get a record. And I now have a record for a temporary amount of time of about 16.5 trillion parameters and use for them so its completly useless. What it demonstrates Hugging Face computes a repository's parameter count from safetensors headers alone — it sums prod(shape) per tensor and ne

According to rumors (spread by myself) Le Chaton FAT will be 26T-a3b and I AM READY for it. Let's be real, I can't afford that many 5060Ti, so I got 12x Gen 4 3.2 TB (two per card). This gives me about 60GBs bandwidth on 30TB. Added 256gb ddr4 just for kv cache, but I can also write KV-cache to the disks, these are high endurance drives. Are you ready for the next era of local inference? Jokes aside, this is what I use for my HF_HOME - model and dataset storage. I'm also setting up a few contain