Rendered at 02:36:15 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
yellowapple 4 minutes ago [-]
> For the setup price alone of a low-end home lab1, you could buy several years of a paid subscription to one of the AI providers. The power costs would come out to around $50-$300 per month, depending on how much inference you’re running: again, the price of a couple more paid subscriptions.
Okay, but I already have multiple computers capable of running local models with acceptable performance, so for me the cost is $0. I suspect that's true for most people of sufficient technical inclination to be interested in and capable of running models on their local machines.
Also, no, the monthly electricity price of even my power-hungriest machines ain't anywhere close to that figure. Hell, at the high end that's more than my power bill for my whole household.
qudat 41 minutes ago [-]
> Or maybe models get so good that a 30B model is genuinely smart enough to do everything, so nobody really needs a model like Opus or Sol unless they’re trying to solve the Reimann Hypothesis. I don’t really buy this. Models can do frontier mathematical work today while still being not smart enough to refactor large codebases as well as me, so it’s hard to imagine a world where I don’t just want to use the smartest model available.
Idk, I already don’t bother with Opus and stick with sonnet med. I really care more about speed. I use qwen3.6 27b for personal projects and I think it works pretty great.
So like the article mentions, if scaling stalls and small models get better it’s not impossible to imagine a convergence and hardware costs drop.
Having said that, self hosting will be a niche thing like it is today for other services.
bigbadfeline 39 minutes ago [-]
> No, local models will not win
Win what? Money, fame? That's not the point of small models, freedom is the point. What's this occultist obsession with "There shall be only one" monopolies? Why only one? That's so irrational and childish.
nailer 26 minutes ago [-]
Win as the dominant for of LLM interaction.
Every so often someone hypes local models as "Don't pay for Claude check out this local modal" and the local modal isn't at all equivalent.
That doesn't mean that local models are going anytwhere (as the article motes they're useful) but if you want the best we're stuck with expensive remote models right now.
yunwal 49 minutes ago [-]
I think local models will not be “niche” in the future in the same way personal computers are not “niche” just because most computing happens in the cloud. They have entirely different uses
darepublic 60 minutes ago [-]
What about that edge computing. Those cheap drones
Kim_Bruning 1 hours ago [-]
Is this the old mini vs micro argument again?
cyanydeez 59 minutes ago [-]
If they dont the gap between rich and poor will be unsustainable.
Okay, but I already have multiple computers capable of running local models with acceptable performance, so for me the cost is $0. I suspect that's true for most people of sufficient technical inclination to be interested in and capable of running models on their local machines.
Also, no, the monthly electricity price of even my power-hungriest machines ain't anywhere close to that figure. Hell, at the high end that's more than my power bill for my whole household.
Idk, I already don’t bother with Opus and stick with sonnet med. I really care more about speed. I use qwen3.6 27b for personal projects and I think it works pretty great.
So like the article mentions, if scaling stalls and small models get better it’s not impossible to imagine a convergence and hardware costs drop.
Having said that, self hosting will be a niche thing like it is today for other services.
Win what? Money, fame? That's not the point of small models, freedom is the point. What's this occultist obsession with "There shall be only one" monopolies? Why only one? That's so irrational and childish.
Every so often someone hypes local models as "Don't pay for Claude check out this local modal" and the local modal isn't at all equivalent.
That doesn't mean that local models are going anytwhere (as the article motes they're useful) but if you want the best we're stuck with expensive remote models right now.