Pocster

Yes . But also I want local for stt real time . Can’t exactly do that cloud based and integrating into a local home assistant. Or at least not in any sane way and speed . Also lots of other things I wish to do that can only be done locally .

Oh I see lol ! Well with free chat I’d run out of credits , so I tried free Claude and run out of credits . I then upgraded Claude and run out . went back to chat + For 20 quid a month it’s exceptional I think . Our technical conversations can hit 50k tokens . So I was trying to use chat like a company might pay a proper subscription and use Claude . But I ain’t paying 200 bucks a month

What do you mean ? To build my watering system ? just chat . A bit of work to get it reliable with multi file projects ( it doesn’t reliably support git ) . Now . Me and chat design / spec . Local llm will write code / test . Another 2 local llms will deal with reasoning from voice commands to control / anticipate home assistant stuff

lol I’m just testing it out. So did simple “ how does it work “ tasks building up each time . chat and I spec / architect the real thing then just get local llm to do the code . So yeah I’ll be doing it that way . Don’t really need multiple agents etc . But I’m so surprised at the speed and quality of code when spec’d properly . But before this I got chat to write an entire esp32s3 project for me ( automated watering system - it’s on the forum ) . This though - wow . Not only are my demands far more ; but its capabilities are also .

Ooooo ! Forget code complete …. Used to be anticipate variable etc . Now write function - but tbh a bit crap - “ simplistic “ Rephrase the prompt and let it write the entire program ( just a test example ) then refactor it . Wow ! . Local speed surprised me as did code quality . Tomorrow setup a test local git and see what happens . Awesome stuff !

Sure . Just trying to get visual studio to connect to anything . Will need multiple models anyway for full Avalon

On qwen coder 30b getting a very respectable 80 tokens / sec . Estimate on ultra m5 same model might touch 200 !

Go big or don’t go in 😉

Of course not ! The m3 ultra 96gb will be the prototype!

Thing is @-rick- I was just playing with you a bit . Clearly demand will out strip supply for m5 ultra for sure I.e delivery times will slip like a dog . My intention is to order within minutes of it going live in the App Store . My ambitions probably do not need m5 with 256gb or more but it bugs me to buy “old” m3 ultra at near full whack when we all know new and similar priced is coming .

Oi (expletive deleted)ers ! This is my local llm thread ! Not political or supply issues . Get a (expletive deleted)ing room 👊🏻

M3 96gb arrived ! Now the work begins !!

They dropped the 512gb m3 because they couldn’t supply the bin in a sensible time frame due to ram shortages .They up’d price of ram increases also + of course Apple ram tax . With m5 I guess there might not be a 512gb option initially . Clearly Apple can weather it better than some suppliers but they are experiencing issues . Ram shortages are set to continue into 2027/28 . So Apple are suffering ram increase costs , ram shortage, high demand for high bin and a new version on route . So bit of a perfect storm tbh . https://www.macrumors.com/2026/04/06/mac-mini-and-mac-studio-long-shipping-delays/ or as chat would say …. You may be right that Apple sold through the amount of high-memory M3 Ultras it planned to build. But that still doesn’t prove the wider memory shortage is irrelevant, and it definitely doesn’t prove Apple will offer M5 substitutions to people waiting on M3 orders. So at the moment the solid bit is just this: high-memory Apple SKUs are hard to get. The rest is theory. Honestly though, he sounds like one of those people who wants to win the mechanism, not discuss the outcome.

Source ? lol ! Apple website - try ordering one . Try getting one m3 / m4 from anywhere - high bin model . It’s true Apple uses a different packaging approach to normal server DIMMs, but that doesn’t make it magically immune to the wider memory supply problem. High-RAM M3 Ultra lead times are already very long, so clearly Apple is not unaffected. My point isn’t that Mac Studio uses identical RAM sticks to AI servers; it’s that the same broader supply chain pressure on high-capacity memory is still likely to hit Apple’s larger bins too. So yes, Apple’s implementation is different. No, that doesn’t prove the current delays are unrelated to the wider memory shortage.

Current delivery on high ram bins is 5 months + The ram shortage is effecting Apple now let alone before new hardware appears .