Avalon local LLM

[Pocster]

8 hours ago, -rick- said:

@Pocster How desperate are you?

 

 

Well ! Unfortunately as I presume you now realise the ram shortage is a massive problem - yes , even for Apple .

It did look obvious tbh - so now I have to wait for any indicator of ultra appearing and jump on it ! I guess it could be announced at wwdc - maybe have a date for pre orders . Buys Apple time to acquire ram whilst still be bigging up their product . That’s the best I can hope for - a definite date to refresh Apple Store !

[-rick-]

https://arstechnica.com/gadgets/2026/05/apple-may-take-several-months-to-catch-up-to-mac-mini-and-studio-demand/

 

Interestingly it mentions TSMC being a bottleneck. TSMC doesn't make RAM.

Edited Friday at 14:19 by -rick-

[Pocster]

2 hours ago, -rick- said:

https://arstechnica.com/gadgets/2026/05/apple-may-take-several-months-to-catch-up-to-mac-mini-and-studio-demand/

 

Interestingly it mentions TSMC being a bottleneck. TSMC doesn't make RAM.

TSMC doesn’t make RAM, no one said they do. TSMC is the SoC bottleneck.

 

The RAM shortage is a separate bottleneck, and Apple clearly isn’t immune. Other manufacturers have openly said memory shortages are affecting them, and Cook has said Mac mini / Mac Studio supply-demand balance will take several months.

 

High-memory Mac mini / Studio configs are also exactly where the worst delays and unavailable options have shown up.

 

There are also multiple reports suggesting the memory shortage could last well into 2027 and possibly beyond, especially as AI demand keeps soaking up supply.

 

So pointing out that TSMC doesn’t make RAM doesn’t disprove the RAM issue; it just identifies a different bottleneck.

[-rick-]

9 minutes ago, Pocster said:

So pointing out that TSMC doesn’t make RAM doesn’t disprove the RAM issue; it just identifies a different bottleneck.

 

Sure, I just thought it was interesting to highlight.

[Pocster]

Even if Mac unified memory and iPhone RAM aren’t identical final packages, they come from the same constrained LPDDR supply chain. Apple then has to allocate that supply across products, and iPhone will obviously take priority over niche high-memory Mac configs.

[MikeSharp01]

3 hours ago, Pocster said:

Even if Mac unified memory and iPhone RAM aren’t identical final packages, they come from the same constrained LPDDR supply chain. Apple then has to allocate that supply across products, and iPhone will obviously take priority over niche high-memory Mac configs.

I suppose that consumer high end local LLM schemes need large memory but you can do the same thing with a cloud based one while you await delivery of enough RAM, or whatever other semiconductor you are waiting on. Trouble is that the end of the proverbial rainbow is moving away from you faster than you can stuff memory into machines anyway, so, when your RAM arrives there will be plenty of models it won't run because they have grown, bloated, faster than you can add RAM to your machine. 

[-rick-]

12 minutes ago, MikeSharp01 said:

I suppose that consumer high end local LLM schemes need large memory but you can do the same thing with a cloud based one while you await delivery of enough RAM, or whatever other semiconductor you are waiting on. Trouble is that the end of the proverbial rainbow is moving away from you faster than you can stuff memory into machines anyway, so, when your RAM arrives there will be plenty of models it won't run because they have grown, bloated, faster than you can add RAM to your machine. 

 

Does look that way. New Deepseek one for example, and it's been touted has having huge optimizations for reducing memory usage. It might be much better at storing context but if the model is so huge in the first place it's not easy to run at home.

 

Google seems to have done well with Gemma, but of course that is a lot cut down from the leading models and they aren't releasing those. Doubt any western firm will for 'safety' reasons. So far the Chinese govt doesn't seem to care about that so long as their models spout Chinese propoganda at the relevant moments.

[Pocster]

8 hours ago, MikeSharp01 said:

I suppose that consumer high end local LLM schemes need large memory but you can do the same thing with a cloud based one while you await delivery of enough RAM, or whatever other semiconductor you are waiting on. Trouble is that the end of the proverbial rainbow is moving away from you faster than you can stuff memory into machines anyway, so, when your RAM arrives there will be plenty of models it won't run because they have grown, bloated, faster than you can add RAM to your machine. 

I’ll use chat as my reasoner but I need local to  do my local actioned stuffed until I can source enough ram . Bigger model did mean better but now smaller better optimised models can out perform a larger one . Obviously there’s a point when you can’t go smaller without too much compromise . If you ask ChatGPT itself how long before a model matches its capabilities of today in a ‘ reasonable ‘ size it reckons within 2 years a 256gb model will be comparable to ChatGPT 5.4. That’s insane ! To have that capability sat on your desk in a relatively low ram config.

Moe and potentially’ bolt on ‘ specialists seems to be the next level . A medical specialist , maths specialist etc plugged in to the reasoner.

[Pocster]

After lots of reading ( for probably months ) , YouTube videos etc etc . Enough to confuse anyone I decided to start Avalon .

 

chatgpt and I are architects 

 

we setup llama etc , vs studio , ssh into m3 etc etc , even parsec screen share 

 

Deepseek is our reasoner . Qwen coder next our sweat shop “ just do it ! “ coder !

 

chat managed to create me some scripts so we create a brief for today’s task . Get Deepseek to take that and produce a detailed instruction list for qwen .

 

So process is

 

Create task 

Give to Deepseek 

Approve Deepseek output 

Tell qwen to do it ( for reasons I’m unsure of we don’t use git diffs but json - I didn’t argue as it seems to work )

Then some kind of test . Chat gets excited about this ; me less so when terminal window says “ it worked ! “ 😊

But we are building framework and structure first .

 

I’m blown away by this all locally !

llm , creating document for llm to produce detail for another llm 

Crazy !!

90% of programmers obsolete ! 
Just todays tasks would have been weeks of learning and work .

 

[Pocster]

Notice Mac minis basically out of stock and even a m3 ultra 96gb ( base model ) no where to be seen . I got mine from curry’s of all places ! , check it every few days to judge demand - now they have none !

I think when I sell the m3 I’ll get a pretty good price for it even with m5 released because it’s all going to sell out fast !

[JohnMo]

6 hours ago, Pocster said:

Notice Mac minis basically out of stock and even a m3 ultra 96gb ( base model ) no where to be seen

Bought mine the other week from Amazon, just looked and still available.

Edited 5 hours ago by Nickfromwales

[Pocster]

5 hours ago, JohnMo said:

Bought mine the other week from Amazon, just looked and still available. Look harder

Yeah it said “ out of stock “ so that’s a clue .

Screen grab is from now !

Edited 5 hours ago by Nickfromwales

[Pocster]

4 hours ago, Pocster said:

Yeah it said “ out of stock “ so that’s a clue .

Try (expletive deleted)ing reading . Screen grab is from now !

what is it with pricks on this forum !

I’ll wait for apology 

For completeness yes it’s on Amazon apparently at an inflated price . 

Edited 5 hours ago by Nickfromwales

[MikeSharp01]

Why not get an RTX 5090 card for you desktop pc. True they don't have the shared ram but they should do all you need. I have a 3090 with just 24Gb of ram but it does all I need. 

[JohnMo]

Mac mini, M4 is what I quoted and yes it's for sale, delivery on Wednesday.

 

Edited 5 hours ago by Nickfromwales

[Pocster]

38 minutes ago, MikeSharp01 said:

Why not get an RTX 5090 card for you desktop pc. True they don't have the shared ram but they should do all you need. I have a 3090 with just 24Gb of ram but it does all I need. 

A sane person . Problem is unified ram . Best on Nvidia is 96gb on rtx6000 that’s a 10k gpu card !!!!

This is why Apple have ( unintentionally) jumped to number 1 for local llm .

Edited 9 hours ago by Pocster

[SteamyTea]

I am waiting for the ultimate answer.

[Pocster]

8 minutes ago, SteamyTea said:

Don't lock it down, I am waiting for the ultimate answer.

I’ll give you a 69 

[Nickfromwales]

Thread tidied. As you were people.

 

The management.

[MikeSharp01]

3 hours ago, Pocster said:

Don't lock it down, I am waiting for the ultimate answer.

Like we already said the answer is moving all the time so 42 won't cut it in today's race. 

 

FWIW I think this thread is interesting because this technology has so many implications for us all and although we are just experimenting the opportunity to do amazing things is tantalising and, frankly, we / they have not found the killer application yet and that may come almost as probably from amateurs playing as professionals trying to crack it. The local platform arms race to top or bottom is really an aside. As you say you can already spend 10K on a card always assuming your rings (mains) can supply the power. Pretty soon fan coils and UFH won't be the only things your average ASHP will be cooling!