Avalon local llm

[Pocster]

M3 96gb arrived ! Now the work begins !!

[-rick-]

Well this escalated.

 

Not trying to win an argument or even have an argument. When a response to a post of mine doesn't seem to engage on the points in my post I wonder if I didn't explain it well enough so try again. I think you understand my points now so not worth going further.

 

The one thing I will point out is that the shortages are all on older gen stuff. M4 mac minis, M3 mac studios. Products expected to have announcements of replacements in the next 2 months. It's a well documented Apple tactic to sell down inventory/put long lead times on products near this point (happens with iPhone, laptops, etc). It doesn't always happen because in general Apple has bigger surplus but if things are running low they don't bother to build more.

[SteamyTea]

Helium is essential for making all electronics these days.

Helium is a byproduct of oil and gas extraction, as well as radioactive geological processes.

We are basically running out of helium, so if you want some RAM, keep filling your car up with fossil fuels.

If, by chance, you own a small BEV, say a Renault Zoe, and you want some RAM, then you only have yourself to blame, you selfish (expletive deleted)er.

[-rick-]

7 minutes ago, SteamyTea said:

Helium is essential for making all electronics these days.

Helium is a byproduct of oil and gas extraction, as well as radioactive geological processes.

We are basically running out of helium, so if you want some RAM, keep filling your car up with fossil fuels.

If, by chance, you own a small BEV, say a Renault Zoe, and you want some RAM, then you only have yourself to blame, you selfish (expletive deleted)er.

 

This is not the point you are making but what's going on with Iran is massively constraining the supply of Helium. Qatar is the main producer.

 

US Labs have been told to expect a 50% reduction in supply. This affects MRI's, chip fab and a load of other things. Everything is going to get very difficult very soon.

 

If the global economy takes a real dive, one silver lining is that when the Iran crisis is over the AI companies might not have such deep pockets (or in some cases still exist) and so memory might become more available again. Then again, nobody else will have money either. 🥴

[Pocster]

Oi (expletive deleted)ers !

This is my local llm thread ! Not political or supply issues . Get a (expletive deleted)ing room 👊🏻 

[-rick-]

7 minutes ago, Pocster said:

Oi (expletive deleted)ers !

This is my local llm thread ! Not political or supply issues . Get a (expletive deleted)ing room 👊🏻 

 

Sorry, thought you'd be too busy chatting up your new llm girlfriend to notice 😏

[SteamyTea]

9 minutes ago, -rick- said:

18 minutes ago, Pocster said:

Oi (expletive deleted)ers !

This is my local llm thread ! Not political or supply issues . Get a (expletive deleted)ing room 👊🏻 

 

Sorry, thought you'd be too busy chatting up your new llm girlfriend to notice

What do you think he calls her?

[Pocster]

11 minutes ago, -rick- said:

 

Sorry, thought you'd be too busy chatting up your new llm girlfriend to notice 😏

Thing is @-rick- I was just playing with you a bit . Clearly demand will out strip supply for m5 ultra for sure I.e delivery times will slip like a dog . My intention is to order within minutes of it going live in the App Store .

My ambitions probably do not need m5 with 256gb or more but it bugs me to buy “old” m3 ultra at near full whack when we all know new and similar priced is coming .

 

[-rick-]

1 hour ago, Pocster said:

I was just playing with you a bit

 

Of course you were! (though I didn't see the buy of the Mac as part of that, just assumed you were impatient/had money to burn).

 

1 hour ago, Pocster said:

. Clearly demand will out strip supply for m5 ultra for sure I.e delivery times will slip like a dog . My intention is to order within minutes of it going live in the App Store .

 

Glastonbury tickets all over again 😛

 

1 hour ago, Pocster said:

My ambitions probably do not need m5 with 256gb or more but it bugs me to buy “old” m3 ultra at near full whack when we all know new and similar priced is coming .

 

Amen.

 

Though before you spend money I assume you have a prototype running (at lower model size) on existing hardware?

[-rick-]

1 hour ago, SteamyTea said:

What do you think he calls her?

 

I'm quite happy with my decision to spend zero time trying to imagine whats going on in @Pocsters head!

[Pocster]

47 minutes ago, -rick- said:

Though before you spend money I assume you have a prototype running (at lower model size) on existing hardware?

Of course not ! The m3 ultra 96gb will be the prototype!

[Pocster]

1 hour ago, -rick- said:

 

Of course you were! (though I didn't see the buy of the Mac as part of that, just assumed you were impatient/had money to burn).

 

 

Glastonbury tickets all over again 😛

 

 

Amen.

 

Though before you spend money I assume you have a prototype running (at lower model size) on existing hardware?

Go big or don’t go in 😉

[Pocster]

On qwen coder 30b getting a very respectable 80 tokens / sec .

Estimate on ultra m5 same model might touch 200 !

[SteamyTea]

6 minutes ago, Pocster said:

tokens / sec

Wrong metric.

 

https://www.forbes.com/councils/forbestechcouncil/2025/10/21/why-tokens-per-watt-is-crucial-for-measuring-ai-efficiency/

[-rick-]

You looked at Gemma 4? Supposed to be able to get qwen like performance/capability but in a much smaller model. 96GB bit of a waste for it

[-rick-]

2 minutes ago, SteamyTea said:

Wrong metric.

 

https://www.forbes.com/councils/forbestechcouncil/2025/10/21/why-tokens-per-watt-is-crucial-for-measuring-ai-efficiency/

 

Good job Macs are about the most efficient AI platform then

[Pocster]

55 minutes ago, -rick- said:

You looked at Gemma 4? Supposed to be able to get qwen like performance/capability but in a much smaller model. 96GB bit of a waste for it

Sure . Just trying to get visual studio to connect to anything . Will need multiple models anyway for full Avalon