Avalon local LLM

[Pocster]

6 minutes ago, Pocster said:

(expletive deleted)s me off when I see youtube chatgpv vs opus vs fable all given the same prompot to "write a flappy birds 3d game" and thats a test!. Amazing any of them produce anything at all but (expletive deleted) me - it's not a TOY!. It's super powerful.

or (expletive deleted)ing token speed "test". mac Mx vs rtxY . Mac loses of course. Thats what stopped me getting the m3 at first always slower than rtx BUT when you need a larger model or multiple models Mac wins. For me flexibility over speed is the winner easily. (expletive deleted)ing 10k for an rtx6000 with 96Gb.... Nvidia make macs look cheap!

Edited yesterday at 11:20 by Pocster

[Pocster]

I often say to chat "Are you a BBC Model B?, you are supposedly frontier cutting edge AI. So stop being a prick". Surprisingly it does frquently man up and produce something nearer what I requested. SO the answer is insult it for better results. Like humans I guess!

Edited yesterday at 11:27 by Pocster

[SimonD]

2 hours ago, Pocster said:

I often say to chat "Are you a BBC Model B?, you are supposedly frontier cutting edge AI. So stop being a prick". Surprisingly it does frquently man up and produce something nearer what I requested. SO the answer is insult it for better results. Like humans I guess!

I'm still being nice! And new workflow in place. Discuss with Claude, produces prompt for Claude Code, check output using Claude, approve/disapprove, back to Claude Code to Implement and commit or re-prompt. Scary thing is that Claude Code shows the token burn count as it's chugging away.

[Pocster]

Just now, SimonD said:

I'm still being nice! And new workflow in place. Discuss with Claude, produces prompt for Claude Code, check output using Claude, approve/disapprove, back to Claude Code to Implement and commit or re-prompt. Scary thing is that Claude Code shows the token burn count as it's chugging away.

Erm, watching your ££££ go. This is what I hate with cloud models. Not just the cost. But that they can change costs at any time, it's clear that what cloud model you run now may not be the cloud model you run in 1 mnute i.e. backend changes. I hate it. You become dependent on it like a drug dealer and then they move the goal posts. This is my main reason for enjoying the 20 quid 'near' claude experience with chat. It requires more work to setup, but i cant bitch (though I do!) for £20. But of ourse a local llm will be consistent .... models just aint quite there yet. But you know even though not out an M5 with 512gb still tempting. Really need models to catch up rather than hardware.

[SimonD]

56 minutes ago, Pocster said:

Erm, watching your ££££ go. This is what I hate with cloud models. Not just the cost. But that they can change costs at any time, it's clear that what cloud model you run now may not be the cloud model you run in 1 mnute i.e. backend changes. I hate it. You become dependent on it like a drug dealer and then they move the goal posts. This is my main reason for enjoying the 20 quid 'near' claude experience with chat. It requires more work to setup, but i cant bitch (though I do!) for £20. But of ourse a local llm will be consistent .... models just aint quite there yet. But you know even though not out an M5 with 512gb still tempting. Really need models to catch up rather than hardware.

Well, out of luck on burning too many tokens right now as no Fable any more:

The US government, citing national security authorities, has issued an export control directive to suspend all access to Fable 5 and Mythos 5 by any foreign national, whether inside or outside the United States, including foreign national Anthropic employees. The net effect of this order is that we must abruptly disable Fable 5 and Mythos 5 for all our customers to ensure compliance. Access to all other Anthropic models will not be affected.

[Pocster]

2 minutes ago, SimonD said:

Well, out of luck on burning too many tokens right now as no Fable any more:

The US government, citing national security authorities, has issued an export control directive to suspend all access to Fable 5 and Mythos 5 by any foreign national, whether inside or outside the United States, including foreign national Anthropic employees. The net effect of this order is that we must abruptly disable Fable 5 and Mythos 5 for all our customers to ensure compliance. Access to all other Anthropic models will not be affected.

Hmmmm, your project you're working on must have REALLY upset someone!

[Pocster]

tell you something I love with chat. I upload screenshots when we have issues. For a whiel we had some old text/icons - no issue just left there while I fix other things. It would seem from repeated screen shots it decided to remove them. No "I'm getting rid of this" just gone. (expletive deleted)ING MAGIC I TELL YOU!

[Pocster]

Kid in a sweetshop!.

[Pocster]

OH! My rendering with reflection/bloom (expletive deleted)s m3 gpu even before local LLM does work. Oh!. looks like I need a 2nd pc just for dashboard!

[SimonD]

15 hours ago, Pocster said:

Kid in a sweetshop!.

 

That does look amazing!

[Pocster]

18 minutes ago, SimonD said:

 

That does look amazing!

I can’t draw for toffee so chat does code so everything is procedurally generated . Reflections , fresnel , blooming all the ps5 effects I love . Lipsync on the bots mouth . Fuzzy logic because when you say “ Birdy “ it could be translated as “ birdie “ . Also phonetic matching e.g “mould play “ = “ Coldplay “ . No hard coding of phrases everything just open source . Love it . 

[Pocster]

The thing chat and I suspect all coding AI's are crap at is not 'thinking' about the problem outside the box.

For example ask it for code to draw a circle (ignoring a circle primitive) it would use sin/cos/pi because that's standard maths. But! thats really shit. Will work but slow. I can think of 20 ways that would be 1000% faster.

Equally a Astar algorithm i.e. path finding from baddy to player around scenery. Standard methods will be used. Shite slow, not practical in a real game. So "understanding" the problem for efficient code is the method not the code. That requires a human. I was at 86% gpu usage because its code uses 'standard' methods. Guide it on different techniques and it visually looks the same but halved gpu usage.

[SimonD]

4 hours ago, Pocster said:

The thing chat and I suspect all coding AI's are crap at is not 'thinking' about the problem outside the box.

Yes, it definitely struggles with this and needs guidance. Anything that involve geometric sense seems to be a crux, unless it's very simple. I've also just found something else it's rubbish at. Ask it to build a website and it'll build something very good in the blink of an eye. I asked it to re do a couple of adverts I've got in print media - the output was just terrible, so bad I'm actually speechless so I just said I'd go and do it myself!

 

I've been finding the bliss of work with Claude & Claude code and it's like magic. It's also funny how Claude restricts Claude Code, admitting the constraints have to be very clear so that Claude Code doesn't try to do something 'helpful' - it just makes me laugh 😁

[Pocster]

8 minutes ago, SimonD said:

Yes, it definitely struggles with this and needs guidance. Anything that involve geometric sense seems to be a crux, unless it's very simple. I've also just found something else it's rubbish at. Ask it to build a website and it'll build something very good in the blink of an eye. I asked it to re do a couple of adverts I've got in print media - the output was just terrible, so bad I'm actually speechless so I just said I'd go and do it myself!

 

I've been finding the bliss of work with Claude & Claude code and it's like magic. It's also funny how Claude restricts Claude Code, admitting the constraints have to be very clear so that Claude Code doesn't try to do something 'helpful' - it just makes me laugh 😁

Frequently its "given up" today. Seems to go through hourly phases even with the same code of amazing and shite. Chat's bodging again now. Gave it a specific render description and it did a cheap BBC B demo for me. It's like it's programmed to sometimes offer cheap solutions i.e less compute. I reckon at the end of the day when it goes home to Mrs Chat it bitches about this guy who's only on the 20 quid plan and wants the (expletive deleted)ing world!

 

Now its basically forcing me to use codex cloud ... "I cant do it but codex cloud can patch this" . I'm going to conclude this is deliberate as now each scout/patch uses around 5% of my 5 hr quota. These companies are shit. Need to use codex cloud for major revision or feature add then tell chat to do one and use local llm for scout.

Edited 2 hours ago by Pocster