Avalon local LLM

[MikeSharp01]

7 minutes ago, Pocster said:

I don't understand the question anymore.

No, well that might be where the rest of us are. Here is my summary:

 

You started with using local LLMs to develop some automation, tried to recreate 'Alexa' or some such and had some success doing it but you got taken over by the need swell your memory so you handle even more billions of weights etc then got involved in multi modal stuff and games development and now you are lost up the Orinoco without any memory of the journey or the objective - so essentially without a paddle! 

[Pocster]

10 minutes ago, MikeSharp01 said:

No, well that might be where the rest of us are. Here is my summary:

 

You started with using local LLMs to develop some automation, tried to recreate 'Alexa' or some such and had some success doing it but you got taken over by the need swell your memory so you handle even more billions of weights etc then got involved in multi modal stuff and games development and now you are lost up the Orinoco without any memory of the journey or the objective - so essentially without a paddle! 

Oh that question!

 

I'm on track!

So we have 3D amazing world.

Alexa/chatgpt subsystem is still in there. When you make a vocal request a sphere (avalon) thats rolling around the landscape will come alive and visually (and audibly ) confirm /acknowledge your request. It then continues on it's infinite plight through infinite worlds.

Thought that was obvious ......

Edited Wednesday at 19:58 by Pocster
They say I'm a dreamer ... and I am the only one

[MikeSharp01]

5 minutes ago, Pocster said:

plight

Pretty much sums it up!

 

So are all the walls in your place interactive so you can live in this psychedelic paradise you have prompted into existance- reminds me a bit of a scene from the Ipcress file - @SteamyTea will no doubt find it and add it to this thread.

 

[Pocster]

Just now, MikeSharp01 said:

Pretty much sums it up!

 

So are all the walls in your place interactive so you can live in this psychedelic paradise you have prompted into existance- reminds me a bit of a scene from the Ipcress file - @SteamyTea will no doubt find it and add it to this thread.

 

@MikeSharp01 as I said before brilliance and insanity are different sides of the same coin. Alexa/Siri (even the supposed new one with a face) are shite. This is interactive information/private/ secure and beautiful!

Every other (expletive deleted)er getting frontier AI to write flappy birds or fix a function or do some AI slop video of Donald Trump being a prick (hang on! thats real) - but now you can create beautiful entities that surpass knowledge. You can place your order now if you like?

[SteamyTea]

 

[Pocster]

4 minutes ago, SteamyTea said:

 

yes that sums it up exactly. I watched it with no audio and even with subtitles it made little sense - so just like my project. Only the select few get it.

Edited Wednesday at 21:03 by Pocster

[SimonD]

1 hour ago, Pocster said:

@Nickfromwales Love you Uncle Nick. I know you are just liking to support me through a hard time but a comment would be helpful

You're not feeling lonely are you? I've been away and then just too busy with other distractions! 

 

4 hours ago, Pocster said:

Claude's been (expletive deleted)ing amazing!.

I think I might have hinted at this one a while ago, but you were single track on ChatGPT plus a few other local models to listen at the time. Where have you been? 😉

All looks (expletive deleted)ing awesome I have to say.

I've just been sorting the workflow with Claude Sonnet 5 and then Claude Code and now I'm genuinely not having to look at code any more either, just copy & paste the prompts and off we go. Did 2 days of coding and not a single bug from the new code - it just highlighted bugs in the existing code, but solved so easily, even if Claude did tie itself in knots because it forgot to read and obey my dev principles instructions - but it realised and apologised for this.. It continuously blows me away. I'm beginning to create a list of ideas for things to build, but Claude code then tells me off for not staying focused on what we're doing.

 

40 minutes ago, Pocster said:

Only the select few get it.

I get it. I've spent my life doing what others thought were stupid projects just because I thought they'd be interesting and fun and I wanted to see where they'd take me. No need for any other reason IMHO.

[Pocster]

1 minute ago, SimonD said:

You're not feeling lonely are you? I've been away and then just too busy with other distractions! 

 

I think I might have hinted at this one a while ago, but you were single track on ChatGPT plus a few other local models to listen at the time. Where have you been? 😉

All looks (expletive deleted)ing awesome I have to say.

I've just been sorting the workflow with Claude Sonnet 5 and then Claude Code and now I'm genuinely not having to look at code any more either, just copy & paste the prompts and off we go. Did 2 days of coding and not a single bug from the new code - it just highlighted bugs in the existing code, but solved so easily, even if Claude did tie itself in knots because it forgot to read and obey my dev principles instructions - but it realised and apologised for this.. It continuously blows me away. I'm beginning to create a list of ideas for things to build, but Claude code then tells me off for not staying focused on what we're doing.

 

I get it. I've spent my life doing what others thought were stupid projects just because I thought they'd be interesting and fun and I wanted to see where they'd take me. No need for any other reason IMHO.

You're back! My loneliness is over!

[Nickfromwales]

26 minutes ago, Pocster said:

You're back! My loneliness is over!

You’re never alone, just everyone looking at this is like folk watching a rollercoaster; but the thing needs to slow down or stop so we can get on, 🤣.

 

I’m bat-shit crazy with work coming in from every angle, so have just been a spectator between glasses of Sauvignon. Hic.  

[Sparrowhawk]

9 hours ago, SimonD said:

I've just been sorting the workflow with Claude Sonnet 5 and then Claude Code and now I'm genuinely not having to look at code any more either, just copy & paste the prompts and off we go. Did 2 days of coding and not a single bug from the new code - it just highlighted bugs in the existing code, but solved so easily, even if Claude did tie itself in knots because it forgot to read and obey my dev principles instructions

I'm finding Sonnet 5 works out more expensive than Opus due to the number of tokens it's using (for coding). Not impressed with it TBH. Now Fable, that really is something else!

[Pocster]

2 hours ago, Sparrowhawk said:

I'm finding Sonnet 5 works out more expensive than Opus due to the number of tokens it's using (for coding). Not impressed with it TBH. Now Fable, that really is something else!

yeah my 1st time with calude token wise was fine. Next day I was going to burn my 5 hr limit in about 1 hour. Dropped it to opus medium always clear the context if new subject within project. This seems to have reduced my token burn a lot. Fable; yeah seen some things it's done - insane!. But I'm more than happy with opus 4.8 and what we've achieved - literally makes my jaw hit the floor!

Edited yesterday at 09:43 by Pocster

[SimonD]

3 hours ago, Sparrowhawk said:

I'm finding Sonnet 5 works out more expensive than Opus due to the number of tokens it's using (for coding). Not impressed with it TBH. Now Fable, that really is something else!

 

Interesting, I was looking at my usage stats and despite doing some fairly heavy work - I've only used about 25% of my weekly limit. What I did notice though is that I was using Sonnet 5 for the thinking and generating promps but in VSCode I was still on Opus, so I wonder whether that has got anything to do with it?

 

1 hour ago, Pocster said:

yeah my 1st time with calude token wise was fine. Next day I was going to burn my 5 hr limit in about 1 hour. Dropped it to opus medium always clear the context if new subject within project. This seems to have reduced my token burn a lot. Fable; yeah seen some things it's done - insane!. But I'm more than happy with opus 4.8 and what we've achieved - literally makes my jaw hit the floor!

What's interesting is that on Claude website about choosing model it says that Opus uses more of your rate limit but in VSCode Claude Code model selection, it says that Opus gets 2 x Sonnet usage. A bit confusing really. 

I'm going to give all of the models a little try but I don't think I've got anything complex enough to really test Fable right now. But maybe I'll think of something for my free Fable trial expires.

[Pocster]

4 minutes ago, SimonD said:

 

Interesting, I was looking at my usage stats and despite doing some fairly heavy work - I've only used about 25% of my weekly limit. What I did notice though is that I was using Sonnet 5 for the thinking and generating promps but in VSCode I was still on Opus, so I wonder whether that has got anything to do with it?

 

What's interesting is that on Claude website about choosing model it says that Opus uses more of your rate limit but in VSCode Claude Code model selection, it says that Opus gets 2 x Sonnet usage. A bit confusing really. 

I'm going to give all of the models a little try but I don't think I've got anything complex enough to really test Fable right now. But maybe I'll think of something for my free Fable trial expires.

Its all still a con tbh. You dont know what anything costs tokenwise even when you've done the task. They change the backend ( certainly in chat they do! ). 5 hr window can be quite 'loose'. Still! As AI compute gets cheaper we ultimately will pay less for better models. Perhaps they'll drop subscription and go fully paid API?. Imagine what fable 3 in 2 yrs time will be like!!!!. Mental now!

 

[Pocster]

This is funny!

I screen grabbed a shot for Claude to look at. Unbeknown too me my buildhub page was showing so it got to see my 'signature'. Here's what it said

 

"The note "hoping some hair will grow around it" — if you mean you want vegetation/scatter around the feet to hide the leg-terrain junction, that's a separate feature (prop-adjacent vegetation scatter) I can look at. Want me to?"

Edited 23 hours ago by Pocster

[SimonD]

3 hours ago, Pocster said:

Its all still a con tbh. You dont know what anything costs tokenwise even when you've done the task. They change the backend ( certainly in chat they do! ). 5 hr window can be quite 'loose'. Still! As AI compute gets cheaper we ultimately will pay less for better models. Perhaps they'll drop subscription and go fully paid API?. Imagine what fable 3 in 2 yrs time will be like!!!!. Mental now!

 

 

I'm sure it is and I bet it also depends on the tasks, but I think Claude is pretty good. Today I've use 6% of my weekly total and 8% of my Fable - and that's dealt with some fairly complex reasoning rather than coding itself. I can keep tabs - but maybe that was different with chat? TBH I think what I get for my subscription is pretty good - I've hit my paywall only a couple of times but then in fairness I'm usually due a break anyway so I consider it a health check 😊. I doubt they'll offer it for free, even if it would be amazing if they did.

[Pocster]

2 minutes ago, SimonD said:

 

I'm sure it is and I bet it also depends on the tasks, but I think Claude is pretty good. Today I've use 6% of my weekly total and 8% of my Fable - and that's dealt with some fairly complex reasoning rather than coding itself. I can keep tabs - but maybe that was different with chat? TBH I think what I get for my subscription is pretty good - I've hit my paywall only a couple of times but then in fairness I'm usually due a break anyway so I consider it a health check 😊. I doubt they'll offer it for free, even if it would be amazing if they did.

Oh I didn't mean free I just meant no subscription but paid API i.e. pay as you go!. Honestly though visually I'm flabbergasted at what can be achieved. But I keep adding more effects and tanking frame rate so another round of optimisations!

[SimonD]

Just now, Pocster said:

Oh I didn't mean free I just meant no subscription but paid API i.e. pay as you go!. Honestly though visually I'm flabbergasted at what can be achieved. But I keep adding more effects and tanking frame rate so another round of optimisations!

 

Ah, yes okay. I think the odds are that's the way it'll go probably and it could be a good thing.

[Pocster]

5 minutes ago, SimonD said:

 

Ah, yes okay. I think the odds are that's the way it'll go probably and it could be a good thing.

Cost me an F'ing fortune I bet!. But as I said competitive nature of these firms models getting better, hardware costs will come down, more people use it then costs per user must decline. I read something interesting; people are quite loyal to say netflix,apple tv, even sometimes ISP - but for AI no one gives a (expletive deleted) and we whore about for the best deal. If I did a frontier cloud AI I'd name it "Susie does". I reckon I'd pick up a couple of million subscribers from day 1.

 

Edited 20 hours ago by Pocster

[Pocster]

@SimonD if you dont mind me asking what kind of problems do you need fable to look at??. I'm doing a super advanced 3d engine that basically makes 95% of game ones look shit - so I'm curious what task you are doing that requires even more umph than opus (only answer if it's ok of course). I'm just thinking "WTF is HE doing!"

 

[Pocster]

JESUS! As we struggle to maintain 60fps with SO much more added an optimisation loop again. Every trick I can think of discussed with Claude. Gained 30% again!!. Now to use that on expensive effect! Terrain can cast shadows!. That will make it look like the bollocks with bollocks on it looking at bollocks with golden bollocks.

[Pocster]

erm just tried ultracode to find a nasty bug.... 70k tokkens gone in 20 seconds! 7 agents on the job. Dont really get how this agent thing works but I;ll let it run and eat my 5hr limit in 10 minutes LOL!

 

100k! sounds a lot! I always think of it as £'s!

 

400k!!!

Edited 16 hours ago by Pocster

[SimonD]

16 hours ago, Pocster said:

@SimonD if you dont mind me asking what kind of problems do you need fable to look at??. I'm doing a super advanced 3d engine that basically makes 95% of game ones look shit - so I'm curious what task you are doing that requires even more umph than opus (only answer if it's ok of course). I'm just thinking "WTF is HE doing!"

 

Nothing massively exciting really. It's all down to the reasoning side. As you know I've built Open Heat Loss which has morphed into a full blown project management tool and even materials & quote system now and actually now that the calcs in the tool have been largely completed it's run of the mill development. But, I do some fairly tricky heat pump retrofit designs and installations so I've been building a suite of dynamic modelling tools to understand how the systems will work over time - so I use it to reason through the questions and put together a brief for the tool I want to build and usually build the first iteration to play with it. For example, I recently completed the design for a system for an older house that's on a hill so one side has massively thick stone walls adjacent to ground with no insulation, but the other side is a complete wall of glass facing south and there's ufh throughout. I wanted to model the control strategy for shoulder months to prevent over heating while taking into account the buffering capability of the walls, and then I wanted to model the most effective way to manage defrosts together with system open volume as the system has a pretty large heat pump. With the heat pump I'm using it can use the dhw cylinder by default for defrost, but because I'm plate loading the cylinder that's another control headache and because it's a system with rads and ufh, I can't reliably use the ufh as volume due to the behaviour of the mixing valve so a pure open-loop system isn't a viable strategy. Some of the control questions I'm asking fall on deaf ears in the manufacturer's tech department. The task was to arrive at a simple and effective design that uses both the passive and active components of the system to their best effect opposed to the industry fall backs which are either run fully open-loop or just put in a buffer. And out of this process I arrive at ideas for simple software tools all heating engineers ask frequent questions about and go from there.

 

And like with all these AI models, whether reasoning or generative, when they touch real-world scenarios even using relatively basic physics, they get themselves into a proper bind. I've only used Fable a little bit as the other models can cope with most stuff as although Opus has improved massively, I used to find that it could get hung up on a random constant or variable, or some other context based issue. And sometimes it just couldn't understand how to get from a to b, so I ended up with an output that didn't fit. I've also had a few instances where it's come back and said that what we're looking at is beyond its training cutoff - honest but slightly frustrating.

[Sparrowhawk]

17 hours ago, SimonD said:

I'm sure it is and I bet it also depends on the tasks, but I think Claude is pretty good. Today I've use 6% of my weekly total and 8% of my Fable - and that's dealt with some fairly complex reasoning rather than coding itself.

 

Wish you could transfer any you don't use to me!

 

17 hours ago, Pocster said:

if you dont mind me asking what kind of problems do you need fable to look at??. I'm doing a super advanced 3d engine that basically makes 95% of game ones look shit - so I'm curious what task you are doing that requires even more umph than opus (only answer if it's ok of course). I'm just thinking "WTF is HE doing!"

 

I'm doing open source work on a large 20+ year old C codebase, and designing something that will improve CPU and memory efficiency. Fable is significantly better at planning that than Opus, which leaves holes that I can spot at a glance. Fable still leaves holes, and I have to review it with GPT and have back and forth to find and fix them all, but it's a clear improvement.

 

I've got access to GPT-5.6 Sol this morning and that may be doing a better job planning on this codebase than Fable is. I've also got a lot more OpenAI tokens to burn than Anthropic, and I have to ration my Fable usage. More new toys...

 

21 hours ago, Pocster said:

Its all still a con tbh. You dont know what anything costs tokenwise even when you've done the task.

There's software that will do that and estimate the tokens per conversation (can't remember the name of the programs). That hasn't been of interest to me, so on macOS I use https://github.com/steipete/CodexBar to keep an eye on how my usage is going.

[Pocster]

Thanks guys. So you're both using it for real things not playtime like me!

What isn't clear at all (probably deliberately) is things like is best sonnet better/worse than say opus on lowest setting?. Difference in settings is vague anyway - "reasons more" meaning what? it puts 10% more effort in. How would I know it really did "reason more". I think this cloud AI needs new rules on what you actually get for your money -  feels a bit make it up as they go along. Still love it - NEW TOY INDEED!