Avalon local LLM

[Pocster]

Hmmmm . Proving tricky !

After a lot of talking we have a “ Avalon “ wake word with pretty good success . But radio on etc not so great . Trying to access lms streams and use that to filter my speech from radio not producing good enough results . Also depends where I am in the room - we use direction of speech to aid if I said or radio said it . Solution seems be another uma8 placed on the opposite side of the room ….

[Pocster]

Erm! LOL! talk about confusing.

 

We made good progress. But then we start hitting chatgpt limitations i.e. cant see repo so gets a zip. Its patches drift and we end up with a mess.

SO!

Back to local llm (gpt-oss 120b for now). I've got a headache and I haven't written any code!

Get chatgpt to talk me through config in VS I have never heard of. Then chatgpt wants extra tools to make my life easier ( apparently ) so we install and setup a mcp server (???) - bit lost on what and why.

None the less magically continue now has new special tools. Still don't get it as continue has basic tools but they all seem problematic e.g. can only open and read small files....??!

Clearly beyond my pay grade but moving forward;  moving forward even though I don't know where we are heading!!. Guessing you claude code guys get all this nicely packaged for you.

 

If nothing else chat is taking me into yet another rabbit hole!

[SimonD]

33 minutes ago, Pocster said:

Guessing you claude code guys get all this nicely packaged for you.

Claude does package the stuff very nicely, but it's still not without its problems. As a result I have a specific instruction to take me through what it's doing and why, and then on fairly major decisions, I go and do a but more research and go back to question the proposals to make sure it's right. On a few occasions this has resulted in big changes in direction, sometimes not. Instructions include not writing any new code until I have provided and confirmed the latest deployed versions

The other risk I found was that the packages can contain out of date files that don't contain recent code changes so crashes the app. 

But tbh, nothing beyond what I've seen and experienced in human coding teams. I'm actually constantly finding myself surprised by how it's always helpful and amendable to changes and completely changing my mind about something we've just deployed. That response in me tells me everything I need to know about my experience and history dealing with human programmers!

[Pocster]

20 minutes ago, SimonD said:

But tbh, nothing beyond what I've seen and experienced in human coding teams. I'm actually constantly finding myself surprised by how it's always helpful and amendable to changes and completely changing my mind about something we've just deployed. That response in me tells me everything I need to know about my experience and history dealing with human programmers!

This is why I love chat - we literally discuss the project . I’ll sometimes mention something it hasn’t thought of - which in turn takes it down a different path .

Now I have my own mcp tools I can get exactly what I want from local llm on the project . Then send it to architect boss chat to review ! . If boss says it’s good I let local implement code changes ! . 
It’s all quite bizarre as I never look at the code even though I’m an ex soft engineer !

It’s management of llm ‘s !!! 

[-rick-]

Now we know where all the 256GB m3s went: https://www.theregister.com/off-prem/2026/05/15/aws-racks-m3-ultra-macs-that-boast-specs-you-cant-currently-buy/5240901

[Pocster]

11 minutes ago, -rick- said:

Now we know where all the 256GB m3s went: https://www.theregister.com/off-prem/2026/05/15/aws-racks-m3-ultra-macs-that-boast-specs-you-cant-currently-buy/5240901

No wonder there’s a (expletive deleted)ing shortage !

Guess what @-rick- …. 😊 . Managed to grab myself an m3 256gb 😎

Yes ! Paid a premium - but not insane . Will copy everything over from 96gb and reckon I can sell that at near what I paid on eBay . Last time I checked there were zero m3 96gb on eBay . I’m surprised how capable it is especially after watching all the pointless YouTube videos comparing it against rtx4090 etc - tokens per second blah blah . If the models shit speed does not matter !

Effrctively I’ve now got a near codex workflow locally . Chat is still the boss . It’s SO much more capable than previous version - genuinely surprised. Excellent software engineer / architect for 20 quid a month. Whilst frontier is clearly more capable, local is catching up real fast !

[Pocster]

Avalon wise made significant progress .

 

What I struggled to match is Alexa’s shear speed “ Alexa , what’s the time in Tokyo ? “ 

Virtually instant response . Hard to find exactly how Amazon structure this . A voice response or acknowledgment just a few 100ms slow can feel painful .

So ! We have Avalon ‘skills’ I.e fuzzy word match for speed . So “ Avalon , what’s the time in Tokyo ? “ , is super close to Alexa speed .

Whisper transcribes realtime . Wake word software was all too slow .

We have a weighted graph of probability and misread / mis said word library - so that’s far quicker than an llm trying to work it out .

Chat’s code was a spaghetti mess - so I got it to rewrite as a finite state machine - easier for debugging . We also ‘smoke test ‘ so input can be a text of words , mis spellings etc .

Really happy with “ what’s the time ? “ and “what’s the time in Tokyo ? “ - the fraction of delay or not defining the question .

The time/date skill obviously of little use but it’s a good test for the pipeline .

Next will probably be a ‘general knowledge’ skill I.e anything you ask goes to internet to get answer .

Edited 2 hours ago by Pocster