Avalon local LLM

[Pocster]

Hmmmm . Proving tricky !

After a lot of talking we have a “ Avalon “ wake word with pretty good success . But radio on etc not so great . Trying to access lms streams and use that to filter my speech from radio not producing good enough results . Also depends where I am in the room - we use direction of speech to aid if I said or radio said it . Solution seems be another uma8 placed on the opposite side of the room ….

[Pocster]

Erm! LOL! talk about confusing.

 

We made good progress. But then we start hitting chatgpt limitations i.e. cant see repo so gets a zip. Its patches drift and we end up with a mess.

SO!

Back to local llm (gpt-oss 120b for now). I've got a headache and I haven't written any code!

Get chatgpt to talk me through config in VS I have never heard of. Then chatgpt wants extra tools to make my life easier ( apparently ) so we install and setup a mcp server (???) - bit lost on what and why.

None the less magically continue now has new special tools. Still don't get it as continue has basic tools but they all seem problematic e.g. can only open and read small files....??!

Clearly beyond my pay grade but moving forward;  moving forward even though I don't know where we are heading!!. Guessing you claude code guys get all this nicely packaged for you.

 

If nothing else chat is taking me into yet another rabbit hole!

[SimonD]

33 minutes ago, Pocster said:

Guessing you claude code guys get all this nicely packaged for you.

Claude does package the stuff very nicely, but it's still not without its problems. As a result I have a specific instruction to take me through what it's doing and why, and then on fairly major decisions, I go and do a but more research and go back to question the proposals to make sure it's right. On a few occasions this has resulted in big changes in direction, sometimes not. Instructions include not writing any new code until I have provided and confirmed the latest deployed versions

The other risk I found was that the packages can contain out of date files that don't contain recent code changes so crashes the app. 

But tbh, nothing beyond what I've seen and experienced in human coding teams. I'm actually constantly finding myself surprised by how it's always helpful and amendable to changes and completely changing my mind about something we've just deployed. That response in me tells me everything I need to know about my experience and history dealing with human programmers!

[Pocster]

20 minutes ago, SimonD said:

But tbh, nothing beyond what I've seen and experienced in human coding teams. I'm actually constantly finding myself surprised by how it's always helpful and amendable to changes and completely changing my mind about something we've just deployed. That response in me tells me everything I need to know about my experience and history dealing with human programmers!

This is why I love chat - we literally discuss the project . I’ll sometimes mention something it hasn’t thought of - which in turn takes it down a different path .

Now I have my own mcp tools I can get exactly what I want from local llm on the project . Then send it to architect boss chat to review ! . If boss says it’s good I let local implement code changes ! . 
It’s all quite bizarre as I never look at the code even though I’m an ex soft engineer !

It’s management of llm ‘s !!! 

[-rick-]

Now we know where all the 256GB m3s went: https://www.theregister.com/off-prem/2026/05/15/aws-racks-m3-ultra-macs-that-boast-specs-you-cant-currently-buy/5240901

[Pocster]

11 minutes ago, -rick- said:

Now we know where all the 256GB m3s went: https://www.theregister.com/off-prem/2026/05/15/aws-racks-m3-ultra-macs-that-boast-specs-you-cant-currently-buy/5240901

No wonder there’s a (expletive deleted)ing shortage !

Guess what @-rick- …. 😊 . Managed to grab myself an m3 256gb 😎

Yes ! Paid a premium - but not insane . Will copy everything over from 96gb and reckon I can sell that at near what I paid on eBay . Last time I checked there were zero m3 96gb on eBay . I’m surprised how capable it is especially after watching all the pointless YouTube videos comparing it against rtx4090 etc - tokens per second blah blah . If the models shit speed does not matter !

Effrctively I’ve now got a near codex workflow locally . Chat is still the boss . It’s SO much more capable than previous version - genuinely surprised. Excellent software engineer / architect for 20 quid a month. Whilst frontier is clearly more capable, local is catching up real fast !

[Pocster]

Avalon wise made significant progress .

 

What I struggled to match is Alexa’s shear speed “ Alexa , what’s the time in Tokyo ? “ 

Virtually instant response . Hard to find exactly how Amazon structure this . A voice response or acknowledgment just a few 100ms slow can feel painful .

So ! We have Avalon ‘skills’ I.e fuzzy word match for speed . So “ Avalon , what’s the time in Tokyo ? “ , is super close to Alexa speed .

Whisper transcribes realtime . Wake word software was all too slow .

We have a weighted graph of probability and misread / mis said word library - so that’s far quicker than an llm trying to work it out .

Chat’s code was a spaghetti mess - so I got it to rewrite as a finite state machine - easier for debugging . We also ‘smoke test ‘ so input can be a text of words , mis spellings etc .

Really happy with “ what’s the time ? “ and “what’s the time in Tokyo ? “ - the fraction of delay or not defining the question .

The time/date skill obviously of little use but it’s a good test for the pipeline .

Next will probably be a ‘general knowledge’ skill I.e anything you ask goes to internet to get answer .

Edited Saturday at 10:49 by Pocster

[Pocster]

SWMBO had a hip replacement today so she’s stuck at home with me for 8 weeks . I had to promise her I would not talk about local llm . That’s hard I.e impossible…. Thank goodness for ChatGPT 😊

[SimonD]

On 14/05/2026 at 14:03, Pocster said:

This is why I love chat - we literally discuss the project . I’ll sometimes mention something it hasn’t thought of - which in turn takes it down a different path .

Now I have my own mcp tools I can get exactly what I want from local llm on the project . Then send it to architect boss chat to review ! . If boss says it’s good I let local implement code changes ! . 
It’s all quite bizarre as I never look at the code even though I’m an ex soft engineer !

It’s management of llm ‘s !!! 

Honestly, I haven't been brave enough to give it access and control of my system and docs. If I set up a dedicated machine, I might do that. But yeah, it's amazing. This morning I was working through a problem and within 25 minutes I've got a working app that does exactly what I need it to do and I've haven't touched a single piece of code, job done. But I'm still surprise you haven't had to touch anything. I've still got to roll up my sleeves every now and again but maybe that's because I'm restricting access?

[Pocster]

9 minutes ago, SimonD said:

Honestly, I haven't been brave enough to give it access and control of my system and docs. If I set up a dedicated machine, I might do that. But yeah, it's amazing. This morning I was working through a problem and within 25 minutes I've got a working app that does exactly what I need it to do and I've haven't touched a single piece of code, job done. But I'm still surprise you haven't had to touch anything. I've still got to roll up my sleeves every now and again but maybe that's because I'm restricting access?

Yeah it’s insane ! Today we are going to implement a local version of codex

chat writes prompt 

local reasoner proposes how to do it

local coder implements

local reasoner checks it ; tests 

if fails reasoner adjusts prompt and around the loop we go 🤣

Decided 3 attempts then it fails and we give it to big boy chat . Obviously no commits or anything until proven correct .

The thing that still blows my mind apart from zero coding is that I can create whatever I want . No deciding which languid might have to learn or software stack documentation to plough through , heaven forbid stack overflow !

The prompts from chat are very precise - no running amok across the entire repo. One small task at a time rather than big chunks - all tested . Then ‘ magically ‘ the final bit is added and it works !

[-rick-]

2 hours ago, Pocster said:

The prompts from chat are very precise - no running amok across the entire repo. One small task at a time rather than big chunks - all tested . Then ‘ magically ‘ the final bit is added and it works !

 

Just looking at other peoples experience. It works until it doesn't and then randomly decides to delete a load of stuff.

 

Don't give AI access to your main machine. Give it a VM with controlled access. Regular backups beyond the control of the AI.

[Pocster]

11 minutes ago, -rick- said:

 

Just looking at other peoples experience. It works until it doesn't and then randomly decides to delete a load of stuff.

 

Don't give AI access to your main machine. Give it a VM with controlled access. Regular backups beyond the control of the AI.

I watched an interview of open ai to journalist . Journalist asked “ How far are we away from no human coding ? “ . 
Open ai guy asked the room of spectators “ put your hand up if llm writes 100% of your code “ . He then asked “ put your hand up if it writes around 50% of your code “ . He then turned back to journalist and said “ that was roughly a 50/50 spilt so we are 50% of the way there “

No humans allowed soon!

Edited 3 hours ago by Pocster