Pocster

Going amazingly well! Swapped to pi harness with custom code around it. Codex was dropping connection and being a dog - local YEAH! BS Pi is 100% local. So chat produces a scout prompt. I run it with 1 keypress from custom harness and pi. Upload back to chat if it passed/failed all the tests. If it's good chat does patch. Another single menu prompt and patch applied. Upload back to to chat if passed/failed. This loop is going REAL good. Took a few days to get the harness right with various models and bugs Treating chat as software engineer/architect is the way to go. Alexa's timer feature for example basically added in under 1 hour. - a good test!. Not touched or even look at 1 line of code.

Rabbit hole of rabbit holes! Tried most models ollama and mlx. Annoyingly things like DeepSeek cant do tools!! - just a friggin chatbot! codex,pi, continue cli, our own harness. Round and round we go! Getting largely no where! So! Conclusion! No local LLM is capable of producing the code and understanding my repo. Created a new harness using codex which produces an evidence pack that I uplaod to chat. Chat then does a patch. The new harness has a simple menu to save infinite copy n paste. Unsurprisingly codex works best with OPenAi models.... whom would of thought!

Hmmmm . Rather than reinvent the wheel tried codex and continue cli . Didn’t like them acting on the repo ! So we are back to “ Avalon forge “ as named by ChatGPT . Lots of guardrails , reviews etc - a “ cage “ as chat calls it . Still testing and taken 2 days to get this far with it . Codex by the way isn’t entirely local . Will use local llm , codex installs local but still goes off to OpenAI for “ things “ ….

Yeah there's z lot of fix a problem. Depends on the prompt. Some of these "tests" are to simplistic. I mean I'm writing the entire thing using chat - not claude code or a dedicated coder model. Strangely enough we've jettisoned qwen coder as it wasn't good enough ( I don't get why it couldn't do it ) - but there you go. So you build a harness and you stick a model in and see.

I watched an interview of open ai to journalist . Journalist asked “ How far are we away from no human coding ? “ . Open ai guy asked the room of spectators “ put your hand up if llm writes 100% of your code “ . He then asked “ put your hand up if it writes around 50% of your code “ . He then turned back to journalist and said “ that was roughly a 50/50 spilt so we are 50% of the way there “ No humans allowed soon!

Yeah it’s insane ! Today we are going to implement a local version of codex chat writes prompt local reasoner proposes how to do it local coder implements local reasoner checks it ; tests if fails reasoner adjusts prompt and around the loop we go 🤣 Decided 3 attempts then it fails and we give it to big boy chat . Obviously no commits or anything until proven correct . The thing that still blows my mind apart from zero coding is that I can create whatever I want . No deciding which languid might have to learn or software stack documentation to plough through , heaven forbid stack overflow ! The prompts from chat are very precise - no running amok across the entire repo. One small task at a time rather than big chunks - all tested . Then ‘ magically ‘ the final bit is added and it works !

Migration was relatively simple . But now I’ve got more ram want to be like a sad YouTube video and compare gpt120b against mlx 120b for speed . In theory looking for 20%+ as a pessimistic hope !

I think a 3d hologram would be better . You’ll have to give technical details on how to achieve this !

Hmmmm Maybe simplicity is best ? Rotatory actuator driving a circular platform ?

?

Ooooo ! Looks a bit clunky ! Suppose could “ somehow “ use any ptz camera ?

This one is for you engineer / mechanical brain nerds - got @Onoff written all over it but appreciate input from anyone ! So for my Avalon project I had another brain spasm at 3am ! I’ve order a circular screen so it can have a proper modelled/rigged face . I intend to also have Face ID and tracking also . So …. It only makes sense the face follows you round the room 😆 I thought maybe a sphere on some kind of platform that can rotate ( all axis ? ) . So screen faces user . Maybe camera mounted in the front to help with face tracking . But how to make ( 3d printer here I come ! ) a sphere smoothly move around ? . 2 rubber wheels in base ? One north to south other east to west to give rotation in all possible angles ? . Esp32 to control I presume . But it’s how to make the mechanical part I need help with !

SWMBO had a hip replacement today so she’s stuck at home with me for 8 weeks . I had to promise her I would not talk about local llm . That’s hard I.e impossible…. Thank goodness for ChatGPT 😊

Avalon wise made significant progress . What I struggled to match is Alexa’s shear speed “ Alexa , what’s the time in Tokyo ? “ Virtually instant response . Hard to find exactly how Amazon structure this . A voice response or acknowledgment just a few 100ms slow can feel painful . So ! We have Avalon ‘skills’ I.e fuzzy word match for speed . So “ Avalon , what’s the time in Tokyo ? “ , is super close to Alexa speed . Whisper transcribes realtime . Wake word software was all too slow . We have a weighted graph of probability and misread / mis said word library - so that’s far quicker than an llm trying to work it out . Chat’s code was a spaghetti mess - so I got it to rewrite as a finite state machine - easier for debugging . We also ‘smoke test ‘ so input can be a text of words , mis spellings etc . Really happy with “ what’s the time ? “ and “what’s the time in Tokyo ? “ - the fraction of delay or not defining the question . The time/date skill obviously of little use but it’s a good test for the pipeline . Next will probably be a ‘general knowledge’ skill I.e anything you ask goes to internet to get answer .

No wonder there’s a (expletive deleted)ing shortage ! Guess what @-rick- …. 😊 . Managed to grab myself an m3 256gb 😎 Yes ! Paid a premium - but not insane . Will copy everything over from 96gb and reckon I can sell that at near what I paid on eBay . Last time I checked there were zero m3 96gb on eBay . I’m surprised how capable it is especially after watching all the pointless YouTube videos comparing it against rtx4090 etc - tokens per second blah blah . If the models shit speed does not matter ! Effrctively I’ve now got a near codex workflow locally . Chat is still the boss . It’s SO much more capable than previous version - genuinely surprised. Excellent software engineer / architect for 20 quid a month. Whilst frontier is clearly more capable, local is catching up real fast !

This is why I love chat - we literally discuss the project . I’ll sometimes mention something it hasn’t thought of - which in turn takes it down a different path . Now I have my own mcp tools I can get exactly what I want from local llm on the project . Then send it to architect boss chat to review ! . If boss says it’s good I let local implement code changes ! . It’s all quite bizarre as I never look at the code even though I’m an ex soft engineer ! It’s management of llm ‘s !!!

Erm! LOL! talk about confusing. We made good progress. But then we start hitting chatgpt limitations i.e. cant see repo so gets a zip. Its patches drift and we end up with a mess. SO! Back to local llm (gpt-oss 120b for now). I've got a headache and I haven't written any code! Get chatgpt to talk me through config in VS I have never heard of. Then chatgpt wants extra tools to make my life easier ( apparently ) so we install and setup a mcp server (???) - bit lost on what and why. None the less magically continue now has new special tools. Still don't get it as continue has basic tools but they all seem problematic e.g. can only open and read small files....??! Clearly beyond my pay grade but moving forward; moving forward even though I don't know where we are heading!!. Guessing you claude code guys get all this nicely packaged for you. If nothing else chat is taking me into yet another rabbit hole!

But yeah it is a bit benderish

Ummmm that’s a not allowed in snowflake world statement . You’ll get in trouble from the gender fluid police !

“ completed “ as in my definition…..

Hmmmm . Proving tricky ! After a lot of talking we have a “ Avalon “ wake word with pretty good success . But radio on etc not so great . Trying to access lms streams and use that to filter my speech from radio not producing good enough results . Also depends where I am in the room - we use direction of speech to aid if I said or radio said it . Solution seems be another uma8 placed on the opposite side of the room ….

An army of them ! Controlled from my llm ! Yes !

I want this . I don’t know why as my builds complete , but I want it

Synthetic samples: 10,000–100,000 Your real samples: maybe 50–300 Household/background negatives: hours if possible 😊 so I only have to speak maybe 300 times . Designing a system for this data now .

Good progress today ! We use the default “ hey Jarvis “ to start stt . Rolling buffer so stt starts as soon as wake word is done “ . Works well . Deliberately have radio on in background. Alexa struggles with this sometimes of course . Uma8 can determine direction of sound so even with radio on I’m getting good results for stt . So then I do the easiest cheat . Hear wake word , mute radio then reinstate once sentence finished . Unlike Alexa a “ hey Jarvis …. “ sentence can be as long as you like no fixed listening window . Now though I want “Avalon” to be wake word and also the fact it recognises that Pocster said it . We’ve planned a schedule where I say “Avalon ..” with different phrases and at different points in the room with and without background noise . Reckon we aim for 1000 samples . Going to be a long next few day 🤣