DIY Arduino based solar PV dump controller

[ProDave]

I present here my ideas / design for my home built solar PV dump load controller, based around an Arduino microcontroller.  It is not presented as a "design" but rather ideas for you to use or adapt as you see fit.

 

This is my MK 2 design.  For background, my Mk1 version was based on a design by Eric Ward in Jan 2011 which used a single current transformer, and sampled the ac waveform a number of times per cycle to determine if the measured current was being imported or exported.  This worked okay to an extent but some house loads, like the washing machine really confused it and made it think it was importing when it was exporting and vice versa.  I now believe if adopting a single CT solution you should base it upon one of the electricity meter chips which are far more sophisticated to make this measurement accurately.

 

So my MK2 version operates by having two current transformers. One measures the power consumed by the house loads, and the other measures the power generated by the solar PV.  I am able to easily make this split as there is a convenient split at my outside meter box where both the current transformers reside.

 

I now take the current measurement from the transformers, rectify it and filter it so I have a DC voltage that is proportional to the measured current, filtered with a time constant of about 1 second to smooth out transient currents.

 

I use this type of current transformer, available from many suppliers, the cheapest being direct from China on ebay 

 

These come supplied with a load resistor built in, so as presented they provide a voltage output. But because I am sending the current signal some distance over twisted pair telephone cable, I removed the load resistor and fitted that inside my dump controller, so the twisted pair cable is conveying a current signal, not a voltage signal.

 

Now onto the dump controller itself.  In essence it measures house load current and PV generation current. If PV generation current exceeds house load current, it sends power to the immersion heater, switched by a Solid State Relay.

 

Now a bit about switching and metering,  Most electricity meters, measure imported or exported power based on an "energy bucket" and I believe they will only meter whole watt hours passing in either direction.  So if burst firing a 3KW heater, to match say 1KW of surplus PV, you would have to ensure the duty cycle of the immersion heater burst does not allow more than 1 watt hour of electricity to pass before it turns off, and then lets the surplus PV flowing the other way empty the "bucket" again before it is metered.  Based on a typical 3KW immersion heater, that will consume 1 Wh in 1.2 seconds.  So you want the immersion heater on/off cycle to be shorter than that.  So I have based my immersion heater on a half second period.  Within that half second the immersion can be fully on, fully off or on for part of it, depending on how much surplus PV is available.

 

To achieve this, and enable some program timing, you need a reference.  The Arduino as standard does not have a real time clock, though you can buy add on boards to add that function.  However, as a hangover from my Mk1 design, I have a "phase reference" input, which is basically a raw 50Hz ac input connected to a digital input pin.  By sampling the rising and falling edges of that, I can time program operations in 10mS steps for any period that I want. This in effect creates an accurate clock signal for everything.

 

So my Pulse Width Modulation for the immersion heater cycles once every 1/2 second, or a total of 50 10mS program cycles.  This means I can divide it's on / off period into 50 such 10mS time slices and it can be on or off for any number.  So that sets the resolution of different "power" levels that I have available.

 

Other program functions run much less frequent.  The basic sums that calculate the import / export power operate every half a second, and there are some display functions that show values on the screen if hooked up to a PC operate once every second.

 

I also do some data logging that keeps a running total of the immersion heater on time, converted into watt hours.  This gets stored to eeprom once an hour and recovered from the eeprom on program startup.  The once per hour eeprom write is because they have a finite number of write cycles. Also using the eeprom.put()  function, this only writes to the eeporm if the value has actually changed.  I estimate this eeprom write regime should be good for 30 years eeprom life.

 

I have a moving coil meter hooked up, and depending on the position of a switch on the unit, it will either display total import / export as a centre zero meter, or it will just display total PV being generated.

 

Lastly a bit about how to calculate how much power to send to the immersion heater.  You can't just measure the available power and set the immersion heater to that much, otherwise next time the calculation is done, it will measure no power as being available and turn the heater off.  Instead the heater power is incremented if any spare power is available and decremented if used power exceeds generated power.  So it ratchets up and down relatively slowly to obtain balance at which point all surplus is being sent to the immersion heater but no more.

 

And now onto the code.  Pick it apart but please don't be too critical of my programing style, some of the purists will be ashamed at my use of some global variables to save passing values back and forth between functions for example.

 

Remember this is still under development, if I make significant changes I will update this thread,  The code is posted as a .txt file, just paste it into the arduino programing environment.

 

This is written for the little Arduino Nano board, you may need to change some I/O designations if using a different variant.

 

 

arduino_pv_dump_controller.txt

[Onoff]

Respect. Wouldn't know where to start.

[Ed Davies]

That's neat. Worrying about style is a bit over the top for a 200 line program.

 

One thing to think about, though, is the printing baud rate. There are 72 characters of fixed text in the message plus whatever is needed for the actual values. At 9600 baud (960 characters per second) that's at least 75 milliseconds to print so once a second your code will stutter for few cycles. Probably doesn't really matter but not nice. Maybe increase the baud rate.

[ProDave]

Thanks. I will do some timings to see if it really pauses the program or if it all just gets buffered in hardware.

[Ed Davies]

Having written that I did wonder if the baud rate would matter for a purely USB connection. I know it did on the original Arduinos with a separate USB/serial chip from the main processor but I wasn't sure for the modern ones with the USB integrated. I have some newer ones but they're packed away somewhere so I tried with an ESP32 which, though different, is a similar concept from this point of view. Indeed it was slower at 9600, 67 ms to send 70 plus a few characters:

 

18:49:33.538 -> 123456789 123456789 123456789 123456789 123456789 123456789 123456789 647
18:49:33.626 -> 123456789 123456789 123456789 123456789 123456789 123456789 123456789 648
18:49:33.693 -> 123456789 123456789 123456789 123456789 123456789 123456789 123456789 649

 

Indeed, there might be some buffering. Guessing wildly, maybe 16 bytes; that's what the cleverer UARTs on PC had and it's not entirely unlikely that's been emulated down the line.

 

 

[ProDave]

I can't remember why I settled on 9600. As it's an emulated uart via USB I guess I could just whack it up a lot higher.

 

I don't think it will upset program operation, but as it's also the basis of timings, it would make my Wh tally a little inaccurate if the program pauses every second.

[Ed Davies]

FWIW, the ESP32 seems to have 128 bytes of buffering. Print 127 bytes plus newline and it goes “instantly”, 128 + nl and it takes 1 millisecond. 200 total takes 75 ms.

[TerryE]

Some devices such as the ESP8266 and the ESP32 have internal hardware support for serial port FIFOs. I don't think that the Atmel IoT devices do, but someone can correct me if I'm wrong. The standards serial to uart chips typically have a 128 byte FIFO on them and will happily support baud rates up to 256Kbaud. Using this sort of high baud rate makes avoiding timeouts and handling timing within your code a lot simpler.

Edited February 25, 2019 by TerryE

[ProDave]

According to the Arduino reference:

 

"As of version 1.0, serial transmission is asynchronous; Serial.print() will return before any characters are transmitted."

 

But I will increase the baud rate anyway.

[Jeremy Harris]

Nice to have the FIFO buffer, as serial comms, particularly receive, can be pretty clunky without one.  I've been faffing around getting my new EVSE to work, so that it preferentially uses excess PV generation to charge the car, rather than grid power (made more complex because of the way J1772/IEC61851 protocol works, with its minimum 6 A charge rate).  Working around the delays caused by waiting for serial data (my main power monitoring unit broadcasts the import/export true power via a 433 MHz wireless link) has been the most time consuming bit.

[Ed Davies]

1 hour ago, ProDave said:

"As of version 1.0, serial transmission is asynchronous; Serial.print() will return before any characters are transmitted."

 

It doesn't say, though, that it'll return before characters from previous calls to Serial.print have been transmitted. I also doubt that that statement is completely true: if a single Serial.print() call queues more characters than the buffer can hold there's bound to be some delay. I suspect that would be better worded as “may return before all characters are transmitted.” It's just a question of how big the buffer is. If it's the same 128 bytes as the ESP32s, etc, then you're fine. But, yes, upping the baud rate is the simplest way to be sure.

[TerryE]

55 minutes ago, Ed Davies said:

If it's the same 128 bytes as the ESP32s, etc, then you're fine

 

Most of the chips with a H/W FIFO support near full and near empty interrupts so in the case of the ESPs for example, the RTOS and non-OS SDK drivers use an ISR (interrupt service routine) to top-up and/or to empty these FIFOs to / from a program buffer.

[ProDave]

If I am sending of the order of 100 characters once per second, then by the next send a second later, all 100 will be pretty well guaranteed to have gone unless I go down to 75 baud.

[Ed Davies]

Absolutely. But if you're sending at less than about 100 kbaud and the  (not-yet-known) transmit buffer size for the Arduino Nano is noticeably shorter than your message then your code could still be delayed beyond the end of the current 10ms time slot potentially causing a blip in your PWM code or whatever.

 

Here's the Arduino code I was using on the ESP32 to test the buffer size:

char str[200];

void setup() {
  Serial.begin(9600);

  for (unsigned i = 0; i < sizeof(str); i++)
    str[i] = '0' + (i % 10);
  str[sizeof(str)-1] = '\0';
}

void loop() {
  unsigned long start = millis();
  Serial.println(str);
  unsigned long end = millis();
  Serial.print("Start: ");
  Serial.print(start);
  Serial.print(", end: ");
  Serial.print(end);
  Serial.print(", length: ");
  Serial.println(end - start);
  sleep(1);
}

Just tweak the size of str, declared on the first line, to try different buffer lengths until the operations start and end at the same time (“length: 0”).

[ProDave]

Thanks @Ed Davies

 

I ran your code at 9600 and deduced my Ardiono Nano has a buffer of 64 bytes.  Beyond that I start seeing big delays up to about 20mS

 

So baud rate now up to 250000 (the fastest the serial monitor will run at) and I am down to 1-2 mS for my 100 character message

 

Until you mentioned this I had never considered that writing a few messages to the screen might have messed up my timings.

[TerryE]

3 hours ago, ProDave said:

Until you mentioned this I had never considered that writing a few messages to the screen might have messed up my timings.

 

Oh, the wonders of real-time programming. Welcome to the club!

[ProDave]

17 minutes ago, TerryE said:

 

Oh, the wonders of real-time programming. Welcome to the club!

I have done lots of real time programming in previous employment, and to be honest, something that only has to do a few lines of code once every 10mS did not seem a  particularly critical task.  I never expected the serial comm's to be an issue and don't recall it being so in the past. I can only assume the systems I worked on before had larger buffers on the serial ports.

[ProDave]

A bit of an update.  I spent the day sorting out mostly hardware issues.

 

My plan to put the current transformer loads in the plant room, thus sending 2 current signals down a 5 pair telephone cable proved a spectacular failure.  I was getting massive crosstalk between the house current measurement and PV current measurement.  Eventually I reverted to having the CT loads in the meter cupboard and just sending the resulting voltage signal up the phone cable to the plant room.  That seems to work much better.

 

Initial tests showed I needed more filtering of the house current signal. As soon as the first burst of immersion heater firing turned on, the house current monitor was reading that as way more than the average current it was. So some playing with filter time constants was needed.  I also added some software averaging of the house current to smooth out transients.

 

Then I found I had scaled the PV current transformer for a max of 4KW, but scaled the house current transformer for a much higher load. That meant at the sort of values it was working at, it was down in the noise.  So I re scaled that so the house current monitor now saturates at about 10KW and gives better resolution at low power levels.  It is only low (<4KW) power levels that are important for this. The fact that a >10KW load will saturate the monitor is irrelevant.

 

By the time I had done all this the nice sunny morning had passed and it's grey and raining.

 

I await the next sunny day to check it's operation under real conditions.

 

Frustrating business.

[Jeremy Harris]

When I was playing around with the CT burden resistors I found that things worked best with them inside the CT, despite the length of the cable.  Having the burden in the CT case lowers the source impedance of the CT a lot, as the burden resistor(s) are a pretty low value usually.

 

There's also a safety advantage in having the burden resistors inside the CT case, as with no burden the output voltage of the CT can be pretty high.  There's a risk that with the burden remote from the CT, plus a relatively high current in the meter tail, if a connection comes loose then the voltage on the open circuit cable could reach a dangerous level.

[ProDave]

While I wait for the sun to come out....

 

Are there any Linux / ubuntu experts on here?

 

The arduino programming environment has a Serial Monitor built in.  To access it you first open the Arduino IDE then  you can open the Serial Monitor.

 

I want to be able to open the Serial monitor directly.

 

I did this previously on a PC by locating the executable file for the serial monitor and creating a shortcut to it.

 

But I don't know my way around the Linux / Ubuntu file system and a cursory look has failed so far to find where all the Arduino IDE executable files are stored.

 

Any help?

[Jeremy Harris]

You should be able to use pretty much any terminal emulator easily enough.  I tend to use PuTTY, as it has versions for Linux and Windows: https://www.ssh.com/ssh/putty/linux/

 

It's pretty versatile and covers pretty much any use case where you might need a terminal interface.

[PeterW]

That’s one for @TerryE

[ProDave]

I have just been trying PuTTY.  All I get is giberish on the screen.  I have set the Arduino baud rate to 250000, the fastest it will go when @Ed Davies found the buffer / timing issue.  The Arduino Serial monitor works at that.  I wonder if PuTTY only does "standard" baud rates and not 250,000?

 

 

EDIT yes that is it. PuTTY is now working at 230400 baud.  

[Ed Davies]

The classic Unix tool for this sort of thing is cu. Haven't used it myself; these days I'd use screen. PuTTY is sort of a mixture of ssh and screen written originally for Windows (which had neither screen nor ssh) but ported to other systems.

 

Neither the Arduino serial monitor nor screen work for me at 250''000 baud. The Arduino monitor says “Error while setting serial port parameters: 250,000 N 8 1”. Both work fine at 230'400 baud.

[ProDave]

I did try Screen, but it would not seem to recognise the "serial" port dev/ttyUSB0  But anyway PuTTY is working fine.

 

So I can report success on the first day of live testing the dump controller.  After a grey wet start the morning turned out ideal testing conditions with a rapid change from bursts of brilliant sunshine then back to cloud, giving a good chance to see how the immersion power ramped up and down with changing loads and changing generation.  Later the sun came out for longer but behind high thin clouds so  generation has been lower this afternoon.

 

It never quite got to enough surplus power to run the immersion at full power, the highest it got to was 80% duty cycle.  So far today the PV has generated 6.3KWh and 2KWh of that has been sent to the immersion heater lifting the temperature in the HW tank 3 degrees.

 

Total import for the day is registering as 1KWh most of that will have been this afternoon when the washing machine was on, and it's heater power exceeded the generation at the time.

 

Next job is to connect my export meter and see how much power is "escaping" and tweak things a bit if too much is going "to waste"