Ramblings on automatically reading meters

[TerryE]

I use my electricity consumption actuals the help optimize my usage.  I currently have an OVO account with a Smart Meter, and OVO provides decent online reporting of usage by 30min reporting period.  A little bit of retro-engineering of their undocumented AJAX API gave me a daily batch script that run downloads these data into my HA MySQL DB (see this OVO Post for more details if you are interested).  Unfortunately my fixed rate contract is coming to an end and I am now looking at OVO's over 30% price hike -- ouch -- so it's time to move supplier again, and this time I plan to switch to Bulb which is about 20% cheaper than OVO.  The downside is that they are slow in rolling out smart metering, so I am going to lose my ½-hourly rate feed for the foreseeable future.  Damn!  What to do?

 

So I am planning to implement a meter pulse counter.  All modern electrical meters use a LED to flash at either 0.01 or 0.001 kWh unit intervals. (Actually most smart meters measure kVAh not kWh, though in our case nearly all of our load is resistive so the two are pretty close, but that's another story).  So how?

 

• The key component here is an LM393 Light Sensor Module, which cost £2-5 on eBay.  the LM393 is an analogue comparator and this little module has a trim-pot on it so you can set the analogue output threshold on the diode output to convert it into a clean digital signal for input into a GPIO input.
• The next thing is a WiFi enabled micro to do the pulse collection and reporting, such as an ESP8266, ESP32 or RPi0B.  in my case this will be an ESP8266.
• The micro can either count the pulses directly or you can stick a counter in the way.  I good choice here is the PCF8593P. This is primarily used as an RTC, but it also has an event counter mode which accumulates pulse falling edges in an internal BCD counter that is accessible to the micro via I2C.  Hence the micro can deep sleep for 30 mins; wake up and read the event count, connect to the Wifi and report the count to my HA RPi3 and go back to sleep again.  Don't use a Maxim DS1682 as some articles suggest as this powers up and load the count from EEPROM on the rising edge, then stores the new count back in EEPROM on the falling edge, and this read/store is done every event.  This make the device ultra low power and really good for applications such as measuring uptime and up events for devices such as vending machine, but the EEPROM only has a nominal 50K write cycles.  Do the math: for this application, the DS1682 will only last a few days before the EEPROM fails.

 

The ½hr cycle for the ESP8266 makes it very low power, but the kicker is that the PCF8593P has a nominal power rating of 30mA, so the circuit has just too much base draw for battery power -- a couple of NiMH batteries with a capacity of 5000mAh will last about a week. ?

 

So I will need to feed power into my external meter box to run the data collector.   Bugger.   It looks as if I am going to have to bring forward my plan to have a 12 or 24V 5A power bus for all of my garden and yard projects.  60W peak should be quite enough for most if not all of of my planned projects, and 12V is nice to work with thanks to availability of very low price and efficient Buck step-down DC-DC convertors; it is also safe enough to run around the garden with minimal or no safety protection.   

 

Comments and advice gratefully accepted.

[Ed Davies]

Could you have the pulse detector directly wake the ESP8266 then store the count in whatever registers are preserves in deep sleep? AIUI (not actually programmed either myself, yet) the ESP32 is better at wake up from external signals and simpler to preserve data during sleep so might be worth investigating. Most of my limited understanding comes from this video and a couple of others by the same chap:

 

 

[MikeSharp01]

There must, in the modern world, be a low power equivalent of the 

1 hour ago, TerryE said:

PCF8593P

Must there not. How about the maxim  DS1371 which has 800nA current demand and looks like it is RAM focused rather than Eeprom. Eds idea could work as well but it may rum up agains start up latency at high power demand as pulses cone thick and fast.

 

 

[TerryE]

ED,  TerryE is also one the core developers of the NodeMCU Lua firmware for the ESP8266 / ESP32, so I am also a very useful resource to know when it comes to programming ESPs and this is also why I prefer to use them over Atmel or RPI0 devices for IoT applications

 

Deep sleep on both the ESP8266 and the ESP32 powers down everything apart from the RTC and a small RAM scratch pad in it.  Wake from Deep Sleep is effectively a cold boot, except for any context saved in the RTC RAM and in the Flash SPIFFS.  The power draw here is ~20μA. 

 

I have yet to check whether pulses are at 0.001 or 0.01 kWh but if they are at the former then a power draw of 30kW through the meter (which is entirely possible at peak) will result in 30,000 pulses per hour, or one every 120mSec or so, and a more typical average busy draw of 3kW will still be waking the ESP every sec or so   I need to benchmark how quickly the ESP comes out of deep sleep, but this will be tight and still have high peak draws if it powers the Wifi modem.  Perhaps a better alternative here is the light sleep option which draws about 1mA and this keeps the CPU registers and RAM powered but shuts everything else down.  GPIO events will trigger wakeup from from light sleep and wakeup here takes under 5 mSec.

 

The big advantage of using a separate chip to do the counting is that you only need to wake up from time to time -- say once every 30mins, interrogate the counter and go to sleep again.

 

@MikeSharp01 Looking at the DS1371 datasheet, it doesn't look as if it can be programmed in the mode that I want.  It seems to be driven off the mast Xtal and the counter provides a watchdog function, viz it triggers an event after a preprogrammed number of ticks.  Of course there is always the fallback option of using an ATTiny85 to do the event counting, which is even cheaper to implement and will bring the overall power into the battery domain,  but this is going to involve getting into Arduino development -- ho hum.  

[Jeremy Harris]

You might also want to look at energy metering chips, as used in electricity meters.  They are dirt cheap (around 50p or so) and have pretty accurate 16 bit A/Ds for measuring voltage and current at a fast sample rate (much, much faster than something like the Open Energy Monitor, that uses the Arduino A/Ds).  I'm using an Analog Devices AD7755 (now obsolete, but there are loads of others around) which measures current and voltage right at the meter and sends instantaneous power data back to the house via an 868 MHz data link.  Anything can then use that data.  I have a display unit built in to a plug top that shows import/export plus the power, the house data logging system records the data and also the PV diverter unit uses the data to turn the Sunamp heating element on and off, via a 20 A SSR.

 

It would be dead easy to add one of these metering chips to something like an ESP8266, as the output is a pulse train with the frequency proportional to power plus a pin that indicates direction.  No need to break into the meter tails, either; I use a 100 A current transformer for the current signal and a small mains transformer to give the voltage signal.  This old datasheet for the AD7755 might be of interest: AD7755.pdf

[Guest Alphonsox]

1 hour ago, JSHarris said:

 I'm using an Analog Devices AD7755 (now obsolete, but there are loads of others around)

....

This old datasheet for the AD7755 might be of interest: AD7755.pdf

 

They are still available - I bought a bunch from China via Ebay a month or so ago. The big advantage is that they are still available in DIP packages rather than the more modern versions which only seem to be available in surface mount.

 

https://www.ebay.co.uk/itm/1PCS-AD7755AAN-AD7755-Energy-Metering-IC-with-Pulse-Output-/221119026566?hash=item337bb89586

[Ed Davies]

9 hours ago, TerryE said:

ED,  TerryE is also one the core developers of the NodeMCU Lua firmware for the ESP8266 / ESP32, …

Ah, OK.

 

9 hours ago, TerryE said:

and still have high peak draws if it powers the Wifi modem

Interesting point. Do you have to power the Wi-Fi modem on each wake up?

 

9 hours ago, TerryE said:

The big advantage of using a separate chip to do the counting is that you only need to wake up from time to time -- say once every 30mins,

There's a lot to be said for having higher temporal resolution than 30 minutes. My logging is done at 6 second intervals because that's what my CurrentCost meter gives which might be a bit quicker than is really necessary but still useful. E.g., I can see what the power consumption of the circulation pump and the oil pump and blower on the central heating is as the boiler short cycles. Also, the duty cycle of the fridge. I.e., with higher temporal resolution it's pretty easy to separate out the various power sinks in the house.

 

The CurrentCost only measures current and assumes the voltage so is a bit inaccurate if the voltage is wrong but, probably more significantly, can't measure phase so only approximates VA rather than true watts. Still, that's a separate discussion. Except that if you have a transformer for voltage/phase measurement as @JSHarrissuggests then can't you also power the electronics off that and not worry quite so much about micropower? Seems odd to be worrying about a few mW when there's kW flashing by in the same box.

 

Of course, this might be academic if it turns out your new meter has an LCD display and flashes a blib on that every 1 or 10 Wh. A lot harder to watch.

Edited October 28, 2018 by Ed Davies
Add plot

[Ed Davies]

9 minutes ago, Ed Davies said:

Except that if you have a transformer for voltage/phase measurement as @JSHarrissuggests then can't you also power the electronics off that and not worry quite so much about micropower?

 

To do this in a way which won't cause raised eyebrows perhaps you could get an electrician to put a normal 13 amp socket in the meter cupboard run off an appropriate (or new) circuit in the CU then plug a wall wart with a, say, 12V AC output into it. Half-wave rectify then off-the-shelf regulator or switch-mode down to 5 or 3.3 volts with separate voltage divider for voltage/phase measurement.

[Jeremy Harris]

15 minutes ago, Ed Davies said:

The CurrentCost only measures current and assumes the voltage so is a bit inaccurate if the voltage is wrong but, probably more significantly, can't measure phase so only approximates VA rather than true watts. Still, that's a separate discussion. Except that if you have a transformer for voltage/phase measurement as @JSHarrissuggests then can't you also power the electronics off that and not worry quite so much about micropower? Seems odd to be worrying about a few mW when there's kW flashing by in the same box.
 

 

Yes, you can.  The only thing to watch is the effect of small changes in current demand by the electronics, which changes the sampled voltage.  I got around this by using a transformer with two separate secondary windings for the power supply, as the voltage drop is almost entirely due to the winding resistance.  This is the circuit diagram for my unit; the PIC outputs serial data to the wireless link.  The local display port connects to a small LCD with another PIC that translates the frequency and direction signals into import/export power:

 

[TerryE]

I think that we've split the discussion in to to separate usecases:

• The ability to collect periodic aggregate power usage -  say every half hour -- mainly so you can automatically monitor and plan your ToD usage and overall power billing.
• The ability to collect fine grain usage so you can get a perspective on individual device usage.

Jeremy's solution is very elegant but is also invasive in that it involves intervention in the primary power circuit, and this is something that I would prefer to avoid which is why I tend towards pulse counting as this still gives me 0.001kWh granularity.

 

In my view, the whole issue of designing power constrained devices is interesting.  A typical gaming PC will draw perhaps 300W, a modern laptop with SSD perhaps 20-50W depending on the processor, a RPi about 10W.  A pair of NiH battery might has ~ 5000mAh capacity and if you want the batteries to last 3 months, say then the average draw of your device must be around 1.5mA.  Whilst ultra-low power devices do exist, many active components have non-trivial draws.  For example the PCF8593 has a typical draw of 300mA; the LM393 200mA; the light sensor diode 100mA, so the only way that you can run an device for months on a battery is to do a repeat of <sleep a long time at 10s of μA> then <wake up for a short  time and do some measurements>.

 

So IMO, battery powered devices are limited to a small number of periodic monitoring applications.  But it is pretty straightforward designing IoT devices with a ~1W power draw. This is why I am currently thinking of stringing a 12V supply around my house exterior and garden to power my devices as needed.  

[Jeremy Harris]

56 minutes ago, TerryE said:

Jeremy's solution is very elegant but is also invasive in that it involves intervention in the primary power circuit, and this is something that I would prefer to avoid which is why I tend towards pulse counting as this still gives me 0.001kWh granularity.

 

No, it doesn't require anything other than clipping a current transformer core around the meter tail.  I have the power cable for the unit hard wired to a 6 A DP RCBO, but it could just as easily be wired to a 13 A plug to power the unit and obtain a voltage reference.

 

This is the clip-on current transformer I used, which is similar to those used by PV diverters: http://vctec.co.kr/web/product/yhdc/pdf/SCT013.pdf

 

Here's a photo of the (blue) clip-on current transformer in the meter cabinet:

 

[Guest Alphonsox]

49 minutes ago, JSHarris said:

 

No, it doesn't require anything other than clipping a current transformer core around the meter tail.  I have the power cable for the unit hard wired to a 6 A DP RCBO, but it could just as easily be wired to a 13 A plug to power the unit and obtain a voltage reference.

 

This is the clip-on current transformer I used, which is similar to those used by PV diverters: http://vctec.co.kr/web/product/yhdc/pdf/SCT013.pdf

 

Here's a photo of the (blue) clip-on current transformer in the meter cabinet:

 

 

How easy was it to modify the internal resistors in the current transformer? The ones I've come across seem to be entirely sealed.

 

[Jeremy Harris]

2 minutes ago, Alphonsox said:

 

How easy was it to modify the internal resistors in the current transformer? The ones I've come across seem to be entirely sealed.

 

 

Dead easy on this one, as the case is just clipped together.  Inside there is a small circuit board with the burden resistor and lugs and strain relief for the two core cable that's normally fitted.  I removed the cable and burden resistor and fitted a three core screened cable and two burden resistors in order to get the balanced signal needed to feed the AD7755.  These current transformers are readily available on eBay, just search for SCT-013 and loads will pop up, with different current ranges.

[MrMagic]

Regarding the two use cases - 

 

For standard, long term meter readings - good ol Excel at the moment, manual monthly readings. Smart meters due to be fitted in the next few weeks.

 

Live/graphed power usage - Loads of options for 'CT' clamp based monitoring but went for a simple Pi+Hat - http://lechacal.com/wiki/index.php/RPICT3V1 and http://lechacal.com/wiki/index.php/Raspberrypi_Current_and_Temperature_Sensor_Adaptor - I chose this one for simplicity, it presents itself as a serial device on the Pi which I can just open and monitor with a bit of python. This one also seems to be more accurate by reading the actual AC voltage from an AC-AC adapter (but the maths here is beyond me).

 

I like the idea of a 5V bus - I'm getting to the point where I'm starting to use good quality multi port USB chargers now that USB is almost becoming defacto for low voltage peripherals. 

[Ed Davies]

Thank you @MrMagic that LeChacal range looks interesting.

 

Just for giggles I integrated the output of my CurrentCost meter for the year getting a total of 5001675759.992456 watt·seconds (joules). Dividing by 3.6 million to get kilowatt hours that's 1389 kWh. From a meter reading this evening comparing with one done at 22:09 on the first of January I've used 1509 kWh. So the CurrentCost underread by about 8%. That's actually closer than I thought it would be.

[TerryE]

1 hour ago, MrMagic said:

I like the idea of a 5V bus

5V is a bit marginal unless you go for heavy duty cabling. 60W of devices will require 12A and remember that at a PD of 5V, the diversity will become a big issue. IMO,  better to use 12 or 24V and per-device Buck step-down DC-DC convertors (which cost pin money) to drop the voltage to 5 or 3.3V as required.

[MikeSharp01]

Sorry, just catching up with things after a very hectic rush about weekend and aftermath.

 

On 28/10/2018 at 21:24, Ed Davies said:

So the CurrentCost underread by about 8%. That's actually closer than I thought it would be.

I guess that is why using the 'blink' has merit as it should be exactly what your meter reads and you can get the actual current consumption from the blink interval, should you wish to go in real time mode rather than only waking up every 30 seconds as @TerryE system does. 

[Jeremy Harris]

An alternative for reading accurate data is to get a cheap (~£30) second meter installed at the same time as your CU is fitted.  I've done this in the house, fitted an Elster A100C meter (https://www.jsgsolutions.co.uk/kwh-meters/elster-a100c-kwh-meter-with-pulsed-output-single-phase.html ) in the tails that feed the house CU. 

 

The Elster A100C was chosen because it has an IrDA Tx port that constantly transmits data from the internal registers.  It's dead easy to just secure an IR receiver over this port and read data from several registers in the meter that repeatedly transmit serial data at 2400 baud, with a start bit, 8 data bits, 1 stop bit and no parity, with short pulse widths, so a pulse expander is needed to translate the data to RS232-type serial. 

 

The data transmitted is a 110 byte frame that includes the meter make, model and serial number, total accumulated import W/h at rate 1, a reserved data register for rate 1, total accumulated export Wh for rate 1, total accumulated import W/h at rate 2, a reserved data register for rate 2, total accumulated export Wh for rate 2, reserved data, status, error, anti-creep, rate 1 time, rate 2 time, power up time, power fail time, watchdog, reverse warning flag, reserved data.  By parsing the meter make and model data it's easy to determine the order of the transmitted data, and using this meter as a data collection device has the  advantage that if any data logging system glitches no data will be lost, as the registers will just spit out all the data again next time around.

 

Here's the full data sheet on this meter, including a description of all the data that it squirts out: A100C_Operating_Instructions.pdf

 

[TerryE]

As far as the external power bus goes, my current thinking is to use something like 20awg multistrand twin.  Tuofeng do a decent cable that gets 4½+ ? reviews. This has a resistance of 1 ohm / 30m, which gives an acceptable diversity for the size of my garden.  This will get enough power to any of my external projects, including monitoring my supply meter.

Edited November 2, 2018 by TerryE