Home Automation System - Two PZEM-004T and a single ESP8266

After my successful experiments with the PZEM-004T and a Nodemcu ESP8266 aim to monitor power consumption (devices are actually working well in my ground floor power monitoring system, see https://domoticsduino.blogspot.com/2018/05/home-automation-system-energy.html), I decided to update my photovoltaic production monitoring system, actually based on a Wifi 433Mhz receiver (see https://domoticsduino.blogspot.com/2017/08/home-automation-system-mqtt-and-rf433.html).

Actual system has a problem due to an invalid read of power values when the inverter is switched off. In fact it gives me about 250W power production...

My idea is to use the same devices used for the ground floor power monitoring system...so a PZEM-004T and a Nodemcu ESP8266.

These are the devices:

NODEMCU ESP8266

PZEM-004T

Because my inverter is near the electrical panel in the garage, I decided to let the things more complicated adding another PZEM-004T to also monitor the basement consumption with the same ESP8266.

Then I added also a temperature and humidity sensor to the same circuit.

My following step was to check that a single NODEMCU ESP8266 was able to control both PZEM-004T...My tests were successful

Here is it my first PZEM-004T linked to the main output cable of the inverter...on the right you can see the DHT11, temperature and humidity sensor (it's a not so accurate sensor, but it's right for my kind of measurement...)

This is the second PZEM-004T, linked to the main cable of basement power, inside the electrical panel

Complete view:

Now it's time to write the software and test it.

I decided names for the MQTT topics and I created items and sitemap objects for OpenHAB

This is what we could see in my OpenHAB BASIC UI

Energy Consumption value is the sum of all the sensor power (2 of them are already running)
Values are different related to the previous screenshot because they continuously change

Last step: I created some charts in the Grafana dashboard

My primary target was reached: my production monitor give about 0W when the inverter is switched off

Home Automation System - My first sensors...

After all my experiments, now I'm able to use OpenHAB with familiarity; so now it's time to build my first sensors.

Let's start from the kitchen where I would like to monitor gas, temperature and humidity.

The MCU will be a ESP8266 NodeMcu board. I chose this board because it has wifi, several digital I/O and an analog one, you can write firmware in the same way you could with Arduino and above all...it's very cheap...On Banggood I bought two at only € 2,76 each, not bad...

If you are interested in it, this is the product link:

Geekcreit-Doit-NodeMcu-Lua-ESP8266 on Banggood.com

I will use these sensors:

- MQ-4 gas sensor: for detecting of methane mainly, (I'm interested on it...), propane and butane gas. I already used it months ago. Here is it:

MQ-4 gas sensor on Banggood.com

- Humidity / Temperature sensor DHT11: it'a a very cheap sensor (with about €5 i bought 5); its error is +/- 2°C and it's able to get only positive temperature. Its precision is 1°C, so no decimal number...For these limitations I decided to buy other sensors, the ds18b20 (for internal and external use)...I will use these DHT11 just for humidity

5 pack on Banggood.com

3 Pack on Banggood.com

Single sensor on Banggood.com

- Dallas ds18b20 temperature sensor: compared to the previous one it has a better precision (0,5°C) and it's able to get negative values, so it's ideal for external use

Single sensor on Banggood.com

5 Pack on Banggood.com

External probe on Banggood.com

I built some prototype using a breadboard...

...till I found the right configuration...and so I made a "definitive" board...

I added a led used for the indication of network activity and a connector used for external power. The device will be powered through a little 5V 450mA power adapter, connected to the blue terminal blocks. In this way I could power up the device and also the sensors.

The led, ds18b20 and DHT11 sensors are linked to digital I/O and the MQ-4 gas sensor to the analog one.

Now it's time to write the firmware...I use Visual Studio Code with platformio extension; I think it's better than the Arduino IDE (personal opinion). I chose this IDE because it also has an extension for openHAB that let me edit configuration files in a better way (just like Eclipse SmartHome Designer).

My device will connect to the MQTT broker of OpenHAB through wifi, fetch the data from the sensors and send them as json formatted string with nodemcu/mainfloor/esp8266mcu11/temphumiditygas/state topic.

The sample period is 5 seconds and each 60 seconds the average data will be sent to the MQTT broker. When the data will be sent, the green led will blink twice.

Here is an example of a message sent to the MQTT broker:

{"gas":{"error":"","sensorValue":46.33333,"ppm":0.431016,"percentage":0.000043,"realValue":829.0371},"tempHum":{"error":"","humidity":63.66667,"tempC":22,"tempF":71.59999,"heatIndexC":21.91774,"heatIndexF":71.45234},"temp":{"error":"","tempC":21.66667,"tempF":70.99999}}

The gas object is related to the MQ-4 sensor, the tempHum object is related to the DHT11 sensor and the temp object is related to the ds18b20 sensor.

For now I decided to use both sensors for temperature reading; so I can compare them.

Ok, it seems to work as expected...I put my device in the right place in the kitchen...

Now it's time to setup OpenHAB...and build a new dashboard on Grafana.

In OpenHAB I created a new file called esp8266mcu11.items (esp8266mcu11 is the unique id of the device) in the /etc/openhab2/items folder; in this file I appended the following lines:

Number MainFloor_KitchenTemperature "Kitchen Temperature [%.1f °C]" <temperature>
(gPersistence)
{mqtt="<[openhabianpi-mqtt:nodemcu/mainfloor/esp8266mcu11/temphumiditygas/state:state:JSONPATH($.tempHum.tempC)]" }

Number MainFloor_KitchenTemperatureNew "Kitchen Temperature NEW [%.1f °C]" <temperature>
(gPersistence)
{mqtt="<[openhabianpi-mqtt:nodemcu/mainfloor/esp8266mcu11/temphumiditygas/state:state:JSONPATH($.temp.tempC)]" }

Number MainFloor_KitchenHumidity "Kitchen Humidity [%.1f %%]" <humidity>
(gPersistence)
{mqtt="<[openhabianpi-mqtt:nodemcu/mainfloor/esp8266mcu11/temphumiditygas/state:state:JSONPATH($.tempHum.humidity)]" }

Number MainFloor_KitchenGasPpm "Kitchen Gas [%.2f ppm]" <carbondioxide>
(gPersistence)
{mqtt="<[openhabianpi-mqtt:nodemcu/mainfloor/esp8266mcu11/temphumiditygas/state:state:JSONPATH($.gas.ppm)]" }

Number MainFloor_KitchenGasValue "Kitchen Value [%.2f]" <carbondioxide>
(gPersistence)
{mqtt="<[openhabianpi-mqtt:nodemcu/mainfloor/esp8266mcu11/temphumiditygas/state:state:JSONPATH($.gas.realValue)]" }


Now the sitemap:

Frame label="Gas Detection" {
Text item=MainFloor_KitchenGasPpm valuecolor=[>800="red",>600="orange",<=600="green"]
}

Frame label="Kitchen" {
Text item=MainFloor_KitchenTemperatureNew
Text item=MainFloor_KitchenHumidity
}

And this is the result:

Good, I added a new brick...now I can create the new Grafana dashboard with the gas, humidity and temperature of the kitchen:

Last step: adding the device to the network monitoring...so OpenHAB will notify me when the device will be offline.
So, using PAPERUI I added the device using the Network Binding and create the items. Then I added the gNetworkCheck group to the "status" item, so its network presence will be monitored by OpenHAB.

Rf 433 Mhz data "sniffing"

It's been a long time when I wrote my last post...My job and family matters keep me far from my hobbies...priorities won

As some of you could remember (see post http://domoticsduino.blogspot.it/2016/03/data-acquisition-photovoltaic-production.html), I acquired the production of my photovoltaic plant using the OWL CM160 wireless receiver, plugged into the USB port of Raspberry.

Unfortunately, this system was not so stable as I wished, because of the USB driver...and so Raspberry needs to restart...not so convenient...isn't it?

I found an internet blog where someone was able to sniffing wireless data packets trasmitted by the sensor to the OWL, through a simple wireless receiver, as the one I used to receive the gas sensor data...(see post http://domoticsduino.blogspot.it/2015/08/data-acquisition-receiver.html)

This it the article:

https://github.com/onlinux/OWL-CM180

It's using a CM180 wireless receiver, but the explanation works well for the CM160 too.

These systems use the Oregon Scientific communication protocol...My CM160 use the version number 3...

Here is a doc about Oregon Scientific protocol...

DOWNLOAD

Data is transmitted using che Manchester OOK (on off keying) code at 433 Mhz

So I connected my wireless receiver to an Arduino UNO and I downloaded the sketch which allows me to decode the OOK transmission. Playing with bits I was able to decode the transmitted value of the solar power...

Some example in hexadecimal chars:

2a8081bf01501a011900 ==> power 434 W
2a81817f02d0fdf71800 ==> power 627 W
2a81813f0980f3b71300 ==> power 2366 W

Output on the serial port

The next step will be the transmission of received data to central server

Sniffing all the 433 Mhz packets, I noticed that there are packets with a different format...these packets may be transmitted by other sensors. One of these sensor could be the temperature gauge, placed outside the house. At this time I'm not able to decode this particular signal...maybe in the future??

Data acquisition - Photovoltaic production

Another kind of data I would like to acquire is the production of my photovoltaic plant, a 3 kwp plant built over my roof and made by SUNAGE 250 swiss modules and AROS SIRIO inverter.

I also received a little energy monitor tool, which allows me to verify power production through a wireless display. This display is able to store data in a little sqlite database. Through a USB connection, it lets me view and download data using a PC.

This tool is called OWL

http://www.theowl.com/energy-monitors/standalone-monitors/owl-usb/

My model is the OWL+USB CM160.

The OWL has a little bug because it also detects reactive power when the inverter is switched off (during the night for example...)

For my purpose I have to connect OWL to Raspberry. Unfortunately this is not supported and there aren't official drivers. Making some research over the internet, I found a good linux open source interface for OWL: EAGLE-OWL https://github.com/cornetp/eagle-owl

I compiled and installed this software directly into the Raspberry. It is a background process which detects the CM160 e gets the database data.

This is the shell of Raspberry where you can see the eagle-owl software and database files.

This process will run automatically on startup. Then I wrote a little php script which is able to read CM160 database file e put its data into mysql. In this way I could store all historical data which will be displayed through a web interface.

This script will run on Raspberry every minute (standard cron task).

Data Acquisition - Remote Gas Detector

Now, it's time to connect a remote sensor to my new receiver and begin a real data acquisition.

I think I'll go back to my sensor gas prototype...I already made some experiments with it:

First experiment: a simple gas detector

I'm going to use it as a remote gas sensor with MQ-4 [Datasheet].

and a wireless RF-433 transmitter.

with a lot of wire, led and a push button to reset sensor status.

HARDWARE

This is the fritzling design:

As well as Wireless transmitter and gas sensor, we could see a red led, a green led and a RGB led. On the left side of the schema, there is also a push button. Led are powered by Arduino 5V power line, protected by resistors...Arduino is powered by a 12V transformer just like the wireless transmitter, to ensure the best transmission power.

PIN 11: push-button

PIN A1: analogue pin linked to GAS sensor

PIN 9: wireless data transmission

PIN 12: red led for wireless activity indication
PIN 8: green led for data acquisition activity
PIN 3, 5 and 6: RGB led for showing system status
 - off: normal values;
 - yellow: an out of tolerance value under the danger threshold;
 - rosso: an out of tolerance value over the danger threshold;

Here is it:

FUNCTIONALITY AND MESSAGES

When the system is powered up, the gas sensor warm up stage starts; this is necessary to ensure better precision...during this time, RGB led blinks.

Then, data acquisition starts.

Every second the system gets 10 samples data from the sensor (the green led flashes every second) and calculates the average...Every 5 seconds the average of the collected values is sent to the receiver (the red led flashes every 5 seconds).

When the average is out of tolerance, RGB led lights up.

The push buttons reset the system status and turns off the RGB led. If a value is out of tolerance and then other samples return to normal values, the led still lights on...to get the attention...only through the push button, led could be turned off.

The wireless data sent every 5 seconds is a text string with the following format:

GAS;VALUE;STATUS|

VALUE is the calculated average, STATUS is a numeric value indicating the system status according to the following schema:

 - 1: ok;
 - 2: warning
 - 3: danger

SOFTWARE

You can download the complete Arduino Sketch from here:

DOWNLOAD

As for the receiver, I used RADIOHEAD libraries to manage wireless transmitter.

Setup:

- wireless receiver setup;

- status led setup;

- MQ-4 sensor warm up;

Main loop:

- First, the system checks the push button status...If it was pressed, the variable which manages the system status will be set to OFF;

- Then, the system read 10 samples from the gas sensor and calculates their average; during this time the green led lights up (PIN 8);

- Every 5 seconds the system calculates the average and set the system status according to this value; then, this value is sent to the receiver;

- In case of out of tolerance value, RGB status led (PIN 3, 5 e 6) will light up;

- The average value is sent to the receiver; during this time the green led (PIN 12) will light up;

And finally, here is a short video:

Data acquisition - The Receiver

Let's get started with the first component: The Receiver.

Here is it still as a prototype

It's made up by an Arduino Uno Board, with the ethernet shield to get network access and the wireless RF433 receiver

Functionality and communication protocols

The wireless receiver receives messages sent by sensors and then route them towards WEB/APP server, using http protocol. It acts as a simple router.

Communication protocols are very simple and currently not secure, so for now this system cannot be in a public network.

The message sent by sensors is a character string made up in the following way:

SENSOR;VALUE;STATUS|

where SENSOR is the name of the sensor; VALUE is the current value of the sensor and STATUS is a particular condition for the sensor (DANGER, WARNING, and so on...)

The receiver gets this string and send it to the WEB/APP server, using the following HTTP GET request:

http://SERVERNAME/data.php?u=PASSWORD&d=STRING

where SERVERNAME is the domain in which WEB App is installed, PASSWORD is a secret key known also by the WEB/APP server and STRING is the sensor's data.

As already said, communication protocol is very simple and without ACK...for now it's just a prototype and it aims only to debug system

HARDWARE

Fritzling Schema (approx)

PIN 2 OUTPUT for RF status led (RED), reporting wireless activity;
PIN 3 OUTPUT for ETH status led (GREEN), reporting ethernet activity;
PIN 5V to power up wireless received and led;
PIN GND linked to wireless receiver GND pin;
PIN 7 INPUT to receive wireless data;

The Ethernet shield is linked to Arduino UNO; all the system is powered up by external 5V.

SOFTWARE

Setup

- wireless receiver setup using RadioHead library;

- ethernet shield setup using standard Ethernet library; it this case we use a static IP (but you can use DHCP if your network provides it);

- status led setup;

Main Loop

- the main thread wait for a wireless message;

- after received a new message, RF status led lights up and the message is sent to WEB/APP server using HTTP request;

- during the http request, ETH status led lights up and it turns off after the HTTP response;

Here you can find source code for the receiver: DOWNLOAD

And here is a video about the receiver doing its job...

My first project - data acquisition system

I started design my first data-acquisition system with Raspberry PI and Arduino. The idea is to have a sensors network which send data to a receiver which, through ethernet, send these data to WEB/APP server.

The main components are:

- one or more sensors
- a receiver
- a DB Server for storage purpouse
- a WEB/APP server to acquire data and let remote user monitoring the system

Here is the design:



More details:

SENSORS

They can send data in different ways:

- wireless RF-433 towards receiver
- ethernet towards receiver
- USB directly towards server

They can be Arduino board or Raspberry PI or others...
Depending on the type of communication, there can be other interface boards or control led...

RECEIVER

It receives data via wireless RF-433 or via Ethernet
Through ethernet, it transmits data to WEB/APP server, for storage and monitoring purpose

It's built over an Arduino boards with Ethernet Shield and wireless RF-433 receiver. There are also control led.

WEB/APP SERVER

It receives data from receiver through ethernet or from sensor directly connected on its USB port
It stores data on DB server
It runs a web application for remote monitoring

The WEP/APP server software is the world famous Apache HTTPD, installed on Raspberry, running Fedora linux distro.

The monitoring web application is written in PHP / HTML5 using Bootstrap framework, which make it responsive.

DB SERVER

It stores data received from WEB/APP server.

The RDBMS is Mysql Community Edition installed on the same Raspberry of WEB/APP server