Skip to content

Home

Jump to bottom Edit New page
Jack Kelly edited this page Jun 8, 2013 · 76 revisions

Table of Contents

Motivation and Aims

I need to monitor the power consumption of every appliance in my home (for my academic research on smart meter disaggregation). I tried using 30 Current Cost IAMs (with 4 EnviR receivers) but this setup suffered from lost data due to RF collisions. So I now plan to use 30 EDF IAMs to monitor appliances, plus 3 Current Cost whole-house transmitters to monitor things like my boiler and kitchen ceiling lights. EDF IAMs differ from CC IAMs in that the EDF units wait to be polled before transmitting data and hence we should be able to avoid RF collisions. But the EDF EcoManager "base" can only handle 14 EDF IAMs and only reports data over XML once a minute; and there's little point using multiple EcoManagers (because this setup will suffer from collisions) so I need to build my own base station for EDF IAMs.

The ultimate aim is to build a DIY base station which can talk to an arbitrary number of EDF IAMs and a small number (say 0-3) CC transmitters. The idea is that the Nanode will connect to a PC which will log data using rfm_ecomanager_logger. The Nanode should do the bare minimum to allow software running on a PC to perform its data logging function.

Hardware Requirements

  • The easiest way to get started is to buy a Nanode Classic and a 433MHz RFM12b wireless transceiver. Note that the Nanode RF kit comes with an 868 MHz RFM12b, which is not what you want. Or buy a pre-built 433MHz SMT Nanode.
  • If you build a DIY board, or if you connect an RFM12b to an Arduino, make sure the RFM12b's nFFS pin is set permanently high.

Installation

Uploading pre-compiled hex file

First change directory into this project's Release/ directory.

Uploading to AVR using FTDI serial cable:

avrdude -pm328p -carduino -P/dev/ttyUSB0 -b57600 -F -V -Uflash:w:rfm_edf_ecomanager.hex:a

Uploading to AVR using STK500v2 programmer:

avrdude -pm328p -cstk500v2 -P/dev/ttyACM0 -Uflash:w:rfm_edf_ecomanager.hex:a

Building from source

Using the Arduino IDE

With the Arduino IDE closed, place the rfm_edf_ecomanager directory and files into your sketchbook directory. You will also need to add the nanode_rf_utils library, as per the instructions in that repository's README.md file.

Then start the Arduino IDE and go to Open -> rfm_edf_ecomanager. Please note that I'm currently using Eclipse to develop this code (not the Arduino IDE). However, I do try to test in the Arduino IDE regularly. Many thanks to robomotic for help converting my code to compile in the Arduino IDE.

Using Eclipse

  1. Setup Eclipse for Arduino development as described by Francesco Rigoni.
  2. Add the nanode_rf_utils library, as per the instructions in that repository's README.md file.
  3. Clone the rfm_edf_ecomanager git repository into your Eclipse workspace directory
  4. Create a new C++ AVR Cross Target Application project called 'rfm_edf_ecomanager'. Be sure to modify the build settings for this project as described in Francesco's guide.

Building without Eclipse or the Arduino IDE

I'm afraid building this code isn't especially user-friendly at the moment because I'm developing on Eclipse and am using the auto-generated makefile at the moment while there's a lot of code churn. If I have time I'll manually create a more user friendly makefile.

I haven't tried this so I'm afraid I don't know if this will work but here are some pointers.

First, open Release/subdir.mk. You'll almost certainly need to modify the include directories to point to the relevant directories on your system.

Then run:

cd Release make all

Then upload your `rfm_edf_ecomanager.hex` using the "Uploading pre-compiled hex file" instructions above.

Build variables

Compile with LOGGING set to enable logging to the serial terminal (verbosity can be controlled using the v command: see the section on Serial commands). With LOGGING enabled, the program takes up about 82% of the available flash memory; with LOGGING disabled, the code takes up 69%.

Setting up a complete logging system with related projects

The sister project rfm_ecomanager_logger is a python script which talks to rfm_edf_ecomanager and allows the user to easily pair the system with new sensors as well as to log data to disk.

powerstats can then process the data logged by rfm_ecomanager_logger to produce graphs and basic stats.

babysitter can monitor the health of rfm_ecomanager_logger.

See "Build a complete logging system" for a step-by-step guide to building a complete power data logging system using rfm_edf_ecomanager.

Two Types of Wireless Sensor

The code makes a fundamental distinction between two types of Current Cost wireless sensor: CC_TX and CC_TRX sensors. The two types of sensors use fundamentally different RF protocols.

CC_TX

Current Cost transmit-only sensors ("TX" is an abbreviation for "transmitter"). These units can only transmit and hence cannot do any carrier detection; and lost data cannot be re-sent e.g.:

  • Current Cost IAM
  • Current Cost "Sensable" Whole-House Transmitter
  • EDF whole-house transmitter

CC_TRX

CC_TRX transceivers ("TRX" is an abbreviation for "transceiver"). These transceivers require a "polling" request from the base station to send data and hence RF collisions can be avoided. e.g.:

  • EDF IAMs

Introduction to the Behaviour of this Code

The code keeps two separate lists: one of CC_TXs and one of CC_TRXs. We need a list of CC_TXs so we can learn when each CC_TX is due to transmit and cease any transmissions around the time that we expect a CC_TX to transmit in order to avoid causing RF collisions with CC_TXs (because there is no way to ask for lost data to be reset with CC_TXs). We need a list of CC_TRXs so we can poll them in order.

Quick start guide

  1. Watch the serial port. On Linux this can be achieved with the command: screen /dev/ttyUSB0 115200
  2. All valid received information will be displayed
  3. Inform the Nanode how many TXs you have by typing s<N> where <N> is the number of TXs and where s is short for set. Use to S<N> command to state the number of TRXs. Don't worry if you add more sensors later, the code will dynamically expand the space allocated for remembering each type of sensor. The advantage of telling the system the number of sensors up front is that you can avoid memory fragmentation, which can become an issue if you have many tens of sensors.
  4. Now we need to teach the Nanode the specific IDs of our sensors. This can be done in one of two ways: either press the pairing button on each sensor (and the code will automatically pair with the sensor) or you can state the ID over the serial port using n<ID> or N<ID> for TXs and TRXs respectively.
Please see the section on serial commands for details of other commands.

Data Format

The code sends received data over the serial port in simple JSON format like this: 

 {"type": "tx", "id": 2431, "t": 29048, "sensors": {"1": 0, "2": 0}}
 {"type": "tx", "id": 2339, "t": 31920, "sensors": {"1": 316}}
 {"type": "trx", "id": 2200180225, "t": 40274, "sensors": {"1": 0}, "state": 0, "reply_to_poll": 1}
  • type is type of wireless transmitter. It can take two values: tx or trx
  • id is the radio ID of the transmitter
  • t is the time in milliseconds when the first byte of the packet arrives. The time is the number of milliseconds since the Nanode was powered on (but note that the ATMEGA's clock isn't very accurate because millis() stops counting when interrupts are disabled e.g. while an interrupt service routine is running; hence this clock falls behind quite rapidly)
  • sensors is the reported power consumption in watts for sensor each sensor connected to the transmitter (the Current Cost Sensable TX can accept up to 3 CT clamps). Sensor indices start from 1 not 0.
  • state is the state reported by EDF IAMs (1 == on; 0 == off).
  • reply_to_poll indicates whether this data packet was the response to a poll request or not. If it wasn't a reply to a poll request then it must have been due to the power button on the IAM being manually pressed.

Serial commands

  • You can send simple commands over serial to control things like the verbosity of output, which packets to report, to control pairing etc. (these are detailed below).
  • The Nanode talks to the host PC over serial at a baud rate of 115200. If you're using Linux then try the command-line application screen -L /dev/ttyUSB0 115200.
Command Description
a auto pairing. Turn auto_pair mode on. (default)
m manual pairing. Turn auto_pair mode off.
p<ID> pair with ID (only valid in manual pairing mode)*. e.g. p1234
s<SIZE> set size of array for CC_TXs*. e.g. s20
S<SIZE> Set size of array for CC_TRXs*. e.g. S20
n<ID> Add new radio ID to list of CC_TXs*. e.g. n1234
N<ID> Add New radio ID to list of CC_TRXs*. e.g. N1234
d delete all stored CC_TX radio IDs
D Delete all stored CC_TRX radio IDs
r<ID> remove single CC_TX radio ID*. e.g. r1234
R<ID> Remove single CC_TRX radio ID*. e.g. R1234
l list all stored CC_TX radio IDs
L List all stored CC_TRX radio IDs
t timecheck. Just respond with milliseconds since last reset (after a 10ms delay). No ACK.
0<ID> Turn TRX ID off
1<ID> Turn TRX ID on
v<level> verbose logging threshold. e.g. v0. Only works if code has been compiled with the symbol LOGGING defined. The lower the number, the more logging messages. 0=DEBUG, 1=INFO, 2=WARNING (something went wrong but it shouldn't affect us) (default), 3=ERROR (something went wrong badly enough to affect us but we can continue), 4=FATAL (something has blown up and we can't continue).
k Only print data from known IDs
u (default) Print all valid received packets including from unkown IDs; but don't print broken packets
b Print all received packets, even broken ones. Note that broken packets will be displayed as raw packets in hex like this: C FF 80 0 0 0 0 0 Packets from CC_TXs will be displayed after being de-manchesterised, even though the demanchesterisation failed. How does the code know if packets are broken? CC_TX packets don't have checksums so we use the manchesterised nature of the data as a primitive validation check (broken packets slip through this validation though). CC_TRX packets, on the other hand, have a checksum.

* Numbers can be decimal e.g. 12345 (no leading zero), octal e.g. 01234 (with leading zero) or hex e.g. 0xABCD123 (with leading 0x or 0X). Number input blocks until carriage return '\r' is received (but RF packets will still be received by the interrupt service routine. But don't take too long to enter the number of the receive buffer will overflow!) Backspace key does what you'd expect it to do.

Commands are either acknowledged or rejected by ACK or NAK respectively. A short description usually follows the ACK or NAK.

Pairing

The code can pair with both TX-only sensors (like Current Cost whole-house transmitters) and transceivers (e.g. EDF IAMs).

There are two pairing modes: auto pairing and manual pairing (see "Serial commands" section for how to change modes).

In auto pairing mode, the Nanode will accept all received pairing requests.

In manual pairing mode, the Nanode will notify the user that a pair request has been made with the JSON {pr: <id>} (where 'pr' is short for pair request). If the user wants to pair with this ID then send the Nanode a p<id> command (see "Serial commands" section for more info). The Nanaode will then pair with that ID if a further pair request is received.

In both modes, if the pairing process is successful then the ID is added to the Nanode's internal list and the pairing is acknowledged over serial with the JSON {pw: <id>} (where 'pw' is short for 'paired with').

If a pair request is received from an ID already known to the Nanode then no action is taken.

Development

Bug Reports and Feature Requests

  • We're using the GitHub Issue tracker for bug reports and feature requests.
  • If you have a problem with compilation in the Arduino IDE then please turn on "Show verbose output during compilation" in File -> Preferences and copy-and-paste the entire console output into the bug report.
Please note that my ultimate aim is to produce a data logging system suitable for my PhD research and hence I'm afraid I won't have much time to work on features which don't directly relate to my PhD research, sorry. But please go ahead and fork the code, add a feature and submit a pull request.

Code Style

  • Code style is roughly the Python PEP8 code style adapted for C++. i.e.:
    • CamelCase for class names
    • UPPERCASE_WITH_UNDERSCORES for constants
    • lowercase_with_underscores for variables and functions
    • Use 4 spaces for indentation (not tabs)

RF legal issues

  • The 433MHz ISM band is only legal in ITU Region 1 (which excludes the Americas!)
  • Some countries impose restrictions on RF duty cycle for the 433MHz band. Please check your country's regulations to make sure you are not violating these rules! Please note that the code's license states that the author of the code is not liable for any legal issues arising the code's use.

Acknowledgements & further reading

  • A massive thank you to Graham Murphy, Matt Thorpe and Paul Cooper who helped to decode the Current Cost and EDF RF protocols.
  • Many thanks to the good folks who wrote JeeLib as it provided lots of help.
  • Of course, I am hugely indebted to Ken Boak and his collaborators for creating the Nanode.
  • As did this tutorial on SPI on AVR devices.
  • If you want an in-depth insight into the RFM01 then read the RFM12B.pdf manual BEFORE or INSTEAD OF RFM12B_code.pdf as the latter has far less detail. The only advantage of the RFM12B_code.pdf is that it contains a code example in C.
  • For more details please see the CurrentCost category on my blog.

Known issues

  1. Struggles to receive packets from CC IAMs. See issue 13.

Related projects

  • rfm_ecomanager_logger A Python script specifically designed to interface with RFM_EDF_ECOMANAGER for adding and editing transmitters and then for logging power consumption data to text files.
Add a custom footer
Add a custom sidebar
Clone this wiki locally