Home Automation Controls - Simple Esp8266 Wifi Relay Part 1

Follow Along as I play with this. I'm just wingin it. Relay Control Server The esp8266s are great. They've got a lot of punch and built in wifi. Plus, being Arduino compatible, getting a prototype up and running is pretty fast.

A Simple Relay Server

A relay server should be able to operate a relay and accept wifi Http requests. The requests should include getting the state of the relay and switching the state of the relay. That seems like a decent set of simple requirements. So, this first iterations should be pretty simple with items we listed above. We should have a GET endpoint /relaystate?relaynumber=#, where relaynumber is a number we define for an individual relay. This will be more clear later. Let's also make two other GET endpoints to turn the relays on and off, /relayon?relaynumber=#, /relayoff?relaynumber=#. Using these endpoints we can set the state of each relay individually. This seems like a good enough start and we can get fancy later on.

Arduino Esp8266 Webserver

This webserver library makes putting things on a network a super simple. Check out the esp8266 Webserver example that comes with the esp8266 library in the Arduino ide.

Driving Relay Coils

Like many microcontrollers, the esp8266 has plenty of power in terms of computing and peripherals, but the output from the pins are not enough to power much. At 3.3V, the esp8266 lacks the current and voltage to drive current hungry modules, but this is common. We'll need to select a device to help us drive the relatively large coils. 

A transistor is a great way drive larger loads, like a motor or an LED. They're pretty simple devices that can be thought of like a voltage controlled switch. The PN2222A is a general purpose transistor and just about every tinkerer has a few of these floating around their tool box. They can be picked up at Radioshack. The interesting part about the transistor is when we put the transistor into "saturation" mode, the transistor becomes a sort of switch. You can also use other devices that are analogous to a transistor like a mosfet or motor driver IC's.

I happen to have a few L293D motor drivers, another fantastic little IC to use in your projects. Inside a L293 is four half H bridge drivers with flyback/kickback diodes. Awesome. I'm going to use these to drive my relays. 

Home Automation Controls - Why I'm not Using Qt 5.8

Follow Along with my Development on this project. I'm just kind of wingin' it Github - HomeControls

Simply, it's a pain. Qt 5.3 is available from the repositories and an all you need is an apt-get install on the Rpi to be in the game. While it seems simple enough to compile 5.8 for the Pi (setting up the cross compiler or compiling natively), it's not more simple than just typing apt-get install qtcreator qt5-default, which installs the ide as well as the dependencies. Yes, I'm going to be missing some of the new features in 5.8, but my goal right now is to create automation controls. The next iteration will be to make them more whizbang.

Also, I'm going to compile the qt projects natively on the Pi. I'm not going to worry about setting up a cross compiler environment for the time being. I'm going to use SCP to copy the qt project over to the pi, then compile the project natively. The process is simple enough SCP -> qmake -> make -> run the program. I'll detail the build steps in greater detail.

Hardware Setup

Adafruit makes these nice little TFT shields for the Raspberry Pi. The frame rate may be on the low side, but the compactness is pretty awesome. I may switch over to a small HDMI screen to see if there is a performance boost. Adafruit also makes a similar product, but with an HDMI screen.

The touch screen is a resistive film type and seems to have good enough resolution to poke even the smallest of window controls. A stylus is a good addition for this screen or make sure your style your buttons large enough for fat fingers like mine.

Software Setup

For the interface side, I'm going to rely on Qt's awesome qml. It's flexible, fast, and anchoring. Anchoring is everything you wish HTML had. Here's a novel idea in qml, centering something in a container is easy. What a thought....

If you've never seen qml before, I recommend you find a tutorial online. The official Qt documentation is quite good. I'll throw in the practical bits on hooking things up.

Home Automation Controls - Raspberry Pi - QT

Home automation is a great way to tinker and learn some new skills. I'm going to use Qt and its fantastic UI system and the Raspberry Pi to build some nice interfaces and controls for my home.

Rate Limiter with RxJS

Use Subjects and Ovservables to create a stream of operations that get executed at a set rate. Subjects allow you to create an entry point to the stream and insert new items. An Observable Interval and a zip operator will give a set interval to move items down the stream. We can use a map/flatmap/concatmap operator to do a request or operation. Finally, a share will allow the stream to be sent to different observable streams based on conditions from a filter.

function foo()
{
  let itemCount = 0;
  let rateInMs = 500;

  // Only emits interval when items are available
  let rateInterval = Rx.Observable
                       .interval(rateInMs)  
                       .filter(x=>intemCount > 0);
  let itemsSubject = new Rx.Subject();
  // Zip the itemsSubject and rateInterval to move 
  // items down the stream at a set rate
  let rateLimStream = Rx.Observable
                        .zip(rateInterval, itemsSubject)
                        .map(x=>{
                          // Do something that returns 
                          // a promise/observable
                        })
                        .share();
  let successStream = rateLimStream.filter(x=>!x.error);
  let errorStream = rateLimStream.filter(x=>x.error);

  successStream.subscribe(x=>{
    itemCount--;
    // Do something with them
  });

  errorStream.subscribe(x=>{
  itemCount--;
  // Do something with them
  });

Now all that would have to be done is expose itemsSubject somewhere, possibly through module.exports so that you may call itemsSubject.next(item).

Be careful though, items that throw errors during an operation will cause the stream to stop and subsequent calls to itemsSubject.next will not cause anything to happen. To avoid this, pass on custom objects instead of a Promise.reject or an Observer.error.

Making a Custom Signed Access Token

// The Access Token
In an earlier post, I talked about access token and how they're built. They're comprised of a payload and then a hash of the payload and some secret which makes it a signed token.  A popular implementation of this approach is the JWT, but we'll cover that another time and we'll just make our own simple access token. Lets call it the DSBAccessToken. Sounds official right?

// DSBAccessToken
The DSBAccessToken is our own simple access token that is signed and contains a payload of just a string. The structure of it, might look like the figure below. On the left is the message, the plus sign indicates our delimiter which separates the message and signature, and finally the signature.

A very simple token consisting of a string payload and a hash of the message and a secret

For our case, the message is just a string, but a lot more information can be stored in there. You can place pretty much anything in there that is serializable or binary data. This includes JSON, which is very convenient and the basis of the JWT. JWTs for example, are very structured and contain several parts to the payload. We'll see some steps here in order to make a token.

// Steps to Making a Signed Access Token
The steps to creating a signed access token are pretty straight forward and simple. Below is a simple graphic showing the flow.

1. Pass the secret and payload through SHA256
2. Base64 encode the message and the signature
3. Concatenate the Base64 version of the payload and the signature with a delimiter, usually a '.'

Very basic flow of creating a signed token

// Verifying the token
Verifying the token is pretty simple as well. Since we should know the secret, we just need to make sure the payload of the token we received produces the same signature with our secret. The flow for this process is pretty much like creating the token but in reverse and the last step is to compare the signature we create with the signature of the access token we received.

1. Split the access token on the delimiter. For us its the '.'
2. Base64 decode the message and the signature.
3. Pass the message we received and our secret through SHA256
4. Compare the signature we just made with the one we received.
5. If the signatures are the same, then the access token was untampered with. If the signatures are different, then the data changed or tampered with.

// NOTE:

I've been using base64 and base64url interchangeably. If you plan on passing your token along in the query parameters, then base64url needs to be used. If the token is being sent by some other means, either way is fine, just use the correct decoding on the other end.   

Digital Signing

When an artist signs a painting, they are leaving a mark distinguishing the work to be authentically by them. Later, other people can identify the work as authentic because of the signature as well and style of the painting. The unique signature on the painting acts as an identifier and a way for people familiar with the signature to recognize and confirm the painting is indeed authentic. In software engineering, we often have the same problem of trying to discern real things from forgeries, such as access tokens. Fortunately for web developers, there are some clever techniques that can be used to prove tokens are authentic.

// Forgeries/Intercept
There have been some incredible forgeries of paintings in history. Unfortunately, for some art collectors, there are some very talented painters with the skill to reproduce other painters styles and then top it off with a near perfect signature. Leaving enthusiasts, collectors, and experts alike with something less than it claimed to be. Access tokens experience the same problem.

Access tokens, while they are not works of art, can be passed off as forgeries or even stolen. If access tokens can easily be forged, anyone can masquerade as another user and do things on their behalf. Intercepted access tokens, for example tokens being passed over an unencrypted connection can be intercepted and used. This is where a signed token comes in.

// Signed Tokens
A signed token is like a normal token, but it has an extra bit of information tacked to verify the authenticity and contents. Sometimes signed tokens are used to identify users or grant access to a protected resource. A signed token, or a signed anything, is a payload concatenated with the digest of the payload and a 'secret'. A payload is whatever bit of information we care about and the digest is the output of our hashing function, which we'll get to. This signature allows us to verify the content hasn't been tampered with and is authentic, because only people who know the secret can generate the right digest. It's like we have a machine that takes in a message, signature, and a secret then tells you if its good or tampered with. So, how would we create such a machine? First, we need a hashing function.


// Hashing function
A hashing function is some math thing that creates a unique signature from arbitrary data and/or a secret to make the signature even more unique. I've mentioned digest, hash, and signature a few times already, but they're all the same thing: the output of the hashing function. A hashing function is only one way, it can't be reverted and you can't reproduce the original data from just the digest. A hashing function will produce the same digest if it is fed the same data and secret, but if even one character or bit is changed in the input or secret, the digest will be very different. We'll look at SHA256, hashing function that can take in an optional secret.

SHA256 takes in data and a secret then produces a constant length signature or hash.

// Example Code

https://github.com/bdcannon/sha256-test

• Clone the repository
• cd ./sha256-test
• npm install --save
• node hash-test.js

Node comes with an implementation of the SHA256 hashing algorithm in the crypto module. The following example has a preset secret and will hash whatever line is typed into the terminal.

The output of the hashing function radically changes even for a small change in the input

Try few things and see what the outcome is. Typing in the same message twice will produce the exact same output because the data and the secret haven't changed. Try changing the original message just a little big and see what happens to the output. As you can see in the image above, even a small change makes a huge difference! I only omitted one character from the original message. The same thing will happen if the data remains the same, but the secret is only changed slightly. How can we use this to our advantage and prove that messages or tokens haven't been tampered with?

// Next Time

Now that we have a little bit of an understanding of how a hashing function works, we'll make our own access token. 

Reading Serial Sensor Data With Node and Arduino

Getting data into your application is key to being successful for any experiment. Node has a nice implementation of a serial port library called Node serial-port. You can install it pretty easily with a npm install --save serialport.

Node's evented nature fits nicely with responding to data or messages from microcontrollers, serial sources, or anything that happens asynchronously. Serialport's readline and delimiter mechanisms allow parsing individual messages or creating your own protocol a breeze. For example if you selected the readline parser, the "data" event would be triggered every time some data followed by a new line character arrives on the port. You can also specify a sequence of custom characters so that you can construct your own communications protocol.

Let's look at a simple example. First, we need a serial input to feed our Node app. I'm going to use an Arduino, more specifically the Adafruit Feather 32u4 Bluefruit LE. The simple sketch below will just print the string "hello world!" followed by the return carriage (\r) and new line character (\n)  over and over again.

Just prints "hello world!" over and over and over

Now we need to set up the Node side to receive the data. Start yourself a Node project and install serialport. I'm on a MacBook pro so my Arduino shows up under /dev in the file structure. Try an ls /dev/tty to show everything in the dev directory that starts with a "tty". My Arduino showed up as /dev/tty.usbmodem1421.

If you get an error that the port couldn't be opened, try a sudo chmod 777 on the serial port in /dev. 777 is the nuclear option for permissions and you probably should't be giving to too many files, but that should give you access to the port... and everybody else. If you run the node application, you should see "hello world!" pop up every 500 ms. 

If you wanted to get the same bit of data, but have it arrive as bytes instead of a string, use "SerialPort.parsers.byteDelimiter([13,10])". Why 13 and 10? Those are the Decimal value of the ascii bytes for \r and \n. So, you can do the same with any bytes or characters, just use their byte value.

Creating Your Own Simple Protocol (Book Ends)

Sometimes when you're transferring information, its a good idea to construct the message in a way that its easier to process or to add some reliability. If you are trying to transmit data over some unreliable transport layer (maybe you're trying to send data through salt water or something weird) it may make sense to check that the data arrives properly or that you just didn't read in some garbage.

A simple and limited way to check your data's validity is to use start and stop byte sequences. A start and stop sequence will allow you to delimit or 'bookend' your data and simply check if data is where it should be. For example, lets say we were trying to send a 4 byte number over and over again to our node app. What we want to do is read in byte0, followed by byte1, and byte3, and then, at last byte4. If we accidentally start reading in the middle of the number, we'll end up with a few bytes from one number and bytes from another. This would give us the completely wrong number. How can we remedy this?

If we start and stop the transmission with a fairly unique byte sequence, we can check the first and last bytes we received to check they match up with what we expect. Let's make an example.

Writes out the start sequence, followed by our 4 byte payload, and the stop sequence. 

The Node side. We're waiting for a byte sequence that ends with '%^' and begins with '!&'

In the code above, we're getting data from the event, checking the start sequence, and logging the payload. At first, we just get rid of the last 2 bytes. Since we're using the byteDelimiter, its safe to assume the last bytes are the delimiter because we got the event. Now, we check the first two bytes to make sure they match and if they do we can assume we got a complete message.

Where Bookends Come Up Short

Bookends are a simple way to ensure some reliability in your messages, but it does not guarantee your payload is what you originally intended to send. While your bookends may arrive at the correct time and in place, there is nothing about this method that guarantees the validity and integrity of the payload.


TODO: Give you code instead of pictures.