Author Archives: mostlyrobots2012

Home Automation – Automatic Garden Watering

An automatic garden watering system was my original goal for my home automation system, especially as this year we have had very sunny and dry conditions. I was out watering my plants nearly every night.

I have several different things I needed to water; raised beds with vegetables, a wall of strawberries, some fruit trees, and we tried our hand at potatoes this year as well.

Automatic Watering System - Beds

Automatic Watering System – Beds

Four zones fitted nicely with my four channel MOSFET board I had designed. There is a dedicated blog post about the process I went through to design and manufacture the MOSFET shields and you can buy them on Tindie for you own projects. 😊

I sell on Tindie

I designed a water manifold with four 12v solenoids which would give me four individually controlled watering zones and a pass-through connector so I could daisy chain additional controllers later as required.

I needed to split the watering zones up because my water pressure could not drive all the drippers and sprinklers at the some time, if they were all connected in series. You need a reasonable amount of pressure to get the mini sprinklers to work properly and you can’t drive more than 15-20 at once as the pressure drops too low.

I used Hoselock Easy Drip system to supply water to the plants and beds I needed to water. The system is simple to configure. You run a standard hose around the bed, and clip on sprinkler heads which pierce the hose and provide irrigation. There is also a micro drip system which uses smaller 4mm hose for pots. I used the Easy Drip for all the beds and Micro Drip for the strawberries.

Hozelock Easy Drip System

Hozelock Easy Drip System

Hozelock Micro Dripper

Hozelock Micro Dripper

Design

I built the manifold out of ¾” threaded pipe connectors and 12v solenoids. Next time, I might try making the manifold from soldered fittings, but the threaded fittings gave me the flexibility to change the design as I built it. I have also found some ready-made manifolds since building mine, so I might try those too next time.

Automatic Watering System Manifold

Automatic Watering System Manifold

I built a wooden box out of some old decking boards I had, so I could mount the manifold and controller on the fence.

Automatic Watering System Box

Automatic Watering System Box

The device would be in my garden, so I used a Wemos D1 Mini microcontroller with an external antenna to give me a bit of extra WiFi range. It will be powered by a 12v battery, so I mounted the Wemos and MOSFET shield in an IP60 weatherproof case, with a DC-DC step down convertor to give me 5v to power the Wemos and 12v from the battery for the solenoids.

The plan is to charge the battery from a small solar panel, so its completely wireless in the garden. The current draw is tiny. The solenoids only draw about 600mA each and they are only on for short amounts of time. Even without a solar panel to charge the battery, it would last maybe 2 weeks between charges.

Automatic Watering System Final Installation

Automatic Watering System Final Installation

The main water feed from the garden tap comes in on the right hand side via a manual valve so I can quickly turn off the water if I need to, without having to walk all the way back down the garden. I leave this on and connected all the time, so the watering system can water at any time of the day or night.

There are four watering zones, one on each solenoid. The four zones are;

  1. Potatoes
  2. Vegetables Beds
  3. Fruit trees
  4. Strawberry Wall

The last connection on the left is a pass through from the main feed, also via a manual valve, where I can either daisy chain another four zone controller, or attach a hose and hand held sprinkler for manual watering.

Testing

To test the controllers and manifold worked correctly before I installed it permanently, I wrote a simple script that turned on each channel sequentially, with an MQTT message, every 2 seconds.

I positioned the controller with its external antenna just on the edge of my house WiFi range, so it was as far up the garden as it could be, but still in range.

Automatic Watering System - Potatoes

Automatic Watering System – Potatoes

The automation is driven by NodeRed. I water each zone for 20 mins every morning at 6am. I can add more logic later if I need. There are all kinds of modules and sensors I could add to measure soil moisture or rain sensors and decide not to water if its been raining or the ground is already moist, but there’s no risk of over-watering anything at the moment, so it just waters every day for 20 mins.

In theory if the summer gets really hot, I could measure soil moisture and decide to add an extra watering cycle midday if the ground is really drying out in the sun. But that will be a phase two.

I also have a manual override using the NodeRed dashboard buttons, so I can manually turn zones on and off from my mobile phone.

Automatic Watering System NodeRed Flows

Automatic Watering System NodeRed Flows


Automatic Watering System NodeRed UI

Automatic Watering System NodeRed UI

The results have been amazing. The strawberries have really benefited for daily watering. We got a fantastic crop of potatoes too. Now the automatic watering system is in place we will increase the volume of plants next summer, now I’m confident they will all survive, and I don’t need to spend hours watering them every day! 😊

Lots of Strawberries

Lots of Strawberries

Design, Fabrication and Testing of a MOSFET Shield for a Wemos D1 mini ESP8266 Microcontroller

Design

I needed a MOSFET Shield for a Wemos D1 mini to drive solenoids and relays. I could not find anything suitable online, so I decided to design and build my own. The board size of a Wemos D1 mini is quite small (1″ x 1.3″) and part of that is the PCB antenna, so a shield should avoid covering the area over the antenna with copper or lots of components.

Wemos D1 Mini ESP8266 Development Board

Wemos D1 Mini ESP8266 Development Board

I wanted the shield to be the same size as the Wemos D1 mini. I need to be able to switch about 12v at about 2A. Most use cases I had were small relays and solenoids.

I found a very small dual channel MOSFET in a 2mm x 2mm package (FDMA1024NZ – Dual N-Channel MOSFET 20V / 5.0A). This MOSFET is so small I could fit two on the shield, giving a total of four channels.

Wemos D1 Mini MOSFET Shield 4ch v1.3 Schematic

Wemos D1 Mini MOSFET Shield 4ch v1.3 Schematic

Wemos D1 Mini MOSFET Shield 4ch v1.1 Board

Wemos D1 Mini MOSFET Shield 4ch v1.1 Board

Fabrication

I created the board design and ordered the PCBs and a solder stencil for the first prototype. The 2mm x 2mm package is very difficult, if not impossible, to hand solder, so the stencil would allow me to stencil the solder paste and hand place the components, before reflowing it in an oven.

MOSFET Shield for Wemos D1 mini before solder stencilling

Before solder stencilling

MOSFET Shield for Wemos D1 mini during solder stencilling

During solder stencilling

MOSFET Shield for Wemos D1 mini after solder stencilling

After solder stencilling

MOSFET Shield for Wemos D1 mini close up of solder stencilling and component placement

Close up of solder stencilling and component placement

MOSFET Shield for Wemos D1 mini v1.1

First prototype completed

The first version worked perfectly, but the 2mm x 2mm MOSFET was a little obscure and difficult to hand solder. In addition, JLC PCB had launched a pick and place PCB fabrication service which was very economical. I thought I’d try it for my MOSFET board, but the MOSFETs I was using were not supported. So I replaced the two obscure dual MOSFETs with four discrete MOSFETs which JLC support as a basic components.

Wemos D1 Mini MOSFET Shield 4ch v1.3 Board

Wemos D1 Mini MOSFET Shield 4ch v1.3 Board

The JLC PCB fabrication service built a batch which again all worked perfectly.

MOSFET Shields for Wemos D1 Mini v1.2

MOSFET Shields for Wemos D1 Mini v1.2

Testing

I needed to test the board design to ensure it could handle the current rating. The MOSFETs are rated at a maximum of 5A each. So, each individual channel should switch 5A. In addition all four channels can be connected together which would switch about 20A in total. I wanted to make sure the tiny PCB would handle this load. I designed the power traces on the PCB to be as wide as the physical dimensions of the board would allow.

I bought a 180W battery tester from BangGood to use as a variable load which could test up to 20A and 180W. This device has lots of useful features. You can configure it as a constant current load in 10mA increments. It measures voltage, current, power and even temperature using a plug in thermocouple, and displays all the readings in real time on a colour LCD. Its a very functional device for a very economical price.

To source 20A, I decided to use a lead acid battery. I don’t have a bench power supply that can easily provide 20A. I used thick 12AWG wire to connect the battery to the variable load and the shield under test.

MOSFET Shield for Wemos D1 mini Power Testing Setup

MOSFET Shield for Wemos D1 mini Power Testing Setup

The test setup consisted of;

  • A 12v lead acid battery
  • The variable load
  • The shield under test – with the MOSFET outputs shorted together

Power Testing Setup

Power Testing Setup

I planned to test each individual channel at 1A, 2A, 3A, 4A & 5A, while monitoring the temperature of the MOSFET, PCB traces and connectors. In addition, I want to connect all four channels together and test up to 20A.

The first few tests went well. Each channel can handle up to 5A. The MOSFETs get a little warm at 5A. If the board is used at maximum capacity for long periods, it would benefit from either forced air cooling or a small heat sink on the MOSFETs.

MOSFET Shield for Wemos D1 mini Power Testing Temperature

MOSFET Shield for Wemos D1 mini Power Testing Temperature

I struggled to test all channels to a total of 20A. The battery tester I had can only handle 180W, and the only convenient 20A source I had was a 12v lead acid battery. 12v @ 20A is 240W which is more than 180W, and the tester cuts out when it hits 180W.

I managed to test up to 16A with the lead acid battery setup, which was right on the limit of the 180W limit.

MOSFET Shield for Wemos D1 mini Power Testing 16A

MOSFET Shield for Wemos D1 mini Power Testing 16A

I had a small 2S LiPo at 7.2v and managed to complete a final test of 20A using this LiPo, but at 20A is the battery discharged very quickly and the voltage would drop to 6v very quickly and the tester cuts out at 6v to protect the battery from over discharge.

MOSFET Shield for Wemos D1 mini Power Testing 18A

MOSFET Shield for Wemos D1 mini Power Testing 18A

I’m confident the boards will operate to 5A per channel, to a maximum combined current of 18A as I managed to successfully test to both these levels. At higher currents the individual MOSFETs get hot and the Dupont header does. I think the header pins are only rated to 5A, so for 20A the board would need a redesign with more power pins to distribute the load. 4A per channel is way more than most of my projects need, so I’m happy with the design. If you really need to switch 20A for short periods of time, use a heatsink and solder the wires directly to the board rather than use pin headers and a Dupont connector.

Batch production

The shields were used in three of my home automation projects:

  • Automatic Garden Watering to control four water solenoids connected to garden hoses.
  • Immersion Heater Controller to control 2 high current relays to switch two 3kW hot water immersion heaters.
  • Automatic plant Pot Watering to control a small peristaltic pump to water pot plants based on moisture reading from a capacitive moisture sensor. This was the project that I added the analogue in header on the shield for. This allowed me to connect the moisture sensor and the peristaltic pump to the same shield.

Once the shields had been running in projects for a couple of months, without any issues, I decided to build a small batch to sell online. If I couldn’t find something similar online, then maybe others would like to buy these to use in their robotics or home automation projects.

MOSFET Shield for Wemos D1 mini v1.3 Panel for Fabrication

MOSFET Shield for Wemos D1 mini v1.3 Panel for Fabrication

The four channel MOSFET shield can be purchased here on Tindie.

I sell on Tindie

Final Specification

  • Dimensions : 1″ x 1.3″ x 0.063″ (25.4mm x 33.0 x 1.6mm)
  • Four MOSFETs : AO3400A – 30V N-Channel
    • Maximum Drain-Source Voltage : 30v
    • Continuous Drain Current : 5A
  • GPIO pins used : D5 (GPIO14), D6 (GPIO12), D7 (GPIO13) & D8 (GPIO15)
  • ADC pin A0 available on 0.1″ header with power pins (selectable 3.3v or 5v)

Future revisions

I have already starting thinking about additional features that could be useful.

Using four digital GPIO pins to control the MOSFETs is sometimes a pain as the GPIO pins are shared with other features on the microcontroller. The D5, D6, D7 & D8 pins used on the shield are also the SPI port, so when the MOSFET shield is connected you cannot use SPI at the same time.

An improvement would be to use a I2C device to control the four MOSFETs rather than discrete GPIO pins. Multiple I2C devices can be connected in parallel to the same pins, so if I switched the shield to use I2C you could stack other shields in parallel and all of them would work.

There is a 4-bit I2C I/O expander IC (PCA9536) which could facilitate this. The board layout would be very tight to fit this additional chip on the same top layer. I could try and place it on the bottom layer, but a dual sided board is more expensive and complex to produce.

Or I could also try and add solder jumpers on the bottom layer to allow different GPIO pins to be user selectable.

I’m interested in any feedback on the board or any additional feature requests you would like to see. Leave your feedback in the comments section below.

Home Automation – Temperature Sensors

I wanted to monitor internal and external temperatures as part of my home automation, so I decided to use Wemos D1 mini boards for this, running Tasmota, as they are small, feature rich and really inexpensive (about $2 each).

The Wemos D1 mini is a fully functional ESP8266 board with built in WiFi antenna, 11 GPIO pins that support protocols like SPI & I2C, an analog input and USB interface. There are also many shields that provide lots of different features like temperature sensors, OLED Screens, Relays, LEDs, etc. You can buy the boards and shields on AliExpress. I particularly like the OLED displays and the temperature sensor board.

I initially designed a custom PCB for the old ESP01 development boards, which contained a USB to 3.3v power supply and space for a DS18B20. However, most of the ESP01 boards only have 512kB of Flash which is too small to easily enable over-the-air updates for the firmware. So, I quickly switched to the Wemos D1 Mini boards instead.

ESP01 DS18B20 Temperature Sesnor Board

ESP01 DS18B20 Temperature Sesnor Board

 

Although it is trivial to attach a 1-wire DS18B20 temperature sensor to the Wemos, a shield makes the wiring neater.

Wemos SHT30 Temperature and Humidity Sensor Shield

Wemos SHT30 Temperature and Humidity Sensor Shield

 

The Wemos SHT30 Temperature Shield contains a SHT30 sensor, which provides accurate temperature and humidity readings over an I2C interface. Configuring the temperature sensor in Tasmota is trivial. Just configure the I2C pins in the admin interface and temperature and humidity is displayed on the home page.

Tasmota I2C Temperature Sensor Config

Tasmota I2C Temperature Sensor Config

 

The Tasmota code automatically searches for all connected I2C sensors when it boots and displays anything it finds. You do not need to tell Tasmota which sensor it is. A full list of support sensors in Tasmota can be found here.

 

Tasmota Showing SHT30 Sensor Data

Tasmota Showing SHT30 Sensor Data

Note: To get sensor support you will need to flash the devices with the tasmota-sensors.bin binary, instead if the generic tasmota.bin file. Sensor support was recently moved into its own binary to make the default binary smaller.

Reading the data – HTTP

You can query the data directly over HTTP using the url :

http://192.168.1.20/cm?cmnd=status%208

(Substituting for your IP number)

The response is a JSON structure containing all the sensor data.

{"StatusSNS":{"Time":"1970-01-09T03:28:52","ANALOG":{"A0":8},"SHT3X-0x45":{"Temperature":20.5,"Humidity":63.1,"DewPoint":13.2},"TempUnit":"C"}}

On a Raspberry Pi you can extract raw data values using jq to navigate the JSON Structure.

curl -s http://192.168.1.20/cm?cmnd=status%208 | jq '.StatusSNS."SHT3X-0x45".Temperature'

18.1

Note: you must encase the “SHT3X-0x45” in double quotes. It is an artefact of the jq library, it does not like the -0x in the middle of the sensor name. I don’t really know why … it works fine for sensors that don’t have funny characters in the middle. E.g. ‘.StatusSNS.MAX31855.ProbeTemperature’)

Reading the data – MQTT

You do not need to read the temperature directly from the device if you are using MQTT. By default Tasmota automatically sends “Telemetry” messages out over MQTT every 5 minutes containing all the sensor data, which can be received and processed by NodeRed.

An example MQTT “TELE” message:

{"topic":"tele/temp01/SENSOR","payload":"{\"Time\":\"2020-07-01T13:17:44\",\"SHT3X-0x45\":{\"Temperature\":26.0,\"Humidity\":47.4,\"DewPoint\":13.9},\"TempUnit\":\"C\"}","qos":0,"retain":false,"_msgid":"d1a72fd2.4d6fc"}

I configured a dashboard gauge in NodeRed to show the temperature data in real time for my internal and external temperature sensors.

Temperature Gauge NodeRed Flow

Temperature Gauge NodeRed Flow

 

The function to extract the payload strips all the other information from the message and just returns the value in the payload ready for the dashboard gauge to display.

msg.payload = msg.payload["SHT3X-0x45"].Temperature;
return msg;

 

Temperature Gauge NodeRed Dashboard

Temperature Gauge NodeRed Dashboard

Voila a simple, real-time, temperature gauge on my phone over MQTT.

Home Automation – MQTT, Mosquitto, Node Red, Sonoff & Tasmota

There several things in my house I have been meaning to automate for a while. Some are obvious and simple, like lights and lamps around the house and the water fountain in the garden, others are more complex, like the Economy 7 hot water immersion heater and an automated garden watering system.

After reading a lot of forums and watching a lot of YouTube videos, it was obvious that MQTT was at the heart of any DIY home automation system.

YouTube channels, like Andreas Spiess and SuperHouseTV were a hugely useful resources.
I decided to use a Raspberry Pi running Mosquitto as the MQTT broker and NodeRed for some automation.

There are plenty of tutorials online on how to install these and it’s very simple.

For Mosquitto:

apt-get install mosquitto
apt-get install mosquitto-clients

For NodeRed, the instructions are here.

bash < (curl -sL https://raw.githubusercontent.com/node-red/raspbian-deb-package/master/resources/update-nodejs-and-nodered)
node-red-start

MQTT has a simple “Publish & Subscribe” model (sometimes called “PubSub”). Things can publish messages on a topic which are sent via the broker to anything that has subscribed to that topic. You can subscribe to a particular topic, or to all messages, or filter the topic using wild cards and expressions. It’s very flexible and there is a huge ecosystem of devices that support MQTT.

I setup a simple local software test on the Raspberry Pi to check everything was working after I rebooted. By installing the mosquito clients on the Raspberry Pi I can publish and subscribe using command line tools.

I opened two separate SSH windows, in the first I subscribed to a test topic called “test/message”

mosquitto_sub -t "test/message"

This command will stay running and output any messages that have the topic name “test/message”

In another windows I published a message on that topic with the payload value “hello”

mosquitto_pub -t "test/message" -m "hello"

in the first subscription windows I see the message “hello” appear. It’s all working!

pi@localhost:~ $ mosquitto_sub -t "test/message"
hello

After watching a lot of Andreas and SuperHouse on YouTube, it was clear that the simplest way to automated devices in the home using MQTT was with a Sonoff device, and to reflash this device with an open source firmware called Tasmota.

The Sonoff devices do come with a firmware that provides some simple automation and remote control, but it is not open source. Who knows what data it is collecting. It also needs a public internet connection, whereas a local MQTT Server and Tasmota means all my devices stay on the local network and do not need an internet connection to function.

Sonoff devices are inexpensive, well made and easy to hack; plug sockets, switches and other devices that are all based on the ESP8266 chip. Using these devices Theo Arends and others built an open source MQTT based firmware for Sonoff devices called Tasmota.

Tasmota Code is here on GitHub.

Tasmota Documentation is here.

Tasmota

Tasmota has pretty much everything you need already built in to it, in a very small binary. It includes MQTT support and HTTP management interfaces. Over the air updating means you can reflash the device with new firmware releases using the HTTP interface, rather than having to connect a serial cable. It also has support for dozens of common sensors, like 1-wire temperature sensors, RGB LEDs, etc, all supported out of the box in the code.

Almost all the Sonoff devices can be reflashed to run Tasmota, plus a long list of other hardware, including other ESP8266 based devices and development boards. A full list of supported hardware and sensors is on the Tasmota Template Repository.

I bought some Sonoff S20 sockets. These are mains plug/socket combos than allow you to plug in pretty much any mains powered device into a domestic mains plug and have control over it using home automation. The sockets can switch 10A (about 2kW in the UK) and they have a manual override switch so you can manually turn the device on and off at any time as well.

The initial flashing process is very simple, but you need to take the cover off the Sonoff device, and these are mains voltage when they are plugged in. So do this at your own risk, and watch some YouTube examples first. They do not need to be, and should not be, plugged in to the mains while you flash them. Only plug them in after you have disconnected your programming cable and replaced the covers.

What this video from SuperHouseTV which gives detailed instructions.

I made simple programming lead from a USB to Serial dongle to program the boards and I didn’t solder headers in to the Sonoff device, I just pushed the pins from the lead in to the PCB hole while it programmed and held them there manually. It takes less that a minute to upload the new Tasmota firmware and once it’s running you can do everything else through the web interface, so you should never need to open the device again.

You need to install the esptool.py on your laptop / computer, which is used to transfer the firmware binary file to the ESP8266 over a serial cable. Once you have identified the com port your serial cable is attached to, the command line is simple.

Download the latest Tasmota firmware from GitHub if you scroll down in that page you’ll find a long list of binary files to download. There are lots of binary files to choose from, but don’t let this confuse you. I have yet to need anything other than the basic binary. It supports all the common functionality you need, and if you need something specific later, you can change the binary over the air.

Just download tasmota.bin from the release page, and copy it to your local directory. I like to add the version number of the release in the filename, so I know which file is which version next time I need to flash something. In this example I named tasmota.bin to tasmota-8.3.1.bin.

Then run esptool.py to upload the binary file to the ESP2866 chip on the Sonoff device.

cd Projects\Sonoff\Tasmota
esptool.py -p COM11 write_flash --flash_size 1MB --flash_mode dout 0x00 tasmota-8.3.1.bin

You will need to substitute the correct COM port for your serial cable and the correct filename for your tasmota.bin.

While flashing via the serial cable, you should see something like this:

esptool.py v2.6
Serial port COM11
Connecting....
Detecting chip type... ESP8266
Chip is ESP8266EX
Features: WiFi
MAC: DE:AD:BE:EF:12:34
Uploading stub...
Running stub...
Stub running...
Configuring flash size...
Compressed 519920 bytes to 355646...
Wrote 519920 bytes (355646 compressed) at 0x00000000 in 31.7 seconds (effective 131.2 kbit/s)...
Hash of data verified.
Leaving...
Hard resetting via RTS pin...

Once the Sonoff device is flashed with Tasmota, you can reboot it, and you should find a new WiFi SSID called “tasmota_xxxx” where “xxxx” is the last part of the devices MAC address.

  • Connect to this access point “tasmota_xxxx” on your phone or laptop.
  • Browse to 192.168.4.1 on your phone or laptop and you should get the default Tasmota configuration screen.
  • Add the WiFi SSID & Password for your house network, click save and reboot.
  • Once it has rebooted, it will connect to the house WiFi network and you should be able to find the DHCP IP address in your DHCP server.
  • Once you know the local IP you can reconnect to your house WiFi on your laptop and connect directly to the Tasmota device in a browser using its IP number.

Screenshot of the configuration screen in Tasmota where you enter your WiFi details

Screenshot of the configuration screen in Tasmota where you enter your WiFi details

You need to configure a few things in Tasmota to enable MQTT. You need to set the MQTT broker IP address and tell Tasmota what the hardware device is so it know which GPIO pins control which functions.

You can configure this in the web interface. Browse to your device IP number and from the menu select “Configuration” | “Configure MQTT”. Set the “MQTT Host ” to the IP number of your Raspberry Pi Mosquito MQTT Broker you installed earlier.

If you are using a Sonoff S20 plug, from the menu select “Configuration” | “Configure Module” | Module Type = “Sonoff S2X”

Save and reboot and that’s pretty much it. The Tasmota device is fully configured and ready to receive MQTT Messages.

Once you are familiar with the web user interface, it is sometimes quicker to configure your devices via the console. There is a full list of Tasmota commands here.

Once you have flashed your device, do not disconnect the USB cable. Open a serial terminal window with 115200 baud (8N1) and you will have access to the command console.

You can enter you local WiFi credentials using the following syntax :

backlog ssid1 YOUR_WIFI_NAME; Password1 YOUR_WIFI_PASSWORD;

Your device will reboot.

The given IP address will be visible in the serial console output:

00:00:00 Project tasmota Tasmota Version 8.3.1(tasmota)-2_7_1
00:00:00 WIF: Connecting to AP1 HouseWiFi Channel 6 BSSId DE:AD:BE:EF:12:34 in mode 11N as tasmota_3B7DE8-7656...
00:00:03 WIF: Connected
00:00:03 HTP: Web server active on tasmota_3B7DE8-7656 with IP address 10.1.0.73
16:30:48 RSL: tele/tasmota_3B7DE8/INFO1 = {"Module":"Sonoff Basic","Version":"8.3.1(tasmota)","FallbackTopic":"cmnd/DVES_3B7DE8_fb/","GroupTopic":"cmnd/tasmotas/"}
16:30:48 RSL: tele/tasmota_3B7DE8/INFO2 = {"WebServerMode":"Admin","Hostname":"tasmota_3B7DE8-7656","IPAddress":"10.1.0.73"}
16:30:48 RSL: tele/tasmota_3B7DE8/INFO3 = {"RestartReason":"External System"}
16:30:48 RSL: stat/tasmota_3B7DE8/RESULT = {"POWER":"OFF"}
16:30:48 RSL: stat/tasmota_3B7DE8/POWER = OFF
16:30:52 RSL: tele/tasmota_3B7DE8/STATE = {"Time":"2020-06-30T16:30:52","Uptime":"0T00:00:11","UptimeSec":11,"Heap":29,"SleepMode":"Dynamic","Sleep":50,"LoadAvg":19,"MqttCount":0,"POWER":"OFF","Wifi":{"AP":1,"SSId":"HouseWiFi ","BSSId":"DE:AD:BE:EF:12:34","Channel":6,"RSSI":100,"Signal":-39,"LinkCount":1,"Downtime":"0T00:00:05"}}

Yon can either setup the device with extra commands over the serial cable, or do the same in the web console :

backlog MqttHost 10.1.0.90; NTPserver 10.1.0.110; Timezone 0; TimeDST 0,0,3,1,1,0; TimeSTD 0,0,10,1,2,0;
backlog SetOption56 1; SetOption57 1; GroupTopic alldevices;
etc.

Most commands need a reboot, but using the console is so much quicker that clicking through  the web interface.

Testing

I tested mine with a simple lamp plugged in to the socket and the socket plugged in to the mains.

By default all Tasmota devices have a “Fallback” topic. This is an MQTT topic which is unique to that device. You do not need to manually configure it (it is derived from the device mac address) and it is listed in the information page on the device.

MQTT Fallback Topic cmnd/DVES_3DDEE5_fb/

The Tasmota / Sonoff device is automatically subscribed to this topic. So, if you send an MQTT Message to this topic from anywhere, it will control the device.

From the Raspberry Pi terminal enter this command using your fallback topic details.

mosquitto_pub -t "cmnd/DVES_XXXXXX_fb/POWER" -m "ON"
then
mosquitto_pub -t "cmnd/DVES_XXXXXX_fb/POWER" -m "OFF"

The lamp will go on and off ! Genius ! 😊

I flashed about a dozen Sonoff S20 plugs and scattered them around the house on various lamps and devices I needed to control. I use them to control lamp in rooms, the lights on my fish tank, the pump in the garden fountain, etc. Nearly everything needs to come on at a particular time in the morning and go off at night.

Automation

I used NodeRed to automate my smart sockets.

Use the fallback topic for the NodeRed topic name in both elements with /POWER at the end. E.g. “cmnd/DVES_3B7DE8_fb/POWER”

Use the payload “ON” or “OFF” to turn the device on or off.

The Inject node can be set to inject at a specific time, or day, or repeat, etc.

I ran this for a few weeks and none of my devices missed a message. They all came on and went off as defined in NodeRed. The Sonoff devices are stable as are Mosquitto and NodeRed.

After a bit more reading, I discovered a custom node call BigTimer, which everyone seems to use instead of the built in “Inject” node. This can be installed through the interface. Just go to “Manage Palette” in NodeRed and search for “bigtimer” which will find “node-red-contrib-bigtimer” and install it.

This combines on and off messages in a single node, making the workspace a lot less cluttered, plus it has all kind of other useful features like “sun rise” and “sun set” which takes your GPS location and works out when it will get dark. Useful for controlling fish tank lights or outside lights.

You can also define certain days or months the command will operate in, useful for having your outside watering system only on during the summer months, for example.

I replace all my inject nodes with BigTimer and I have been using that without issue ever since.

My garden fountain is on during the day at sunrise, and automatically goes off at sunset !

The final nice to have is a mobile app that I can use to manually override the automation if required and switch things on and off manually. The easiest way to do this initially is to use the Dashboard feature of NodeRed. The limitation of this is that I can only access it from within my local network as my NodeRed server is not open to the public on my firewall. However this is acceptable for the time being. We can change it later and add more feature. There are plenty of full blown home automation interfaces I can add later, including Google Home automation with voice control.

The Dashboard feature of NodeRed can be installed using the Palette Manager, or following the instructions on the dashboard website.

You can add gauges, buttons, switches, etc. To start with, I added a switch for each socket. You can just add the switch to the flow and connect it to an MQTT topic. This works, but if you manually change the state of the switch outside of NodeRed, by pressing the physical button on the socket for example, the dashboard element wont know about this and update the state.

To fix this you have to subscribe to the “stat” status messages coming from Tasmota on the device and feed them back in to the dashboard switch element. I added a simple function before the dashboard switch element to change the topic from the “stat” topic it receives to a “cmnd” topic the switch needs to change state.

For example, For SocketA I subscribed to “stat/socketa/POWER” in NodeRed and the function after this node changes the topic to “cmnd/socketa/POWER” and feeds it in to the dashboard element. The output of the dashboard element send the message out as MQTT again when the switch is pressed.

msg.topic = "cmnd/socketa/POWER";
return msg;

My final NodeRed flow looks like this for all my sockets. I have a dashboard flow for each one, and a BigTimer node for those that I need to automate.

NodeRed flow for control of Sonoff Sockets

NodeRed flow for control of Sonoff Sockets

Screenshot of the NodeRed Dashboard interface on my phone.

Screenshot of the NodeRed Dashboard interface on my phone.

Home automation phase one complete ! 🙂

Hot water tank monitoring

I have electrically heated hot water in my house, and it takes ages to heat a whole tank. It heats up overnight on low cost electricity and we use it during the day. One constant frustration I have is there is no way of knowing how much hot water is left in the tank. I have to canvas my family and interrogate them on who’s had a shower or bath and how much washing up has been done. It would be so much better if I had a web page that showed the remaining hot water level in the tank !?

So, I attached temperature sensors up the side of the hot water tank and monitored them with a Raspberry Pi, of course ! 🙂

I have always wanted to use 1-wire devices in a project, and this was the perfect project. 1-wire devices make the cabling of many sensors easy as they all sit on a shared bus. You can get away with just 2 wires, a signal and a ground, as the devices can consume parasitic power from the signal line before communicating. However, to avoid any power issues and as I didn’t have a problem running more wires, I opted for three wires, power, signal and ground.

I am using the MAX31820 1-wire temperature sensors. They have a simple 3 pin transistor style package (TO-92) and there are lots of existing libraries and support for them. They’re also cheap. I decided to connect them using a ribbon cable, so I made little PCBs for them with a 4 pin header, to make the ribbon cable connection easy. I ended up using 6 way ribbon cable with 6 pin sockets, as it turns out 4 pin IDC sockets are a very difficult to find, where as 6 pin sockets are cheap as chips.

MAX31820 1-wire temperature sensor mounting PCB

MAX31820 1-wire temperature sensor mounting PCB

MAX31820 1-wire temperature sensors on small PCBs

MAX31820 1-wire temperature sensors on small PCBs

Example MAX31820 1-wire temperature sensors on small PCB attached to a ribbon cable

Example MAX31820 1-wire temperature sensors on small PCB attached to a ribbon cable

I already had a USB Adapter for 1-wire, which conveniently is supported by most of the 1-wire Raspberry Pi libraries. However, there are lots of ways of connecting 1-wire devices to a Raspberry Pi. The USB adapter is DS9490R. It takes an RJ11 plug to connect to the 1-wire bus, which also conveniently attaches to my ribbon cable. So, the solution is surprisingly neat.

The tank is pretty standard. Its covered in a insulated foam. I arbitrarily decided that 10 sensors was probably enough resolution. I marked the tank and made 10 equally spaced holes in the insulation with a pencil, then cleaned out the foam with the eraser end of the pencil, until I could see the shiny copper. I dabbed a blob of thermal paste in each hole, to assist with heat transfer from the copper tank to the temperature sensor, as I wouldn’t get direct mechanical contact. The boards and sensors just friction fitted in to the holes, and are kept in by the resistance of the foam on the boards. They are pretty snug and don’t fall out.

The cabling is simple, just attaching IDC connectors at the same regular intervals and then connecting the string of sensors to the Raspberry Pi.

Hot water tank

Hot water tank

Hot water tank with holes for temperature sensors marked

Hot water tank with holes for temperature sensors marked

Hole in the foam insulation showing the shiny copper tank

Hole in the foam insulation showing the shiny copper tank

Temperature sensor on mounting board inserted in the foam insulation

Temperature sensor on mounting board inserted in the foam insulation

Temperature sensors wired together with ribbon cable

Temperature sensors wired together with ribbon cable

Raspberry Pi connected via USB 1-wire adapter to string of sensors

Raspberry Pi connected via USB 1-wire adapter to string of sensors

Raspberry Pi connected via USB 1-wire adapter to string of sensors

Raspberry Pi connected via USB 1-wire adapter to string of sensors

Once the sensors were mounted and connected I installed OWFS (One Wire Filing System) on the Raspberry Pi which is a really easy way to read 1-wire sensors. It maps all the sensors found on the bus to a file system, so by reading the files you can read all the data and configure of all the connected sensors. Again there are loads of ways to access 1-wire devices. Using OWFS just made it easy for me. You could use Python or C, or any other myriad of languages, which all have 1-wire libraries.

Here are some instructions to install OWFS on a Raspberry Pi.

The OWFS maps the devices to a mount point. If you list the contents you get something like this :

root@raspberrypi:# cd /mnt/1wire/
root@raspberrypi:/mnt/1wire# ls -la
total 4
drwxr-xr-x 1 root root 4096 Nov  4 16:17 .
drwxr-xr-x 3 root root 4096 Mar  3  2015 ..
drwxrwxrwx 1 root root 4096 Mar 20 19:46 28.0BA759050000
drwxrwxrwx 1 root root 4096 Mar 20 19:46 28.168F59050000
drwxrwxrwx 1 root root 4096 Mar 20 19:46 28.229659050000
drwxrwxrwx 1 root root 4096 Mar 20 19:46 28.33A559050000
drwxrwxrwx 1 root root 4096 Mar 20 19:46 28.49B159050000
drwxrwxrwx 1 root root 4096 Mar 20 19:46 28.849459050000
drwxrwxrwx 1 root root 4096 Mar 20 19:46 28.AAB059050000
drwxrwxrwx 1 root root 4096 Mar 20 19:46 28.BBA459050000
drwxrwxrwx 1 root root 4096 Mar 20 19:46 28.C8AA59050000
drwxrwxrwx 1 root root 4096 Mar 20 19:46 28.E6B059050000
drwxrwxrwx 1 root root 4096 Mar 20 19:46 81.E1E324000000
drwxr-xr-x 1 root root 4096 Nov  4 16:17 alarm
drwxr-xr-x 1 root root 4096 Nov  4 16:17 bus.0
drwxr-xr-x 1 root root 4096 Nov  4 16:17 settings
drwxrwxrwx 1 root root 4096 Mar 20 19:46 simultaneous
drwxr-xr-x 1 root root 4096 Nov  4 16:17 statistics
drwxr-xr-x 1 root root 4096 Nov  4 16:17 structure
drwxr-xr-x 1 root root 4096 Nov  4 16:17 system
drwxr-xr-x 1 root root 4096 Nov  4 16:17 uncached

In each directory you can read either the cached or uncached version of the data, plus other details about the device.

root@raspberrypi:/mnt/1wire/28.0BA759050000# ls -la
total 0
drwxrwxrwx 1 root root 4096 Mar 20 19:46 .
drwxr-xr-x 1 root root 4096 Nov  4 16:17 ..
-r--r--r-- 1 root root   16 Nov  4 16:17 address
-rw-rw-rw- 1 root root  256 Nov  4 16:17 alias
-r--r--r-- 1 root root    2 Nov  4 16:17 crc8
drwxrwxrwx 1 root root 4096 Mar 20 19:46 errata
-r--r--r-- 1 root root    2 Nov  4 16:17 family
-r--r--r-- 1 root root   12 Nov  4 16:17 fasttemp
-r--r--r-- 1 root root   12 Nov  4 16:17 id
-r--r--r-- 1 root root   16 Nov  4 16:17 locator
-r--r--r-- 1 root root    1 Mar 20 19:46 power
-r--r--r-- 1 root root   16 Nov  4 16:17 r_address
-r--r--r-- 1 root root   12 Nov  4 16:17 r_id
-r--r--r-- 1 root root   16 Nov  4 16:17 r_locator
-r--r--r-- 1 root root    9 Mar 20 19:46 scratchpad
-r--r--r-- 1 root root   12 Nov  4 16:17 temperature
-r--r--r-- 1 root root   12 Nov  4 16:17 temperature10
-r--r--r-- 1 root root   12 Nov  4 16:17 temperature11
-r--r--r-- 1 root root   12 Nov  4 16:17 temperature12
-r--r--r-- 1 root root   12 Nov  4 16:17 temperature9
-rw-rw-rw- 1 root root   12 Mar 20 19:46 temphigh
-rw-rw-rw- 1 root root   12 Mar 20 19:46 templow
-r--r--r-- 1 root root   32 Nov  4 16:17 type

root@raspberrypi:/mnt/1wire/28.0BA759050000# cat address 
280BA759050000FA

root@raspberrypi:/mnt/1wire/28.0BA759050000# cat family 
28

root@raspberrypi:/mnt/1wire/28.0BA759050000# cat temperature
51.375

root@raspberrypi:/mnt/1wire/28.0BA759050000#

I run a shell script every 5 minutes to read the latest data from all the sensors and store it in a text file.

> cat /root/scripts/logtemp.sh 

#!/bin/sh
date | tr -d '\n' >> /home/pi/templog.txt
echo -n ',' >> /home/pi/templog.txt
cat /mnt/1wire/28.E6B059050000/temperature >> /home/pi/templog.txt
echo -n ',' >> /home/pi/templog.txt
cat /mnt/1wire/28.0BA759050000/temperature >> /home/pi/templog.txt
echo -n ',' >> /home/pi/templog.txt
cat /mnt/1wire/28.49B159050000/temperature >> /home/pi/templog.txt
echo -n ',' >> /home/pi/templog.txt
cat /mnt/1wire/28.168F59050000/temperature >> /home/pi/templog.txt
echo -n ',' >> /home/pi/templog.txt
cat /mnt/1wire/28.C8AA59050000/temperature >> /home/pi/templog.txt
echo -n ',' >> /home/pi/templog.txt
cat /mnt/1wire/28.33A559050000/temperature >> /home/pi/templog.txt
echo -n ',' >> /home/pi/templog.txt
cat /mnt/1wire/28.849459050000/temperature >> /home/pi/templog.txt
echo -n ',' >> /home/pi/templog.txt
cat /mnt/1wire/28.BBA459050000/temperature >> /home/pi/templog.txt
echo -n ',' >> /home/pi/templog.txt
cat /mnt/1wire/28.229659050000/temperature >> /home/pi/templog.txt
echo -n ',' >> /home/pi/templog.txt
cat /mnt/1wire/28.AAB059050000/temperature >> /home/pi/templog.txt
echo ',END' >> /home/pi/templog.txt

I then use this data from a simple web page written in Python to display the latest temperatures in the tank. I decided 35 degrees Celsius is “cold”. If the water is below this I show it as blue (Cold), if its above this I show it as red (Hot). This is open to future expansion where I can fade the colour from red to blue depending on the temperature, more red for hot, more blue for cold, rather than a single colour, but that’s for another day.

> cat hotwater.py 

#!/usr/bin/env python
import cgi
import cgitb
cgitb.enable()

form = cgi.FieldStorage()

print 'Content-type: text/html\n\n'
print '<html><head></head><body>'

fn = '/home/pi/templog-latest.txt'
f = open(fn, 'r')
t = f.readline()
f.close()

temps = t.split(',')

o = ''

if form.getvalue("up","true") == 'true':
        up = True
else:
        up = False

for temp in temps:
        if len(temp) > 8:
                d = temp +"<br>"
        elif len(temp) < 2:
                if up:
                        o = o +'<tr><td bgcolor="#00FF00" align="center"></td></tr>'
                else:
                        o = '<tr><td bgcolor="#00FF00" align="center"></td></tr>'+ o
        elif temp.strip('\n') == 'END':
                z = 1
        else :
                n = int(float(temp))
                if n > 35:
                        if up:
                                o = o +'<tr><td bgcolor="#FF0000" align="center">'+ str(n) +'</td></tr>'
                        else:
                                o = '<tr><td bgcolor="#FF0000" align="center">'+ str(n) +'</td></tr>'+ o
                else:
                        if up:
                                o = o +'<tr><td bgcolor="#0000FF" align="center">'+ str(n) +'</td></tr>'
                        else:
                                o = '<tr><td bgcolor="#0000FF" align="center">'+ str(n) +'</td></tr>'+ o

print d +'<table width=150>'+ o +'</table>'
print '</body></html>'

I can access the web page from anywhere in my house, on my mobile phone, and it instantly shows the quantity of hot water left in the tank, plus each of the 10 individual temperature readings from the sensors. The query string parameter in the web page script reverses the order of the sensor readings in the tank. “up=false” or with no query string, is the default setting, where the temperatures are shown in the correct order, with the hottest water “floating” at the top of the tank. The alternate view, is for users who don’t want a view based on physics, and would rather the visualise a tank full of hot water emptying from the bottom. In this view the hot water is seen at the bottom, slowly draining away to empty. Not scientifically correct, but easier to explain to my 8 year old. Either way the bookmark is saved with the appropriate query string parameter, and everyone in my house can now tell how much hot water is left, before they get in the shower. No more showers suddenly going cold half way through.

Hot water monitoring web page. This shows the tank about half full of hot water. The top of the tank at 45 degrees. The bottom of the tank at 16.

Hot water monitoring web page. This shows the tank about half full of hot water. The top of the tank at 45 degrees. The bottom of the tank at 16.

 

Star Wars Death Star Christmas Bauble

A slightly unplanned project. We needed to test a 3D printer in the office, and seeing as it was Christmas, we printed a Death Star Christmas tree bauble. We already had a Star Wars themed Christmas tree, so it fitted nicely.

The Death Star model came from Thingiverse. The print took about 5 hours and as it was only our second 3D print, it was a pretty good quality.

3D printed Death Star

3D printed Death Star

3D printed Death Star

3D printed Death Star

It had a void in the middle, which fitted an Arduino Mini, so I installed some LEDs to make it flash …

I used a 24 RGB LED NeoPixel ring from Adafruit and an Arduino Pro Mini from SparkFun and a high brightness green LED.

Death Star bauble innards

Death Star bauble innards

With the help of some hot glue, I installed the flashing innards. the two halves are held together with bits magnets from an old hard drive. Its powered by a USB power adapter, and the bauble hangs from the cable.

Star Wars Themed Christmas Tree

Star Wars Themed Christmas Tree

I’m pretty pleased with the result.

I got to meet ASIMO at the Honda HQ in Slough.

I got to meet ASIMO at the Honda HQ in Slough. They demonstrated his ability to walk dynamically (i.e. not balancing on a single leg between steps), to run (very impressive), to kick a football, deliver a tray of drinks, face tracking and of course … dance.

According to the guy I spoke to after the event, for that demo, ASIMO had a map of the floor, and the operator was giving him commands to walk from X to Y. The number of steps and the direction, joint control, etc, was then all executed autonomously in ASIMO. This particular model didn’t have sonar, so didn’t have obstacle avoidance, but others do. Same for climbing stairs, the command is given to climb the stairs, then ASIMO calculates the position and joint control autonomously.

Although ASIMO is not truly autonomous, the level of processing is amazing and the aesthetics and design really make you feel relaxed with a robot. You feel compassion for it. I almost felt sorry for it, being made to perform. 🙂 However, the employees who manage ASIMO and careful to refer to ASIMO as an “it” not a “him” or a “her”. Apparently its not a boy or a girl. However, the French member of staff I spoke to, told me there is no French word for “it” and “robot” is a masculine noun, so for him, ASIMO is always little boy ! 🙂

ASIMO at the Honda HQ in Slough - 21 Oct 2014

ASIMO at the Honda HQ in Slough – 21 Oct 2014

ASIMO at the Honda HQ in Slough - 21 Oct 2014

ASIMO at the Honda HQ in Slough – 21 Oct 2014

The Cubli – A balancing, jumping, walking robot cube !

I found this online. A robot developed at ETH Zurich. It is brilliant and beautiful. I had to share …

AVC – 2014 Report

The Sparkfun Autonomous Vehicle Competition this year was lots of fun, as usual. I ran a similar setup to last year, with the same chassis, mBed microcontroller, Magnetometer & GPS. The only difference was a new controller PCB with some extra features, and some changes in the code. The new control board has some incremental improvements from last year, with on-board battery monitor, better layout, built in Mux to multiplex RC and Autonomous control, and an on-board RS232-TLL converter (MAX3221).

AVC 2014 controller PCB v3.2

AVC 2014 controller PCB v3.2

 

AVC 2014 controller board v3.2

AVC 2014 controller board v3.2

The battery monitor uses a shift register to control the 8 LEDs, so I only need 4 IO lines to control 8 LEDs. The mux uses a 74S157D to multiplex the PWM lines from the RC receiver and mBed out to the RC car steering servo and speed controller. By changing the mux select line, I can control the RC car from the RC controller or the mBed, either forced with a jumper (RC_SEL) or by using channel 3 on my RC transmitter. To decode the channel 3 signal and give me a digital output I use a Pololu RC Switch with Digital Output.

The other useful addition was a 1F super capacitor, attached to the vBAT pin on the mBed. One problem from last year was that all my log files were dated “1/1/1970” as when the mBed boots the clock isn’t set, and I create the log file on boot. I set the clock once I have a GPS lock, with the time from the GPS, but until I have GPS lock, I don’t know what the time is. The mBed has a real time clock, I just need to keep it powered between power downs.

Supplying 3.3v to vBAT keeps the RTC running, even when the main power is off. Usually, you would use a small lithium coil cell, but I had just bought some super caps to play with and they seemed perfect. I used a diode and resistor from the 3.3v power rail to charge the super cap while the power is on and limit the charge current. It works perfectly and kept the clock running for days while the main power was discounted. Now, all my log files have the correct date and time, and once I get a good GPS lock, I reset the RTC, just in case it has drifted.

The bot was ready a couple of weeks before the competition, but as usual, I didn’t have time to do much testing, and my flight only got in to Boulder the night before the competition, so I couldn’t spend the day before at the course. The day of the competition arrived and I got there early to setup and do some last minute testing.

I was in group 7 of the Peloton class. Team : “Mostly Robots”. Robot : “Eleanor”.

There are 3 heats (3 attempts at the course). The are timed, with bonus points for navigating obstacles.

AVC Ground Course ((c) Sparkfun)

AVC Ground Course ((c) Sparkfun)

Heat 1

Heat 1 got off to a good start.  I’m always apprehensive as the bot approaches the first corner. Anyone can build a bot that just drives in a straight line and crashes in to the fence. Turning at the first corner autonomously is a good feeling ! 🙂 The bot was swerving badly on the straight, more than last year. Same old magnetometer issues from last year, but it turned perfectly on the 1st corner. Its programmed to avoid the barrels and zig-zag through them, which it did, and made the 2nd corner. However on the 3rd straight it started to wobble more and zig-zag quite badly instead of driving in a straight line. It made the 3rd corner, but then started to get a bit confused. It spun round in circles a couple of times, then crashed in to a bollard. Disappointing, but not bad for a first run.

AVC 2014 - Heat 1 GPS log

AVC 2014 – Heat 1 GPS log

Heat 2

The less said about heat 2 the better :-). It all went wrong, and straight off the start line the bot crashed in the kerb.

Heat 3

Heat 3 started well, the bot made the 1st corner, avoided the barrels and made the 2nd corner. But on the back straight got confused again and crashed in to the kerb.

AVC 2014 - Heat 3 GPS log

AVC 2014 – Heat 3 GPS log

In the end, a fun but disappointing day, as I know the bot can navigate autonomously, as it makes it round three quarters of the course, but the gremlins stopped me from completing all of the three heats.

I came 14th overall out of 25 in my class. Respectable, but I could have done better.

The problem is always the magnetometer. I need to find some time before the next event to work on fine tuning it and experimenting with some different positions to stop interference from the car’s motor and servos.

Roll on next year ! 🙂

 

Raspberry Pi – Stratum 1 NTP Server

This blog is mostly about robots, but not exclusively. 🙂

I needed an NTP server at home, as my security cameras were drifting out of sync. So, I thought I’d create a Stratum 1 NTP Server, with a Raspberry Pi and a GPS receiver.

There are lots of guides on the web, so I just followed the instructions, but it was very easy, once you understand the basics. There are two ways to synchronise the time on the Raspberry Pi to a GPS receiver. The first is just to use the GPS NMEA data to get the time. With a 1 Hz update, you have be accurate to about a second. However, some GPS receivers provide a pulse-per-second output, that provides a digital signal every second that can be attached to a hardware interrupt and this can be accurate to a microsecond.

Nearly all the instructions I used came from here :

I did a couple of other things too, as I had trouble keeping my Raspberry Pi’s running for longer than 3 months on an SD card. After 3 months of being continuously on, I usually got SD card corruption and the server died. I decided to switch this Raspberry Pi to use a USB flash drive to the filing system, and the SD card, just to boot.

The instructions for using a USB flash filing system are here :

I did this first, then configured the GPS / NTP. For this post, I’ll ignore the USB configuration, and it’s not relevant to the GPS / NTP install. It’s up to you if you need it. I’ve heard of lots of people that have run a Raspberry Pi for over a year on an SD card without any issues, and others who have had the same issues as me, with no more than a few months without file system corruption.

The first thing to do is connect the GPS receiver to the RPi. I used an Adafriut Ultimate GPS breakout board.  It has lots of useful features; it’s cheap, it’s easy to connect as it has 0.1” headers, it has a PPS output, it’s 3.3v and it has an external aerial mount.

Adafruit Ultimate GPS Breakout Board

Adafruit Ultimate GPS Breakout Board

The only problem I found with these GPS units is I couldn’t find a way to permanently change the baud rate. If you set the baud rate on the GPS receiver, the setting doesn’t survive a power cycle. If anyone knows how to do this, let me know. Even in production I am running the GPS at 9600 baud. I would have preferred to run it at a faster rate, which would give better accuracy as the NMEA data can be processed faster. But seeing as the clock is set using the PPS, its not a major issue.

I used a Slice of Pi prototype board to make it easy to connect the GPS to the Raspberry Pi. The wiring is straight forward, 3.3v and Gnd, Tx & Rx, and the PPS connected to GPIO pin 8.

Slice of Pi prototype board

Slice of Pi prototype board

I also have a Power Over Ethernet switch, so I added a PoE power adapter, so the Raspberry Pi would have minimal wiring. I bought a plastic box from Maplins and mounted the PoE adapter, RPi, GPS and external GPS connector all in the box. The box had a clear plastic lid, which also means you can see the RPi in action.

Power Over Ethernet adapter

Power Over Ethernet adapter

Assembled Raspberry Pi NTP Server

Assembled Raspberry Pi NTP Server

Assembled Raspberry Pi NTP Server in case with PoE

Assembled Raspberry Pi NTP Server in case with PoE

Once the hardware was attached, the software configuration follows the instruction linked above.

1. Disable serial comms on the console. We need it for the GPS.

Disable console output to serial

sudo vi /boot/cmdline.txt

find this :

dwc_otg.lpm_enable=0 console=ttyAMA0,115200 kgdboc=ttyAMA0,115200 console=tty1 root=/dev/mmcblk0p2 rootfstype=ext4 elevator=deadline rootwait

and edit it to this :

dwc_otg.lpm_enable=0 console=tty1 root=/dev/mmcblk0p2 rootfstype=ext4 elevator=deadline rootwait

Disable the login prompt

sudo vi /etc/inittab

find the line near the end

T0:23:respawn:/sbin/getty -L ttyAMA0 115200 vt100

add # to comment it out.

#T0:23:respawn:/sbin/getty -L ttyAMA0 115200 vt100

2. Configure Static IP

I need my NTP server to have a static IP

If you use DHCP, remove “ntp-servers” from the “request” line in /etc/dhcp/dhclient.conf

Remove /var/lib/ntp/ntp.conf.dhcp if present

sudo vi /etc/network/interfaces

Edit the following entry:

iface eth0 inet dhcp

Change it to (using your local network settings):

iface eth0 inet static
address 10.1.0.128
netmask 255.255.255.0
gateway 10.1.0.1
network 10.1.0.0
broadcast 10.1.0.255

reboot…

At this point you should see the GPS NMEA data coming from the serial port. Test it like this :

sudo cat /dev/ttyAMA0

If you dont see NMEA data, go back and check your wiring and baud settings.

3. Install gpsd

sudo apt-get install gpsd gpsd-clients python-gps
sudo gpsd /dev/ttyAMA0 -F /var/run/gpsd.sock

test it recieves GPS data :

sudo cgps –s

+-------------------------------------------++---------------------------------+
|    Time:       2014-07-07T22:34:15.000Z   ||PRN:   Elev:  Azim:  SNR:  Used: |
|    Latitude:    51.501114 N               ||  20    73    246    32      Y   |
|    Longitude:   -0.142448 W               ||   1    72    117    44      Y   |
|    Altitude:   99.8 m                     ||  32    61    068    51      Y   |
|    Speed:      0.0 kph                    ||  11    45    142    38      Y   |
|    Heading:    67.1 deg (true)            ||  17    43    296    21      Y   |
|    Climb:      0.0 m/min                  ||  33    29    199    00      Y   |
|    Status:     3D FIX (10 secs)           ||  23    28    180    00      Y   |
|    Longitude Err:   +/- 4 m               ||   4    20    294    28      Y   |
|    Latitude Err:    +/- 3 m               ||  31    12    085    42      N   |
|    Altitude Err:    +/- 9 m               ||  14    11    035    17      N   |
|    Course Err:      n/a                   ||  28    08    250    00      N   |
|    Speed Err:       +/- 31 kph            ||  13    02    191    00      N   |
|    Time offset:     0.662                 ||  19    01    161    00      N   |
+-------------------------------------------++---------------------------------+

4. Configure gpsd to auto start

sudo dpkg-reconfigure gpsd

The configuration program will ask you a series of questions :

Start gpsd automatically? Yes
Should gpsd handle attached USB GPS receivers automatically? No
Device the GPS receiver is attached to: /dev/ttyAMA0
Options to gpsd: -n
gpsd control socket path: /var/run/gpsd.sock

reboot…

sudo cgps –s

You should get the same output as before. This shows everything is starting up correctly on boot.

5. Configure NTP

Make a backup of the ntp.conf file. Edit the conf file.

sudo cp /etc/ntp.conf /etc/ntp.old.conf
sudo vi /etc/ntp.conf

Remove the IP access restrictions

Comment out :

restrict -4 default kod notrap nomodify nopeer noquery

#restrict -4 default kod notrap nomodify nopeer noquery

Configure NTP to use the UK NTP pool servers.
I’m in the UK. If you’re not, pick the most local pool to you.

Replace:

server 0.debian.pool.ntp.org iburst
server 1.debian.pool.ntp.org iburst
server 2.debian.pool.ntp.org iburst
server 3.debian.pool.ntp.org iburst

With:

pool uk.pool.ntp.org iburst

Add the local NTP server from your ISP.

server ntp.eclipse.co.uk iburst

Restart NTP and check its working.

sudo /etc/init.d/ntp restart
sudo ntpq -p -n

     remote           refid      st t when poll reach   delay   offset  jitter
==============================================================================
*213.249.130.101 10.100.94.8      2 u    3   64  377  120.208   37.434  53.254
+87.124.126.49   195.66.241.3     2 u   10   64  377   32.008   -8.111  99.007
 82.219.4.30     .INIT.          16 u    - 1024    0    0.000    0.000   0.000
 217.10.144.130  .INIT.          16 u    - 1024    0    0.000    0.000   0.000
-151.230.30.222  103.7.151.4      2 u   25   64  377   47.016    0.743  98.865

Basic NTP is now working, but without GPS. Even without GPS, its still accurate to about 37ms.

6. Add the GPS configuration to NTP

Edit ntp.conf

sudo vi /etc/ntp.conf

add :

# Server from shared memory provided by gpsd
server 127.127.28.0 minpoll 4 maxpoll 4
fudge  127.127.28.0 time1 0.000 refid GPS

Restart NTP and check its working.

sudo /etc/init.d/ntp restart
sudo ntpq -p -n

     remote           refid      st t when poll reach   delay   offset  jitter
==============================================================================
*127.127.28.0    .GPS.            0 l    7   16  377    0.000   -353.23 15.024
-213.249.130.101 10.100.94.8      2 u   62   64  377   52.603   -3.554  77.753
-87.124.126.49   195.66.241.3     2 u    6   64  377   42.207   -5.468  78.352
 82.219.4.30     .INIT.          16 u    - 1024    0    0.000    0.000   0.000
 217.10.144.130  .INIT.          16 u    - 1024    0    0.000    0.000   0.000
+151.230.30.222  103.7.151.4      2 u   16   64  377   47.016    0.743  58.425

Adjust the time1 fudge offset to compensate for drift. The GPS NMEA input is taking about 350ms to process.

fudge  127.127.28.0 time1 0.350 refid GPS

Restart NTP and check again. The GPS inout should be more accurate. (now only 11ms out)

     remote           refid      st t when poll reach   delay   offset  jitter
==============================================================================
*127.127.28.0    .GPS.            0 l    3   16  377    0.000   11.721  38.208
+213.249.130.101 10.100.94.8      2 u   33   64  377   72.434    1.227  98.033
-87.124.126.49   195.66.241.3     2 u   48   64  377   50.215   -0.824  76.877
 82.219.4.30     .INIT.          16 u    - 1024    0    0.000    0.000   0.000
 217.10.144.130  .INIT.          16 u    - 1024    0    0.000    0.000   0.000
-151.230.30.222  103.7.151.4      2 u   56   64  377   48.522    0.481  50.334

7. Install user mode PPS module

For more details see : http://vanheusden.com/time/rpi_gpio_ntp/

sudo wget http://vanheusden.com/time/rpi_gpio_ntp/rpi_gpio_ntp-1.3.tgz
sudo tar -zxvf rpi_gpio_ntp-1.3.tgz
sudo cat rpi_gpio_ntp-1.3/readme.txt

Read the instructions. You must build and install the program:

sudo cd rpi_gpio_ntp-1.3
sudo make install

You probably want to let rpi_gpio_ntp start at boot.

To do so, edit /etc/rc.local

sudo vi /etc/rc.local

and add the following line (BEFORE the “exit 0” statement and AFTER the “#!/bin/sh” line):

/usr/local/bin/rpi_gpio_ntp -N 1 -g 8

This assumes that the PPS signal of the GPS is connected to GPIO pin 8 which is physical pin 24.

reboot…

8. Add the PPS configuration to NTP

Edit ntp.conf

sudo vi /etc/ntp.conf

add :

# Server from PPS module 
server 127.127.28.1 minpoll 4 maxpoll 4 prefer
fudge  127.127.28.1 refid PPS

Restart NTP and check its working.

sudo /etc/init.d/ntp restart
sudo ntpq -p -n

     remote           refid      st t when poll reach   delay   offset  jitter
==============================================================================
+127.127.28.0    .GPS.            0 l   15   16  377    0.000   65.031  38.403
*127.127.28.1    .PPS.            0 l   14   16  377    0.000    0.001   0.006
x213.249.130.101 10.100.94.8      2 u   12   64  377   79.034   16.835  82.844
+87.124.126.49   195.66.241.3     2 u   26   64  377   28.945    1.316  42.948
 82.219.4.30     .INIT.          16 u    - 1024    0    0.000    0.000   0.000
 217.10.144.130  .INIT.          16 u    - 1024    0    0.000    0.000   0.000
-151.230.30.222  103.7.151.4      2 u   39   64  377   36.126    2.120  57.079

This shows the Raspberry Pi system clock is accurate to 1us !!

My final ntp.conf file :

driftfile /var/lib/ntp/ntp.drift

statistics loopstats peerstats clockstats
filegen loopstats file loopstats type day enable
filegen peerstats file peerstats type day enable
filegen clockstats file clockstats type day enable

# Server from shared memory provided by gpsd
server 127.127.28.0 minpoll 4 maxpoll 4
fudge  127.127.28.0 time1 0.550 refid GPS

# Server from PPS module 
server 127.127.28.1 minpoll 4 maxpoll 4 prefer
fudge  127.127.28.1 refid PPS

# You do need to talk to an NTP server or two (or three).
server ntp.eclipse.co.uk iburst

# pool.ntp.org maps to about 1000 low-stratum NTP servers.  Your server will
# pick a different set every time it starts up.  Please consider joining the
# pool: 
pool uk.pool.ntp.org iburst

# By default, exchange time with everybody, but don't allow configuration.
#restrict -4 default kod notrap nomodify nopeer noquery
#restrict -6 default kod notrap nomodify nopeer noquery

# Local users may interrogate the ntp server more closely.
restrict 127.0.0.1
restrict ::1

The final NTP Server installed in my garage, with an external GPS aerial on the roof.

Assembled Raspberry Pi NTP Server in case with PoE

Assembled Raspberry Pi NTP Server in case with PoE