September 2019 Update

New! We're moving to the new forum, eventually it will replace this blog. Please join us there.

View the Most Recent Feature Sheet by clicking HERE!

Sunday, September 15, 2019

ESP-01 Programming Board


I made up a couple of these back when I first got the ESP-01's, but I guess I never documented it, or bookmarked any pages, or anything.  I ended up reverse engineering mine, but had trouble with the color codes on the resistors, so to verify, I scoured google, and eventually found this set of instructions which included the following schematic:




So, combining his schematic, with my existing programmers, I drew the following wiring diagram based on the component placement and board size I wanted. On the right, you can see I'm about half way through assembling it. 




Once I verify it works, I want to build a board with 6 sockets for the ESP's one power supply for all, so it will be like a mini server farm :)  Somehow I'll select which one is connected to the tx/rx of the Serial/USB adapter for programming... but first, will this one work?


Followups posted on the forum at https://growbywire.boards.net/thread/18/esp-01-programming-board

Surrounded!

I'm sitting here at 6am, surrounded by the disassembled components of half the system.  I started off by wanting to build an esp-01 programming board, so I look back through my notes, photos, bookmarks, and no reference on how to build one.  I must be going color blind, because I can't make heads nor tails of the color bands on the resistors.  They are pullups to 23.3v and I believe they would be 10K Ohms, but, the color bands definitely is not 10K, and they measure about 8.8K on my meter.

I only found one schematic online that actually used resistors and they are 10K so that's close enough...  At this point, I started looking at the custom shields I had on the Switch Module, the Auto Watering Module, and the sensor module, and that's what led to all these parts scattered about.


I'm used to having one torn apart and not functioning, but three?  I really didn't accomplish much last night, but I guess I'm gonna have some good days, and some bad, er, not so good days.... It wasn't a bad day, nothing is broken, just disassembled.

Actually, the night started with me wanting to work on the Switch Module v2 code, since it doesn't do lights yet, and now that I bought that contactor, I'm ready to bring the rest of the working code over from version 1 and let it control the light in the flower tent.

The weather is turning, it's not so hot any more, so soon the A/C will come out, and a box fan goes into the window, along with my fancy light baffle, made from black dollar store foam board. That fan needs to be controlled by the Switch Module. Last but not least, I'll need it running the Clone Mister soon.

Saturday, September 14, 2019

Oops! Now what?

Posted: 12:40am

Well, I did it, I knew I would eventually...

The old Switch Module that has no usb plug on it due to my clumsy feet catching the usb cable and yanking it off the desk, ripping the usb connector half off the board, is now my development board until a new one arrives. Of course, that left only OTA as a method to load new sketches.

I just uploaded a sketch to the 8266 that throws an exception immediately and reboots the 8266 without loading the OTA server,  No more OTA updates, no updates over USB...

The Wemos does expose a bunch of pins from the ESP8266 which I think I can use to upload a sketch.



The pins are labeled, so that's not a problem...

Here is the ESP-01 version of the 8266, it contains fewer pins, and requires setting it into programming mode by shorting certain pins, and adding resistors.  The second picture shows the custom programming boards I made for the WebServ er (single board) and maintenance server (dual board)



I'll be doing up some more of the programming boards since I just got 4 more ESP-01's as spares, and I'll do a DIY build post on it with schematics and steps. It''s a simple task.

So I wonder will I need to make up a programming board to plug into these pins, or is it already done for me, I haven't been able to find any info on doing it this way, so we'll see how it goes.

Followups posted on the forum at https://growbywire.boards.net/thread/15/oops


Friday, September 13, 2019

Time to recalibrate?

Since I added the 5v external power supply on the Sensor Modules, it has had an effect on the readings of the sensors. The system looks at historical data to determine the soil moisture as a percentage. If the highest value ever read on a sensor in a pot with dirt was say 800 and the lowest was 200, then rather than calculating the moisture level as a percentage from 0-1023 we would calculate it as a percentage from 200-800. So if the reading on the sensor was 400, that would be roughly 40% if calculated between 0 and 1023, but if you calculate it between 200-800, then 400 is actually about  33%.  With the new external power supply, this could affect the readings, so the sensors should have been re-calibrated to find their values in dry air, and sitting in a glass of water, as this will affect the final calculated value, or percentage.

Here is what I'm seeing now on the web page showing the Grow Room Overview, this is only the Flower section.


There are two plots which are pegged at 100% and have been since they were last watered. It seems the readings now are higher than before, so it will be a while before it sinks below the calibrated WET value, and continue to report 100%. 

Re-calibration involves pulling the sensor out of the pot, touching a button on my phone to calibrate the value in dry air, then putting the sensor in a glass of water, and touching another button. Pretty simple actually, assuming you have easy access to the sensors :)

Also, the estimates of time till next watering are way off, and in some instance, non-existent, they are also based on historical data.


OTA Updates timing out

Now that I've moved all the processing from the Mega 2560 to the ESP8266, the Over The Air firmware updates ate timing out a lot. I presume this is because the 8266 is busy communicating with the 2560 and doing database reads and writes.  Sometimes it can take 5 or 6 tries :(

The unit I'm using for development no longer has the usb connector on it (see previous blog post) it was the switch module, and I caught the cable on my foot and yanked it off the work table, and ripped the connector half off. So now to program it, I use the tx/rx pins on the 2560 (and have to co-ordinate hitting reset and upload at the same time) but without using the CH340 USB-SERIAL chip, I cannot access the ESP8266 except by WiFi using the OTA upload. The ESP's pins are available, but I haven't explored whether I need to use resistors or anything like I do with the ESP-01 boards.

So, my best solution is to keep OTA working.

The solution I came up with takes advantage of the tight coupling between the GBW Terminal software and the Modules themselves. They talk back and forth as it is, exchanging information while you are connected via the USB port. I added a button on the terminal that will pause everything except the esp8266 web server. 

Here's how it works.  Click the button, and it sends {{pauseForOTA}} to the 2560 Module over the serial connection. The 2560 then sends the command PAUSE4OTA to the 8266 and waits for an acknowledgement. It will try 10 times, then abort, and go back to normal processing. If the 8266 acknowledges the command, then the 2560 will no longer do any processing in the loop() method except for a delay(0) and echo received characters to the Terminal program. The only processing the 8266 will do is to run the web server, which will output status info to the serial port, which will be passed to the terminal by the 2560. When the update is finished, you just reboot the whole thing and it goes back to normal processing.

So far, seems to work, sometimes updates are really fast, sometimes slow, but they are much more reliable now.

Windows Updates

I've noticed for a long time now, many years, that when there is a windows update waiting on your computer to be installed, you'll notice your computer misbehaving... Can't put my finger on a particular thing, burt I know that when things aren't working the way they should, I reboot my computer, I've got a LOT going on here with programming, monitoring, running servers, etc. It runs fine normally, but every once in a while, it just needs a reboot, and often when this happens, there's an update waiting to be installed, and my options include update and restart...

That's today's RANT :)

Tuesday, September 10, 2019

A Forum? Really? Why?

I've always liked the way a forum, or message board works as compared to a blog like this one.



The problem with a blog is when I go back and add to a post, it stays buried under all the newer posts, so you don't even realize something new has been added.  On a message board, a thread would be the equivalent of a post here, so for example, this post, the one you are reading now, would be it's own thread on the message board. The most recent post at the top, but if I were to go and add to an older post, then it would appear at the top of the list again because of the new content. I'm not sure why blogs don't allow for this behavior. 

The other reason, although it doesn't appear we need it yet, is on a message board, others can join in the conversation, or even create their own threads.  I've created an area that will be for me to post my blog posts, and only I can create new threads there, but if you register on the forum, you can reply to those posts. although I warn you now, any off topic posts will be removed. That board is for my blog posts, and any relevant discussions, that's it.  

There will be other boards which will offer more flexibility to members to post to, but you will still be required to stay on topic.  

What the forum is: A place to share idea and discuss ways we can use technology to monitor, and even control the grow room environment.

What the forum is NOT: It's not about growing cannabis.  I repeat, it is NOT about growing cannabis, although that's the reason I've created this system, it can be used for any indoor garden, large or small, and eventually, I plan on expanding to include outdoor growing.  There are lots of other sites out there to discuss how to grow.  I won't be hiding the fact that I grow Cannabis in my garden, and neither should you, but be aware the discussion will always focus on the technology, and the methodology.

This blog post will be the first one copied to the forum, and moving forward, all posts will appear on both sites until such time as I stop posting here on the blog site, and focus entirely on the forum.

So, I guess you need the link:  https://growbywire.boards.net/

Hope to see you there, and please register, you do not have to share any personal information, or even use your real name if you prefer to stay under the radar :)  You will need to be a member in order to post messages, and to be able to come back and see just new posts. 

I'm still getting used to the software, so if you have any suggestions or questions, just ask.

Sunday, September 8, 2019

Switch Module Updates

Posted 6:30pm

At this point the Clone Mister is working, so I could use it if I needed, but no rush yet.  Now I  get to work on the lights, a little more complicated, but I have working code to go by if I need to. I completely rewrote the Servo code and expect to do so with the lights, see what improvements I come up with. I look forward to the first click the high current relay makes as it switches my 600W HPS light on for the first time!

I'm going to have a rest before I get started.


Update: 5:30am

Well, THAT's not gonna work any more....



So much for my soldering skills on super tiny stuff.  I wondered why the Serial Monitor kept stopping, and then finally, I couldn't reset it, or even upload new code, and when I pulled the tape off, this is what I found.

There are pin-outs I can use to program each of the boards, but that's a mini project in itself, figuring them out, how to use them, where to plugs stuff in, etc...  In the mean time, I swapped in the spare development using, which actually works out well, since I'm doing development on the switch module :)  The problem will be if the sensor modules need updating or testing, this was the module I would use.


Update: 5:50am

Knowing where something failed is good, it gives you a chance to fix it, but knowing why, or how it failed is good because then you have a chance to prevent it from happening again.

In this case, I'm pretty sure I found the culprit...



On each side of the USB connector are tabs that get inserted into the two holes on the circuit board, and soldered in place. These two tabs will take the strain from the USB cable being plugged in and moving around.  It looks to me like both of these were cold solder joints, and they really weren't supporting much, that it was the tiny tabs soldered to the board that was holding it.  So some preventative maintenance will be to check all the Wemos Megs boards I have to make sure the  are soldered well.

I think I'll have a go at soldering this connector back onto the board, although to do so will be a small miracle given that I have essential tremors, and can't really aim the soldering iron that closely...


Update: 6:40am

That went almost exactly like I figured it would...


Two of the solder pads have lifted off the circuit board...  Sorry for the blurry picture, I'm at the limit of closeups with my phone. Now I need to figure something out, and order some spares, although they will take about 4-6 weeks to get here...

Update: 6:45am

I just noticed something in the picture, that reddish vertical line almost in the middle of the picture?  That is actually the lifted trace from the board, and you can't see it in the picture, but the other trace is also still there, just up in the air...  that means I know exactly where they go, and just might be able to fix this, although probably not a connector, maybe a permanent soldered connection?

Update: 8:30am

I have a schematic for this board, and was able to identify where the wires needed to hook up. This is the relevant part:



Unfortunately I didn't take any pictures, but first I soldered some very tiny wires (wire wrapping wire) to the end of the fuse F1 and to the ground tab on the power barrel connector receptacle. I then hooked the two data wires to the ends of the resistors, using the lifted, but still attached traces to lead me to the right ones. I hooked the + and - power wires to a USB cable which I had cut the end off, and plugged it in, and the board powered up! Fantastic...

I unplugged it, and hooked up the two data wires to the USB cable, and when I plugged it back in, the computer said the device was unrecognized :(

Reluctantly I ended up ordering another unit, it was less than $18 shipped with tracking from China.  I can manage without the development unit for the time it takes to get the new one, but this rams home the point of always having spares on hand...


Update 10:20am

The Mega Shield has a new feature, can you guess what it is?


I just happened to have some of these which I use to connect to the Pro/Mini's and other boards without usb connectors. They provide a TX/RX/VCC/GND/DTR connection, but I'm not sure what to do with the DTR line. If it just shorts to ground, I could hook it to the reset pin, I'll experiment and see, but meanwhile, I just plug a cable in and I can monitor the serial monitor, but if I want to upload a sketch, I need to push the reset button immediately after telling the software to start uploading. It's a pain in the butt, but it works. As for the ESP8266, I can still do OTA updates, but if I need to upload over the usb, Ill have to figure that out when it happens... I presume just connecting up to the pins directly.

Alright, time for a rest, then back to work. I have a dentist appointment at 2, so the day may not be so productive.

Update: 1:05pm

I found this online, it answers my question...

"Next you need to synchronize the bootloader. Either by manually synchronizing the Reset of the 2560 with the starting of the download (of AVRdude). Or by connecting the DTR(out) of the TTL Serial Port through a 100nF capacitor to the RESET pin of the Arduino."

Update: Sept 11, 2019  11:30pm

I tried the suggestion of putting a capacitor between DTR and RESET, but it did not work, in fact, I couldn't get into programming mode at all by hitting reset, which was working.  I googled it, and it seems this person missed adding a 10K pullup resistor from RESET to VCC, so I added that as well, and still no joy.  So, I removed the wire, capacitor, and resistor, and will just hit reset manually. 

Saturday, September 7, 2019

I'm still working on the Switch Module

Posted: 3:20pm

I know I've been sidetracked more than I've been working on this, but I am making progress. I do need to get the rewrite done, and testing complete before I can take more cuttings, since I'm spoiled now, and won't want to have to mist them manually :)

My reason for starting the rewrite when I did was because I bought the contactor, the high current relay that will allow me to switch the 600W HPS on and off in the flower tent. The code for the lights was working perfectly well, it switched a small relay that wasn't up to the task, and it failed, the arc flash when disengaging ended up welding the contacts together. So nothing needs to change in code, I'll simply use the small relay to toggle the 110v a/c coil voltage for the big relay, plug and play as they say...

While misting the clones was very successful, I am seeing such a huge improvement in reliability with the Sensor Modules after rewriting them, that I want to use the same architecture on all my Wemos Mega's.

Anyhow, to wrap my head around the flow, I drew up this little chart...  I don't usually actually draw them out, but I just bought a whole bunch of paper from staples during their back to school sale :)


It makes sense to me :)   Actually, there were a few minor changes to it, but you get the idea.

So, back to it, at least until I get sidetracked again...

Oooh, donuts!    <=== in my best Homer Simpson voice...

Update: 6:45pm

One change I've made that I will do in all the other modules, I moved the OTA Web Server startup to the earliest possible time. I was running into issues while actually working on it that the code would run into an issue before this and never reach the point of loading the web server, and then there was no way to do an update without pulling the shield off, flipping the tiny dip switches, uploading firmware, then flip the switches back, then pop the shield back on.



Update Sept 8, 2019  12:30am

I've managed to stay somewhat focused on the Switch Module, which is good, because this is a bit more difficult than I originally thought. I have the code finished that manages the scan event, and individual switch timings, the module performs a scan however often you configure it to do so, and during that scan, it loads each switch configuration, and looks at how often that switch needs attention. So you might tell the module to scan every minute, but the switch, say for the Clone Misting only needs to be activated every 10 minutes. The idea will be to time the module scans based on how long all the switch scans take, with a little break in between.

I did look at the updated graphs for the dual level soil moisture sensors, and they certainly are puzzling...





The readings on #64's bottom sensor is the reverse of all the others. Since #64 and #65 are the same design, built at the same time, with identical materials, and the same code on both, this is very confusing.  I'm not using these results for determining watering requirements, I still have the single sensors being used for the two plants.

Alrighty, back to work...


From clumsy feet to surgeons hands!

Posted: 3:00pm

I have lots of wires everywhere in my work area, and I have big clumsy feet especially with the rubber Crocs I wear all the time.




Today I caught the USB cable going to the Switch Module that was sitting on my rolling worktop, yanking it off onto the floor with enough force that it broke the solder connections on the USB connector.  The Mega is screwed to a piece of wood which also contains other parts for the Switch Module, and is somewhat hefty.

These are TINY!!!



You can see them highlighted in yellow.  If you wriggle the cable, the module flickers on and off, making it pretty well useless.  Using my big magnifying lamp, and a smaller magnifying glass (it's that tiny!) I was able to see the little tabs lifting off the main board when I wriggled the cable. So faced with having to replace the whole thing, I grabbed my thinnest soldering tip, and figured I couldn't make it much worse (in retrospect, I suppose I could have) and while holding the USB cable so that the tabs all made contact, I touched them with a hot soldering iron. I didn't add more solder, since I knew cleaning it up after would have been risky...  Once I gave each a little reheat, I plugged it in and it works! So from clumsy feet, to surgeons hands!


Update: September 08, 2019  6:40am

That didn't hold very long unfortunately.  Because I'\m actually working on the Switch Module, I occasionally need to unplug and plug the USB cable in, and physically pick up and move the whole thing, the flex on the cable was more than my fix could handle. I only heated the solder joints, I didn't add any more solder. 

This morning I did add solder, a big blob that covered all the little tabs.  There is no way to get solder on to each individual tab by hand, so the technique I'm aware of is to melt a big blob on the whole area, then lift off the extra using solder wick. Solder wick is just braided copper, and when you lay it over excess solder, and heat it with the iron, it will wick up all that solder, just leaving the solder actually making the connections.  This was my first attempt at an actual rescue job using this technique.  I ended up scraping a tiny bridge off with my xacto knife, but when I plugged it in, it worked and didn't short out my USB port :)

I ended up adding a soft rubber pad under the actual USB cable where it connects, and taping it in place to create a more rigid, rugged connection. The USB cable is almost permanently connected now, in fact, I was thinking of epoxying it in place!   I'll just order some spare boards, they are less than $15 each.




Stupid coding mistake

Sometimes it doesn't pay to code when you are really tired.  I was doing ok, been at it most of yesterday and all night last night, it's 6:30am now.  I was working on something, and I noticed a bit of code that took 4 lines, and I figured I could cut it down to one without losing clarity, meaning if I come back and look at it, can I tell what it's doing easily?  That is generally my number one rule, it doesn't matter how much extra code it takes, it must be readable and understood easily. Anyhow, you can see a before and after of the clearStr method.   The first one works...



The second one doesn't work.

The first one for example, let's use a string "testing", it gets the length, 7, and then loops 0-6 setting the character to null for each. strlen() returns the number of characters BEFORE the null terminator.

The second one, loops from 0 to the length of the string, but the first time through, it sets the first character to a null, which makes the length of the string 0, since there are no characters BEFORE the null. However, there ARE characters AFTER the null...
Before: "testing0"  0=NULL Char
After: "0esting0"

Then along along comes new text for the string, lets say the new string is "pass", and it is added one character at a time, so it adds 4 characters "p" then "a" then "s" then "s"
so the string looks like this as the characters are added...
"0esting0"
"pesting0"
"pasting0"
"passing0"

You end up with the string "passing" not "pass"


For the first example, it would start with 
Before: "testing0"
After: "00000000"

Again, 0 = NULL Character..

Now, if we add 4 characters like before, "pass" the string becomes "pass0000" which is "pass"


Now go back and look at the code again, at first glance, it looks like they do exactly the same thing, especially if you are really tired :)  That's my excuse...  It took me nearly two hours to find the problem, after I eliminated everything else, this was the only other change I had made, so I switched it back, and problem solved, thats when I looked carefully and saw why.

Friday, September 6, 2019

Dual Level Soil Moisture Sensor, a new mystery!

In my post on Sept 3, 2019 I talked about re-introducing the dual level soil moisture sensors, and at the time, they seemed to be working correctly, but I would watch them over time...

Well, something interesting is happening, have a look at sensor #64...


The top sensor didn't change much over the 3 days, that's normal, since I had not watered this pot in a bit. The spike upward is when I just watered it today.

The bottom sensor showed a steady increase in moisture, and then when I watered it today, the moisture level dropped!

The two sensors are reading inverse of each other.


Now we look at Sensor #65.


At least here the trend is toward a drier soil for both sensors, although the difference is small. I'll have to wait a few more days and see how it goes. The biggest surprise is why #64 bottom sensor is reading inverse, but #65 bottom seems normal?  They are the same design, and read with the same code... Let's see how this pans out over the next few days.

Thursday, September 5, 2019

Switch Module Rewrite

Posted: 8:00pm

With the great success rewriting the Sensor Module, I wanted to have a basic template for the design, so I started rewriting the Switch Module, and once it had all the basic infrastructure, I made a copy which can be used to create new modules.  I used it when I was trying to isolate a problem I had in the Sensor Module with MySql, and it only took me the day to turn the template into a working Sensor Module.   

Let's see if I find it as easy to finish the new Switch Module. That's what I'm working on tonight.


Update: Friday Sept 6, 2019 10:10am

As usual, I got sidetracked, this time, in a big way...    I ended up re-arranging my entire workspace, to the point of removing every cabinet, every drawer set, counter top, table, big screen monitor, computers, EVERYTHING except the carpet, which I finally managed to clean completely... The carpet in my workspace is a catchall for every wire clipping, dropped part, etc. The rule is, it's in my space, and I can just throw stuff on the floor, like when I cut the ends off wire, or strip the outer jacket, I don't worry about where it flies...  Every once in a while, it gets vacuumed.

Anyhow, that took me all night, and finally at about 4:30 this morning, I was done, and made my way to bed, hunched over is extreme pain, way too much work for my poor back.

Today is sort of a recovery day, not sure how much coding I'll get done.


SensorModule V2 is live!

Posted: 4:15am

It's (a)live!



There it is, the newest version.  I've ended up removing lots of code that wasn't used any more. I'll keep an eye on it over the next couple days, but it looks good so far!

Happy, happy, happy... It's working again!

Posted: 2:55am

I'm so happy right now! 😊

I finally found, and fixed, the problem with reading the sensor configurations when doing a sensor scan. If you've been following along, I had to close the database after each query or I was getting strange results. Generally it would load the first of two parts for the first sensor, and then all the rest would fail.  By closing the database after each query, it all worked like magic.

In my investigations, which have been going on for a week now, I once noticed that when I did the query for part 2 of the config (there are too many fields to do it in one query due to memory) it would come back with a row, but none of my fields, and when I printed out the field names, they were left over from the part 1 query.  I chalked this up to some sort of memory issue, something I must have done wrong, but it could be anywhere...

So, I started over again, I took the code from my rewrite of the switch module, because it was working, and had all the basic infrastructure., I only had to add the sensor module specific code, which for the most part, I just cut and paste. Each time I would add a tiny bit of code, compile it, upload it, and test it.  Everything was fine until I added the code to read part 1 of the sensor config, which it did, the first time, then failed after that....

That's when I discovered the problem...


This sample query is not the one which caused the problem, this is just an example of a pretty standard sql query, and I've highlighted two lines I had intentionally removed from my query code when loading the sensor configuration.  The do ... while loop loops through the resulting records from the query, however, like this query, I was only expecting a single record back so saw no reason to spend the extra time looping when there were no more records.

Once I added these two lines back into my queries (there were three altogether where I had removed them) I can remove the database close after each query, and everything is working again!

In retrospect, the fact I was seeing leftover field names from the previous query could have been a clue. I haven't looked at the library code enough to check, but it appears that a query returns at least one null row at the end of valid rows, the do while look ensures you have all the records.

By skipping the loop, and only grabbing the first record, somehow the next query first had to read the null row (with the field names from the first query) and this is what caused the problem?  That's my working theory...

I'm going to leave it at that, I now know what I did wrong, every example for the library shows the do...while loop.  I've gone back to my original rewrite code and fixed it, and I can move on, thankfully...

Because I don't have to close the database between queries, it runs much faster now too :)

Wednesday, September 4, 2019

MySql Connector for Arduino - what's going on?

Posted 4:15pm

First off, I'm going to quit doing a single post per day. It will just end up being a big list of dates, with no clue as to what's inside.  Perhaps you aren't interested in some aspects of the project, but you are in others, you'd never be able to just find what you want, so back to the old style of posts, each one sorta geared to a topic.

And this topic is a hot one right now for me.  I recently re-installed windows on my development machine, and in doing so, decided to reinstall all the software from the internet so I would have not only the latest versions, but clean copies of everything. I was having real problems with speed on the computer, it was at a crawl, so I didn't want to reinstall anything from a backup if I could avoid it.

When I started coding again, the ESP8266 would just keep rebooting, the watchdog timer was doing it, and for the life of me, I couldn't figure out why, until finally, I checked the MySql Connector GitHub page, and saw there was an update 8 days prior. That update was not present on my computer before the reinstall, but when I downloaded the library after my reinstall, it was. The patch was supposed to somehow provide better compatibility with ESP8266's but not for me :(  In any case, removing the modification solved the problem, and all was well.

Well, sort of, I am still having a weird problem with the connector, but ONLY in the new SensorModule code on the 8266.  Everything is fine up until it does a sensor scan, and it goes to the database to get a list of sensorId's to scan, and then load the configuration for each before scanning them.

The process is this:
Load list of SensorId's
Loop through SensorId's and load sensorCfg for the specific sensor
  - This is done in 2 parts due to the number of fields 
    - load part 1
    - load part 2
  - send scan request to Mega2560 to scan this sensor, pass configuration data required
  - receive results for scan
  - save results to database
Next SensorId

This is the only process which is causing problems...   there are three queries

fetch sensor id list
fetch sensor config part 1
fetch sensor config part 2

In all my code up till now, I always leave the database open between queries. If I leave it open all the time in the new Sensor Module code on the 8266, then it starts failing, miserably!

Just a note, every time it does a query, it will open a connection to the database if it isn't already open.

It will get the list of Id's the first time through, but when it fetches the config, they will return empty rows or no rows at all. randomly, although rarely correct results.

If I add a database close before each of these three queries, then everything runs smoothly, before and after the scans...


I've been over the code with a fine tooth comb, compared it against working code, and find nothing wrong.

I even started rewriting the Switch Module, and it has no such issues.  

I can do one of two things here, just leave the database close calls before these three queries, and not solve the underlying issue, whatever it is, or I can start fresh, rewrite the SensorModule 8266 firmware again from scratch, and see if it still happens. The second option is time consuming, and has no guarantee the results will be any different but at least I can verify the point at which the problems begin to show up.

I think it's going to be the second option...


Update: Sept 5, 2019 3:00am

Check out my next post!

Happy, happy, happy... It's working again!

Tuesday, September 3, 2019

Sept 3, 2019

Posted: 3:45am

I've been working on the Switch Module firmware the past day.
I've managed to get the basics done in the Mega 2560 firmware. It runs, and does basic communication with the ESP8266 code.  I've got a few little details to check, but then I'll copy both sets of code into some text files so I can just cut and paste it into a new project if I want to add a new module type, or upgrade the old software on the automatic watering module, which I'm sure I will

I'm switching tasks for a little bit here, I want to revisit the Dual Level Soil Moisture Sensors I came up with, and then dismissed because the readings seemed incoherent at the time. I just wasn't seeing a pattern in the readings.

So far, I hooked the sensor directly to the computer via the arduino pro/mini which is acting like a slave i2c device, and actually reads the probes and sends the data back to the Sensor Module.  You can read about it here.

This is the original single probe Soil Moisture Sensor..



 Here is the dual probe Soil Moisture Sensor





As I said, the sensor was hooked directly to the computer using a CP2102 USB to TTL/Serial module so I can program it, and also see the debug info on the serial monitor. I can see that the sensor does actually work, and the readings seem accurate, so now it's time to plug it into a Sensor Module and test it.

The problem is, the new firmware for the Sensor Module doesn't have the code in it to read i2c devices. Since I wasn't using these dual level sensors, I had no other i2c sensors I'm using, so I didn't bother with the code. I'll do that now, and then I can test the sensors again.


Update: 5:00am

Looks like at least this one is working fine, perhaps it was a software issue that has since been fixed?



The yellow highlight is just for my debugging, but you see sensor #64, I have a wet paper tower wrapped around the bottom probe, thats why it only reads bottom. I've tested both.  It is dropping as the towel dries out, that's what I was expecting, so it looks good.

Here's the plant I want to try this with first, its a re-veg, after it flowered and I harvested, I cut it back to just some small clusters of green at the base. They grow back with so much vegetation it's crazy...


I transplanted it to a 17L pail the other day, and it is too deep for a single probe to measure accurately, and is an ideal candidate to test the dual probe sensor. Another reason for choosing this plant over the others, which are small clones, is the sheer amount of vegetation will mean it's going to suck up a lot more water, so I should see results faster, including ensuring the patterns of readings are correct. In other words, is the moisture level dropping at the rate I expect, or going up the right amount when it is watered?  I've got maybe two weeks till the pumps arrive, so it gives me time to certify this before I set these 4 pails to automatic watering....


Update: 7:00am

I'm not sure what happened, but out of nowhere it stopped working comlpetely.  The sensor module can't talk to the i2c slave, and when I try to monitor the serial oputput from the slave, it just doesn't display anything, or it displays garbage, depending where it gets it\s power. I can power the pro/mini from the CP2102 module, or the Sensor Module I2C power pin I provide.  I believe its an issue of not having a common ground, although they shouyld via the usb hub. In any case, it's been a long night, I'm tired, my eyes aren't focusing well even with my glasses on, so this isn't the best time to be plugging wires in between boards on a whim... know what I mean?

So, I put that little project in a box till later, which is good, because my attention span, while getting better, is still pretty short. That's probably the main reason I'm always switching tasks and getting sidetracked.


Update: 7:30am

I just realized that when I plugged the pro/mini into the usb hub, it was the usb cable from the sensor module, so there was no common ground, and I was right :)  The sensor module was powered by a 12v wall-wart, completely separate from the usb hub.

Update: 12:00pm


Providing a common ground did resolve the serial monitor issues, but the Sensor Module was still unable to connect with the pro/mini i2c slave device. I even hooked the pro/mini directly to the Mega 2560's SDA and SCL pins without the shield, still no go.  I tried another i2c device, no go... so something on the sensor module, likely software since a direct connection didn't work either. After careful examination, I realized there was no Wire.begin(); anywhere in the code to initialize the i2c connection :(  See what happens when you're tired?  Ok, when I'm tired...

It's working now, I don't understand how it worked earlier, unless the code had that line, and I mysteriously deleted it...

Update: 3:45pm

I've got both Dual Level Soil Moisture Sensors in use now, both in large pails. I can't build any more until I get some more parts from the electronics store in Markham and I'm not sure when we'll be there again.  I'll keep an eye on these two for now, but they are looking good so far.

Update: 7:00pm

I'm taking advantage of 2 of the 5 statuslines available at the bottom of the terminal program window. They were being used to display the number of records read, written, saved, etc. Now I'm displaying the uptime of each board, the esp8266 and the mega2560 since both can be rebooted individually. Uptime is just how long since the board was rebooted.



Monday, September 2, 2019

Daily Review Sept 2, 2019

Posted: 2:55am

I just attempted to create a library with all the ESP8266 specific routines from my code, so each module just uses the library. 

After a couple hours, I realized that I really  do not understand how to create a library. Oh I can create a basic library, but make it more complex, I get caught up in where and how I can or can't access methods and variables, should it be static or not,  all that stuff. 

It's time to go back to school :)  I'm going to take a break from the Switch Module, and educate myself regarding arduino libraries - sidetracked again!


Posted: 3:50am

There are tons of tutorials on how to create the most basic library, but not so many covering more complex things like callbacks, which if I can implement, would mean that the XML processor can go into the library, and just call back to the module code with data for it to consume.

So I've been reading, and think I'm ready to give it a shot, but I'm gonna go off for a nap first. I only sleep an hour or two at a time usually, a couple times a day, I love all the time it gives me for working on Grow By Wire!


Update 6:55am

I'm still trying to wrap my head around callbacks from an arduino library.  In fact, I'm not even sure I need to do a callback, I guess I haven't really thought about my options here.

Update: 12:00pm

Turns out there is very little of the xml code that can be shared due to the extreme differences in the data being received. While keeping the code in the main file will include some code that is duplicated in all modules, the amount of customization makes putting it in a library a rather large task, for no real savings in speed or memory usage.  So I was back to upgrading the Switch Module to the V2 format.

Then I got sidetracked AGAIN!  My wifes computer died, and she can''t get to her facebook games! Big Emergency... Turns out it's the monitor, so I hooked her up to the 50" HDTV (I'm using a 42" HDTV) and she's happy again, and my back is killing me, big time!


Update: 6:30pm

I have the shell of the ESP8266 firmware working now, so I copied the code to a text file, saving it as the 8266 Template, so when I create, or upgrade, any Mega/Esp modules, I can cut and paste and be partly done...

Now I'm starting on the 2560 code, and will do the same thing.

Update: 10:40pm

Not much progress, but I had a rest, so I'm ready to go tonight...  As well as coding, I also want to spend a little bit of time going back and revisiting my Dual Level Soil Moisture Sensors. Initially I was quite excited to be able to measure the soil moisture at two different levels within the pot, but after a while, I realized the readings weren't accurate, in fact, I didn't even see a pattern.

Now that I'm using the 17L pails, they are quite deep, I would love to have these sensors working. I have two that I made, they are I2C devices, so they don't take up analog ports, however, each sensor requires an arduino pro/mini as the I2C Slave which reads the sensors on its own analog ports. I'll likely create a separate post for the whole process of testing and fixing...


Sunday, September 1, 2019

Daily Review Sept 1, 2019

Posted: 4:00am

This will be easier for me, I'm going to create a Daily Review post each calendar day and throw most stuff in there. The post will be updated throughout the day as necessary, up until midnoght, when the next days post will be created as long as there is something to post about.

 If something warrants its own topic, I'll create a new post for it. So on that note, here is today's review!

I initially set out to find the cause of the constant esp8266 reboots when I uploaded the same code that worked before I re-installed windows. There are a couple posts about this, the first one is here, and the solution here. Once I went back to the version of the MySql library that I was using before the re-install, I was back in business.  

Then I got sidetracked again... 

I just tried something to see if I could fix the issue I was having that caused me to add a database connection close after each access. That issue is first discussed here.  I seemed to be able to read from the database up to the point where it reads the sensor config to send to the 2560 to scan the sensor, and then record the results to the database. It was able to read it the first time through, but not the second, There are about 30 or so fields it needs to read from the database, and this is too many for the connector to read, I believe he recommends 23 maximum. I broke it into two parts, the first one reads 14 fields, then a second read of 18 fields. The limit is really the total size of the data row returned, so I thought perhaps I was somehow over that limit? I do think it's a memory problem because of the weird data that was showing up. Well, tonight's I further broke it down to 4 database reads to load the sensor configuration data, it worked out to 7, 8, 9, and 10 fields read for each of the 4 calls. I tested that, it worked fine, so I removed the database close, and it failed again, so that isn't the solution. I put the database close back in. and I'll just continue on and come back to this another time.



So watch for updates to this post throughout the day...


Update: 6:40am

Getting sidetracked is a habit now...  I ended up transplanting a couple clones into bigger pots, then I needed to change the connectors on some soil moisture sensors so I could use them on the new plants, and have some ready.  They are all converted to the new connector now except for 4 that I still have to find :)

Then for some reason, I decided to wash a bunch of dirty pots and trays, and now I'm considering either having a nap, or tearing out the Veg area and replacing the panda plastic, and then set it up again.  I think perhaps breakfast then a nap. We have to go out for a few hours around noon, so I better do the nap thing...


Update: 5:00pm

Had some family commitments today, so not a lot done, but I'm back at my desk, and ready to rock!

After my nap this morning, and before we left, I did manage to tear down, and rebuild my cloning chamber.  It sits in a shelving unit, and was on the second shelf, meaning I had to bend way over to work with it, and with my bad back, that was no good,.  I decided I wanted to move it to the third shelf where it was a lot more comfortable to work.  Because the shelving unit basically IS my cloning chamber, it took me a couple hours. You can read about that project here.


I've been running the new version of the Sensor Module Firmware on module 2 for the past week or so now, and it looks pretty good, so I think I may update it with the latest compile (after my issues with rebooting etc) now that everything seems stable again.

Then, I think I'll work on the Switch Sensor while it'snot being used to control the Clone Mister (I've got no cuttings to root at the moment, I'm at my legal limit for number of plants)

I want to change it so it works like the new sensor modules, the ESP8266 will be the main controller, and the Mega 2560 just the connection to the hardware (switches, lights, etc)
Then, I can hook up the high current relay (contactor) I bought, and try controlling the 600W HPS in the Flower Tent again. Last time I used a cheap relay, and it ended up fusing the contacts together due to arcing when it turns off, and the lights stayed on until I discovered it.

So, let me get some dinner, and get things sorted, and I'll be back for a night of excitement!


Update: 7:50pm

I installed the latest Sensor Module firmware onto module 2, and so far no issues. If it stays that way, I'll install it on module 1, and we'll be live with it.

I started reviewing the Switch Module code and it does something none of the other modules do, it caches various data that it requests, so if it cannot reach the database, it will read the cached copy and use it. I remember doing this, but now that I look at it, it was all a waste. I've changed the way I read data from the database, rather than clear out the old data and then do the read, I wait until I have read the whole record into temporary variables, then I copy it to the actual variables after clearing them out.

We're on our own...

When I started this project, I created a thread on the 420 Magazine forum, and it grew, and grew. It has had over 12,000 views and nearly 1000 posts.  I decided to create this blog because anything I post on their site is under their control, and I was worried I would lose all that information.

In order to keep those who had been following my posts up to date, I was cutting and pasting tidbits from my blog posts with links back to the full post, and in my signature line on the forum, there was a link to the blog.

Just minutes ago I received a private message from one of the admins telling me they had removed my signature, and that I was not to post links to the blog, somehow I was competing with their paid sponsors, or trying to recruit their members?

Well, no more posts for that site, and I apologize to anyone who was following me there, but it is their site, and their rules...

Now, this is my site, and my rules, so let's get going!   

I'm going to open comment posting to guests, so you can post without having to expose your identity publicly here, hopefully you will do so and let me know you are here.

Thanks for your support!