August 2019 Update

While this site is still somewhat new, and still under construction, this project has been underway since roughly August 2018. Please excuse any unfinished pages, I'm still catching up!

New!
View the Feature Sheet by clicking HERE!

New to the site?

Project information can be found on the "Grow By Wire Site Links" panel. The right side of the screen consists of my blog posts, a window into my thought process as I work on the project.

Wednesday, August 7, 2019

Back in the deep end....


It's been a while since I've completely torn apart a section of code and rewritten it. I find myself doing so again tonight, rewriting the way the Sensor Module retrieves the configuration for it's sensors.

A bit of background...


The Sensor Module consists of a Wemos Mega 2560 which has an EPS8266 right on it's motherboard as part of the Mega. The two are set to communicate over a hardware serial ports connection, so the transfer of data is very reliable, and quite fast. (115200 baud)
The ESP8266, being a WiFi module, is responsible for maintaining a WiFi network connection, and using the MySql Database Connector library, it handles all database reads and writes.

When the 2560 requests Sensor Configuration data, it sends a request to the 8266 over the hardware serial port, and the 8266 will generate the proper SQL query, and fetch the data from the MySql server, sending each record back to the 2560, pretty simple...

Now for the problem, and solution...

From the MySql Connector Documentation....


A single Sensor Configuration record is too large and cannot be read, so it has to be processed in two parts. I accomplished this by reading half the data for all the records, and sending it back to the 2560, then doing the same query, but for the remaining fields instead, and sending those to the 2560 one at a time. This means the 2560 has to combine the data for the two halves before it can use it.

I've been noticing that on occasion, something goes wrong and the second half of the configuration data has not been combined into the records. It's either all or nothing, so I know the second half is failing completely, I just don't know where, and what causes this.


I could spend forever trying to recreate it on demand, and then figure out whats going on, and fix it, but I don't really like sending the data back in two parts like that, I just see too many points of possible failure.


So, I'm going to read the first half of a sensor record, then read the second half, combine them on the 8266, and send a complete record to the 2560, then move on to the next record, and repeat until the end of matching records.

I will also take advantage of storing as much data in program memory, although memory hasn't been an issue yet. I'm not quite sure why MySql connector can't use more memory, but this is what the compiler shows me:

Program size: 340,404 bytes (used 33% of a 1,044,464 byte maximum)
Minimum Memory Usage: 37588 bytes (46% of a 81920 byte maximum)


I have no problem so far with holding data in memory, even large arrays, so I am confident this will work.

Off I go, diving in....


6am Update

Well, it's been a couple hours, and I'm getting there...  still coding, just now got it to compile, so I haven't tested anything yet. I'm probably half way through...

I like doing this kind of major hack because I get to review the existing code very closely. I have seen, on very rare occasions, where a sensor will actually be saved with the data belonging to the next sensor (in sensorId order). There will only be one, not all of them, but it's on my bug list as a very low priority because it was so rare... Well, I discovered WHERE it could happen, but not necessarily WHY. 

The 2560 retrieves the sensor Configuration records from the 8266, loops though them and scans each sensor. The data is then sent back to the 8266 so it can be inserted into the database. I had a separate data type (struct) for Data than I was using for Configuration. This meant that the 2560 had to coordinate the two records, and send them back as one. Again, this is just another point of potential failure, and this is likely WHERE this happened.

I've now combined the two, since it was being combined anyways...

Another issue, the 2560 is sending the entire record back to the 8266, when in fact, only the couple fields in the data record (which has now been combined with the configuration record) will actually change, and only those fields need to be sent back. I will be modifying the code so that it only sends the fields that could have their values changed. I suppose if I wanted to take it to the extreme, I could send only those fields where the data did change, but that would require keeping two copies (so I could compare) or keeping a "dirty" flag for each field...  Just easier to send them all, there's probably less than a dozen fields that could change, and the extra overhead would probably result in no time savings...

Back to it...

7:30am Update

I needed this, a good chuckle this morning....  I have no idea who wrote the intellisense, but they had a sense of humor... it would have been included with vsMicro, the addon that allows me to write my arduino code using Visual Studio.



Anyhow, it's going really good, the code worked on the second try... The first try failed because when I was doing the part where the 2560 read the new fields in the configuration, I was cutting and pasting, so I did all the numeric fields first, and was going to come back and do the strings (char arrays) later...  Well, I didn't...  So I fixed that, and it ran, read the configuration properly, and read the sensors.

Since this is my development unit, I didn't have any actual sensors on it, and discovered another bug...  If a sensor fails the "power test" it is skipped when it comes to sending the data to the 8266. The "power test" is when first, the sensor is read without power applied, then power is applied and it is read again. If those readings are the same, then there is no sensor....  Well, it fluctuates a bit, so I compare them, and if the difference is less than 5, it flags it as a fail.

I'm going to try something...  I've learned a LOT since I wrote this part of the code, it was one of the first things it ever did, read a sensor and save the data....

This sounds like a job for a pull-down resistor doesn't it?  That way, when there is no sensor, or a wire breaks, or comes unplugged, the pull-down should cause it to read 0 rather than a value from a floating pin.

Quick note, I can probably use the internal PULL_UP on each analog pin, won't even need to make any hardware changes....

Ok, lots to do...   gotta have breakfast, maybe even a nap, then right back at it...

11:45pm Update

It's been a long day, but a productive one. The new code is now running live on the two Sensor Modules in the grow room. 

Using the internal pullup resistors worked like a charm. By checking both the power test result, and the sensor reading, we can determine 100% that a sensor is not connected.  I also fixed a few other bits that looked like they could cause problems even though I hadn't see them. 


No comments:

Post a Comment