kck

Aaron, The usage notes on github has the install instructions: https://github.com/kevinkahn/softconsole/blob/master/docs/useagenotes.md You should be able to run the piprep script and it will ask you a series of questions and then create directories and populate the console into them. It should also, if you answer yes to the question about autostarting on boot update the rc.local file to automatically start the console. You will need to build and config.txt file to have the console build screens otherwise you will just see a red exit screen when the console starts and immediately exits not having found a configuration. I suggest looking at the config.txt and stuff in the cfglib that are in the example config files at github. If this isn't working for you can you describe a bit more what isn't working and perhaps I can provided more help based on that. You do need to spec the correct display in on of the question in the script since the Adafruit folks have a 3.5 resistive and 2.8 capacitive and resistive models and they vary some in their drivers. Happy to help some more as needed. Kevin

Emile et al., I just pushed a 2.2 release as currentrelease. It improves the handling of an unexpectedly closed websocket stream by catching the situation and reopening a new stream rather than restarting the console entirely. This makes recovery from the close much faster (probably a second or two rather than 15-20 that it was). I still cannot diagnose the conditions that cause the close to happen. It simply manifests as either a close command in the stream or a 0 byte message - at least that is what the Pi library appears to be seeing. It does seem to be related to network noise since it happens more on some networks and systems than others. The release also adds a bit more diagnostic info gathering that may point at the underlying issue in the future. Kevin

OK - Thanks. That would explain why after I did a Query from the admin console things cleared up. So I am assuming that the ISY keeps the last state it had from the device but is saying that it's most recent attempt to communicate with that device failed. Good to know - just couldn't find it documented any place I looked.

When I start the event stream using websockets in the initial dump of device state info there is one record that arrives as: '<?xml version="1.0"?><Event seqnum="195" sid="uuid:70"><control>ERR</control><action>0</action><node>3B 92 51 1</node><eventInfo></eventInfo></Event>' I can't find any documentation of what control=ERR might mean. The device in question is a 6 button KPL dimmer and right after this record I see the Ramp, OnLevel, and 5 Button status events. The device appears to be functioning fine and from the admin console I don't see anything unusual about it. So why is this ERR event being sent and how do I diagnose what is going on? Thanks, Kevin

Nice job on the case improvements! I have heard/seen reference to those metal inserts but haven't used them. I assume you design a hole just a bit smaller than the insert and then heat/push the insert in using a soldering iron? Very professional look. May need to add some of those to my toolbox. Also like the improved bezel. I had one screen that came off as well but I just got some scotch permanent 2 sided tape and fixed it that way. But the light leakage, while not really an issue, does bother me. The case was my first real 3d print design effort and I did it because I couldn't find a case with a top/bezel for that screen. Don't know why I never added that lip once I had gained some more experience. Think I'll put that on the list to fix. As to the software. I verified that the empty scenes were not a problem of any sort. I just noticed them and didn't know whether they indicated some other issue - they don't. I'll keep adding as much diagnostic logging as I can to the comms in the program to see if I can get us a handle on what is going on. I have seen a log like yours once recently on one of my systems. What is bothering me is the fact that there is no actual error message coming up from the websocket library. If I set up a test where I forcibly break the connection using tcpkill (which spoofs a reset into the link) I do get an error message. I'd expect that if the comms just dropped out from a timeout of some sort I'd see some similar exception and there is certainly code to catch and report such an error. So while I think it is a low level comm issue I'm not fully convinced it isn't really a bug of mine. Just as a general observation, I do occasionally see retries on my boxes in the log. There are 2 places where they appear. First is in initially downloading the ISY device/scene info. There I have a significant time delay and a lot of retries since the most common reason for that failure is actually when the house takes a full power outage. The Pi simply gets back a lot faster (<20 seconds) whereas the ISY seems to take a minute or two to get back. The other place I see occasional comm errors is when changing screens out of an idle state where I do a rest query to the ISY for the state of the devices on the now active screen. That uses to have a long timeout which would make it look like the console had locked up when really it was just timing out. I greatly reduced the timeout there since the ISY really should either respond virtually immediately or you should just retry. So all this does suggest that the Pi comms can get into a funny state at times. Still my consoles all hold the websocket stream up for days to weeks without failures. By the way - my configuration has the consoles on WiFi and the ISY on Ethernet off the house router. If you get a failure where you docapture the debug info (will be in log.txt or if you set LogLevel to 0 in the Console.log file) I'd be interested in at least knowing what the last events on the stream were. I'll push a small update later today that has a bit more error capture in it and keep looking at my systems to see if I find anything in them. Kevin P.S. One last random item on a weird comm issue I discovered on a Pi recently. I had one Pi2 with a small plugin Edimax WiFi dongle that would not consistently keep its name active on my network using mdns which is a multicast protocol. I had pulled out my little remaining hair trying to understand why that system, running an identical build to other systems that used USB dongles or Pi3s was the only problem case. I swapped out the dongle for a different one and the problem was cured! There is no "rational" reason why this should be an issue but I found a very obscure reference to a broadcast issue with some wifi chips which seems to be what I had run into. So if you happen to have a different dongle around you might try swapping and see if it makes any difference even though I don't think broadcast/multicast should be involved here.

All - I just pushed a V2.11 that fixes an error I introduced last release that broke the installation script. Sorry for any inconvenience if you ran into it trying to do an install. Symptom would have been a failure with a message about not finding config module.

Looking at the logs I noticed 2 things: 1. You happen to have a screen named "Special". Apparently the way the config file parser I use acts it equates that with a debug flag "Special" that I use internally and sets the flag. Not an issue for the debug flag (although that is why you see some debug output of the form "Special -> . . ." in your logs) but it might cause an issue with the screen definition (although it looks ok further down in the log). So I think this isn't a real issue though an interesting discovery. In the version I'm about to push to fix the prep script error I've renamed that flag to DebugSpecial so it should make it harder for folks to bump into it. 2. You have a number of scenes with no members. One of those (Auto DR) is expected since it has something to do with energy monitoring. The others though seem odd. Are they X10 related? Or do you actually define some scenes in the ISY with nothing in them (guess that is possible). Again don't think this would cause any issues related to the crash/restarts but they at least look suspicious. Looking at the logs it seems like the websocket connection is just closing after a while at the network level and the console gets an error the next time it tries to access it. The message that I get at the application level is that the connection is already closed. I would expect that if I was getting some bad event in the stream I'd log that which I don't seem to be doing which makes me think it is some sort of network level event. Is there anything odd about the comms between your Pi and the ISY box? I noticed that in some of the logs the console had difficulty doing a read from the ISY during startup. There are some messages about "ISY not responding" with some retries until it get through and does the successful node read. Those are definitely odd. The code does retries there because after a full power hit the Pis reboot a lot faster than the ISY. So if you were restarting the ISY I would expect to see some retries waiting for it to come up but not otherwise. In any case perhaps a detailed dump of the websocket stream will shed some light if you get the chance to do that. Kevin

Arg! OK - install failure was my stupidity plus not testing. I added a log message a while back in the githubutils module for normal operation that required importing config and since it isn't there during install . . . I guess I hadn't done a clean install since that change. I'll fix that when I'm home. As to homerelease, no rush to fix but easy to. There should be a file "homesystem" in your pi home directory. Delete it and do a download for the stable version from the maintenance screen. If you are running autoversion it actually looks at the current "versioninfo" in the consolestable directory to know what to check on so you probably want to also edit that file and replace "homerelease" with "currentrelease" if that is still there after the maintenance screen manual update (don't think it will be but not sure as I type this). To possibly help me with the real issue of the rebooting/loss of websocket connection if you get a chance could you set the debug flags 'DaemonCtl' and 'DaemonStream' both on for a crash. That might give me info to see what is actually coming in the websocket stream prior to it closing itself. As I said - I'll be home tomorrow and look at this more easily there. Thanks for your patience, Kevin

One thing I did notice that could be an issue at some point. I see that you installed watchdog. That probably shouldn't be in the script (or at least isn't necessary) since it is a program to reboot the Pi if it loses the network (also handles things for my 3d printer if attached and can power cycle a modem and router in another configuration). In any case, if you installed it you'd need to update where it finds your local router or it may think it has lost local network connectivity and do a reboot. It's a yaml file in the watchdog directory. Otherwise just run without it is probably safer (it has pretty convoluted logic that seems to handle my cases but hasn't been really stress tested). The restarts though seem to be coming because you lose websocket connectivity to the ISY for some reason. That will require a more detailed look on my part to figure out why that might be happening. By the way, I see you set to run "homerelease". Should work but be warned that is a flag for me to run knowingly in my house and could have stuff that is flakey in general. That's why the script has a prompt that suggests not setting that. At the moment homerelease = currentrelease so it should make no difference. Just a heads up.

I'm away at the moment but will look at this Thursday when I'm back home. I can't imagine what the catch.txt thing is - I don't recall any file by that name and I'm pretty sure I don't create/use such a file. All those 2 python files let me do is to avoid doing a manual install and setup. They just mimic what happens in the automatic download and install that you can access from within the program. The codeerror is definitely my issue in that is generally means I got to a branch or other place in the code that I should never be able to get to or that I don't know how to recover from. It is probably some status info from the ISY that I'm mishandling. Kevin

I had one of my KPLs fail this way. Then the triac dies it can die off or die on it seems. Assuming that this worked at some point in the past I suspect your dimmer is dead. ETA: rereading your post I'm not sure if the load can be turned off from elsewhere. If so then my diagnosis is wrong since the load control is working. But if you're saying that the switch reports the load off but the load stays on in all cases then the load control is dead but the logical control in the switch is working. (BTW - if the dimmer is dead and the switch out of warrantee then I have repurposed such switches to spots where no local control is used but a KPL is desired to run other scenes.)

For folks with Pis and the like there are a variety of other similar relay boards in configurations from 1 relay to 16 and where the relays operate on 5v (good for Pi GPIOs) as well as 12v available from ebay sellers and also on Banggood.com (a good Chinese source that if nothing else is worth browsing since they sell everything from relays to lingerie ). I use a couple of the 5v relays on one of my Pis to allow for automated reboot/power cycling of my router and/or cable modem if I appear to lose network connectivity for a while. The 5v relays can be driven really easily from one of the Pi GPIO pins. I also use one of these to remotely power on/off my 3d printer so that it turns off and fails off if I forget it or there is a power outage.

I just pushed a 2.1 version that makes the fix from the above discussion part of the release. It mainly moves the program internals to pure websockets from using one of the other ISY libraries to make it simpler and a bit faster. It also continues the effort to make the program more robust against configuration errors and network problems. At least at my house it seems to survive a pretty wide range of power outages, network storms (don't ask - I haven't figured those out but have something to do with my router), and other communications issues. Full comments in the release announcement on GitHub - note that there is one shell level command you need to give before upgrading to this release to install a library. I have added a way to handle things like this automatically in the future. The release can be downloaded and installed from the console maintenance panel or via the autoversion updating mechanism if you enable that. Kevin

Great! and sorry for the difficulties. To the questions: 1. I don't even really remember to be honest. I think I might have had a system with no VNC and just grabbed one. TightVNC seemed to work fine for me so I never gave it another thought. Just FYI - I personally do relocate its port. I wanted to be able to access a RPi from outside through my firewall and discovered that I was getting clobbered with attacks when it opened the normal port. Nothing go through since I use good passwords but it was annoying and put traffic on my home net. Moving it to some other random port doesn't actually increase security since an attacker can try all my ports find the open one and then probe to see what service is on it. But most attackers aren't that motivated so in practice it pretty much eliminated attack attempts through my router. Of course, if you don't need to access from outside that is much better. 2. OpenVPN is in the script only because it simplifies my install in a case or two. I run a VPN to allow myself into the home network and also to establish a permanent connection between this house and my vacation place. I should probably delete it from the script (along with things like the ddclient) since it doesn't apply to anyone else but . . . I'll be interested in seeing what happens with the X10 devices. I'm going to guess that they will cause a crash but with any luck I can figure out why and fix the issue if so. If you happen upon questions drop me a line and if you have suggestions do the same. The program structure is pretty clean these days so changes are often not too hard. Kevin

I have pushed a version that I think handles the thermostat issue (since mine doesn't return a blank I'm a bit limited on testing the fix but it was pretty simple). It should now show your tstat as 'Idle' if it isn't running the heat/AC. This version is marked as 'currentbeta' so you should be able to try it by loading your current version, going to Maintenance/Select Version then choose "Download Beta" which should show the button pushed for a couple of seconds while the download happens, then "Use Beta Release". Then if you Return, Exit Restart, Restart Console, and confirm the console should reboot in the beta version. You'll see that listed at the start of the new console log where it tells you what version is running. Let me know if this fixes things. One small caveat - I put this fix into my current development version which has moved to a simpler event subscription model with the ISY. I don't think this will cause any other issues since I have been running this code base on my systems for a couple of weeks with very solid performance. Still - it is a slightly different code base than 2.0. I was planning to push this as 2.1 soon anyway so I think you'll be good. Kevin P.S. Once I do push it as 2.1 you should use the maintenance screen to reselect stable release and download the then stable release so you're back on the normal stuff. P.P.S. Arg - almost forgot. The new code base uses a Python library that the V2 install script didn't load. You will need to run "sudo pip install websocket-client" before you upgrade to the beta. If you don't you'll see a crash which says it can't find the websocket module.

Yep - your tstat reports blank apparently when it isn't heating or cooling. Mine reports a code that maps to "off". My display from the ISY console shows the heat/cool state as "Off" where yours is blank. My tstat looks to have a slightly later firmware (v.0D) so that may be it. My system also has the 3 nodes for each thermostat as you saw when you reinstalled. No problems though. I'll add code to expect a non-numeric code in the ISY response and map it to something sensible. Most of the bugs I find from folks are things like this where there have been subtle changes in the way things get reported when the ISY is queried or in the event stream. I'll try to get a version posted as a "currentbeta" for you to try in a couple of hours. Not sure why the flags would crash - mine works fine and that's just a debugging convenience so not critical. Still it should work so if you get a chance at some point could you grab the log.txt from a crash and send it my way. In general, that log.txt file captures the crash info from python and usually will pinpoint the exact line of code that blew up and approximately why. I'd like to find a way to capture that to the Console.log file but haven't figured out how to do that (it's kind of a meta problem since you want to recapture control of the python system after it has errored out but I think I may be able to do it at some point). Just FYI that LogLevel = sets a detail level for log entries. It defaults to 3, 0 gets you everything including any debug that is turned on. Those flags let you turn on specific types of debug info. They can also be set in the config file by using <flagname> = True so that they are on from the start of the console. Sorry for the rocky start on this - hopefully this will get you up and running. Nevertheless, do continue to ping me with problems or suggestions. Kevin

That's weird - that says that one of the properties that the ISY is sending back for your tstat has a non-integer value which I didn't think should happen. I expect an integer value and get something else. I forgot to have you add a "LogLevel = 0" line to the front of your config.txt which I think is why the debug line that should have been output to the log if you turned on the Main debug flag didn't come out. Just as a quick test: if I hit the ISY from my browser with a "http://192.168.1.15/rest/nodes/22%2018%20A3%201"where the IP address is the ISY and the 22 18 A3 1 is the tstat address I get a response whose "properties" look like: <properties> <property id="CLIFS" value="8" formatted="Auto" uom="n/a"/> <property id="CLIHCS" value="1" formatted="Heat On" uom="n/a"/> <property id="CLIHUM" value="24" formatted="24.00" uom="%"/> <property id="CLIMD" value="3" formatted="Auto" uom="n/a"/> <property id="CLISPC" value="46" formatted="23.00" uom="degrees"/> <property id="CLISPH" value="42" formatted="21.00" uom="degrees"/> <property id="ST" value="135" formatted="67.50" uom="degrees"/> </properties> I am guessing that if you do the same one of those value= items won't be an integer. It would be good to know what it is and what it means. I'd guess that it will be a blank (i.e., " ") which I can check for and turn into a value but I'd like to know what to turn it into. To give you some context: CLIFS is fan mode, CLIHCS is heating/cooling state, HUM is humidity, MD is mode, SPC is cooling setpoint, SPH is heating setpoint, and ST is current temp. the temp values are all 2x the actual (i.e., 46 is 23 C). The HUM is a percent. If you could tell me what your tstat is telling you and what it is actually set to I should be able to deal with whatever it is reporting via the ISY.

The crash has to be happening in the screen display routine for the tstat but there isn't much that happens there - the Python crash message will probably point at it. The screen names you are using are fine and the fact that you get to the screen from the nav button says it has been parsed and loaded ok. Really all the screen display routine does is query the ISY for the tstat info and format it. I even tried switching mine to C from F and that all worked ok since I didn't know if you were running F or C. In addition to telling me what the log.txt file says one other thing you could do is to start the console and before you go to the tstat screen go to maintenance (5 taps), set flags, and set the "Main" flag on. Then return/exit maintenance and go to the tstat. I would expect that in the log.txt file you would see what came back from the ISY and it should be something like: Main -> Active Subscription List will be: Main -> Enter to screen: Thermostat - Main Main -> CLIFS : 8 : Auto Main -> CLIHCS : 1 : Heat On Main -> CLIHUM : 23 : 23.00 Main -> CLIMD : 3 : Auto Main -> CLISPC : 46 : 23.00 Main -> CLISPH : 42 : 21.00 Main -> ST : 40 : 20.00 (This was from a tstat in C mode - numbers would of course be higher for F). If the problem is in the actual comms to the ISY (don't know why that one call would fail given everything else is working though) you won't have gotten to the debug line so won't see this. In any case, I'd guess I can diagnose stuff pretty quickly from that point.

Quick answer on the last question: that's just a header for a log section that lists screens that aren't on one of the screen lists - I should suppress that line if there aren't any actual items that will show below it. I'll take a look at the Tstat issue when I get home later this afternoon. Just so that I understand: when you say you tap the Tstat button I assume you mean the navigation button at the bottom of the previous screen that should take you to the thermostat screen? It then crashes attempting to display the thermostat? (By the way the thermostat screen should look more or less like the display on the actual thermostat with red/blue temp up down arrows and mode and fan control buttons.) Do you happen to have the log.txt file and Console.log from when it crashes. That would probably point pretty clearly at what is happening. Interesting on the X10 stuff. As I said I haven't got any of those or that module so running blind there. I could probably at least try to catch any anomalies from that and try to be more graceful. Do let me know if there is functionality there that I can support that you need and maybe I can figure something more out for you. Kevin

I'm sure that's it. I suspect what is happening is that the node ISY creates for that X10 device has no properties and so I don't know what to do with it and don't add it to the list of nodes I create. Then later in processing some scene I find that address as a member of the scene but then can't find it in the node list. I think there is a background scene that ISY creates that has all the nodes in it - don't remember right now and not at home to check. So it may be that is the scene. Otherwise it is appearing in some scene of yours. If it is the latter and you don't use it any more then getting it out of the scene or just deleting the node should avoid the problem. If it is a node that you use and want to keep then I'll have to figure out in more detail what is going on so that I can avoid tripping on it when I encounter it. Kevin

Sorry for the problem. You are reading the ISY fine. The problem seems to be when I am building the internal graph of ISY nodes by parsing what the ISY sends me (i.e., certainly a bug of some sort). I am traveling at the moment and won't be able to take a detailed look until Wednesday but some starting stuff. It fails while processing some scene that has a node with address FF 0C 07 2. What kind of device is that (I'm guessing it may be a KPL given the subnode 2). Is there anything special about any scene it is a part of? Also, which ISY firmware series are you running v4 or v5? Also, do you have any zwave devices? The last 2 questions are because I run v4 and no zwave right now and I have seen some previous issues where I just didn't know about a quirk of some sort related to those that made my code wrong. Just for your context where it is blowing up is in trying to add nodes to the member list of a scene. For some reason it looks like it isn't finding the node it wants to add (which from a code perspective shouldn't happen so it is definitely some sort of but). So anything you can tell me about the device and scenes it is a part of may help diagnose. Kevin

Michel, Thanks. I do this as a hobby since I'm happily retired after a lifetime in high tech, so no plans to commercialize. I think it would be pretty easy to do if someone wanted to and I'd be supportive of that and happy to assist but I'm not up for being the driver of the effort since I've got a lot of retirement pursuits elsewhere. But since it is a hobby I'm also happy to try to add things that folks might want to it and it is open sourced on GitHub so doesn't really depend on me for ultimate support. Also, I haven't looked closely at your v5 stuff and node servers yet (running v4 at my house) but I think that using the node server structure might allow better communication with the ISY for scene information which right now I have to monitor via a proxy device. So at some point I'll probably try to see if my guess is true - just not wanting to make the house depend on v5 quite yet. (May try getting a v5 unit at some point for our winter place and try a small scale experiment there.) Kevin

For anyone using or thinking of using my softconsole system there is a major new release now up (2.0). This release is a major restructuring of the system's architecture that permits me to now do things like alert screens and procedures. Something I had wanted to have for a long while and that was also requested by others was an ability to grab the screen with a very visible (from a distance) image if some condition arose (e.g., water sensor went off or garage door left open etc.) This is now possible. The system allows you to define an alert condition based a state change of a node or a variable, or periodically. It allow the condition to be delayed some period after the change (e.g., so I can alert if the garage door is open for 10 minutes but ignore normal open/closes). Alert conditions can either cause a screen to appear on the console with an (optionally) flashing message and an (optional) key that is expected to clear the alert condition. Alerts can also be deferred for a predefined period of time if so configured so that you can choose to ignore the condition but still have it reappear if not dealt with. Multiple alert screens defer to one another cyclically. If the condition clears via other means the screen returns to normal in response to the corrected condition. An alert can alternately cause a python procedure to execute quietly. I am currently using this at my house to check periodically for new releases on my "production" units and download/install any new release (I got tired of trying to remember if I had loaded the most recent code on the units). An alert procedure is just a python class/procedure so easy to construct if you have a new one. This release is also much more robust against network failures and things like badly formed answers from weather underground about weather. My experience is that this version pretty much stays up with few exceptions and restarts itself if it does encounter odd errors. The overall structure of the code is a lot more straightforward as well for anyone who might want to play with the source as it is now essentially a single control loop that reacts to a variety of events and dispatches screens, procedures, etc. based on time/events/variables and the like. For you info I typically have 4 units running in my house and my experience with them has been pretty good. As always, if anyone wants to play with this stuff or has bugs, requests, or whatever feel free to pass along. I'd been wanting to work on this version for quite a while but it took the return of Oregon fall rainy weather to justify being indoors enough to get it done. Feedback always appreciated.

FYI - I just pushed a new general release (v1.4) that picks up a bunch of bugs and cleanups including the issue from a while back were the ISY was providing bad results when queried repeatedly for status (this ISY issue should be now fixed in UDI's code in any case). The main change in 1.4 is that Adafruit's PiTFTs don't install drivers correctly using Adafruit's own scripts if you are using a release of Raspbian newer than March. This is significant for folks with RPi3s because releases from March don't support the included RPi Bluetooth so using Adafruit's versions you can have either touch screen or BT but not both. Luckily there is another user on the net (in Australia it appears) who has install procedures that do work with the latest Raspbian releases so there is now a new install script consoleprep2.sh that will basically take a clean install of current Raspbian and get you to a functioning console that will start automatically on boot with minimal user intervention. Basically you have to provide a few passwords and an intial config script to get up and running. I've tested this script with the current September Raspbian release on a Pi3 and while installing everything takes a while it seems to get the job done with little user effort.

There are definitely advantages to being retired - I can attest to that! Glad it will help you with the "boys and their toys" effort. I think I am up to 2 Pi3s, 2 Pi2s, and 2 PiZeros at this point and I keep toying with getting one more just to have one purely for development. I've been working "hardening" the software load on 2 of them so that I can reliably maintain a VPN that will merge my home network with a home network in a winter house we recently bought. I have watchdog code now that reacts to network or other outages by rebooting itself and if that isn't enough rebooting my router and/or my cable modem. See - so many fun projects you can do when you don't have one of those pesky day jobs. So by all means, should you need other assistance or have enhancement ideas drop me a not either in the forum or via private message. Kevin