kck Posted January 6, 2019 Author Posted January 6, 2019 Thanks for the quick feedback. I'll probably push that as the standard release tomorrow.
chrisb5 Posted January 6, 2019 Posted January 6, 2019 Thanks! Using the desktop distro made all the difference. Install ran all the way through and I did finally get a console display... though I think there is an issue communicating with my ISY? I'm currently running v4.7.3 (https) though I'm hoping to make the jump to v5 shortly to experiment with polyglot. The one switch I named in the setup shows up with a big "X" across it on the console and I see ISY related errors in the log file - though I do seem to be able to turn the switch off and on via the touch screen - though I'm not sure how and it's not clear from the display if it's on or off? Anyway the log is showing lot of messages like these: 01-06-19 17:34:07 Sev: 3 ISY: WS stream thread 1 setup 01-06-19 17:34:07 Sev: 4 ISY WS stream 1 closed: None:None 01-06-19 17:34:07 Sev: 4 ISY QH Thread 1 exiting 01-06-19 17:34:08 Sev: 4 ISY Failed Thread Restart 01-06-19 17:34:08 Sev: 3 Starting helper thread (1) for: TouchHandler 01-06-19 17:34:08 Sev: 4 Thread for: ISY is dead 01-06-19 17:34:08 Sev: 3 Restarting helper thread (2) for: ISY 01-06-19 17:34:08 Sev: 5 ISY Unexpected error on WS stream: ISYUnknown 01-06-19 17:34:08 Sev: 3 ISY: WS stream thread 2 setup 01-06-19 17:34:08 Sev: 3 ISY Delaying Hub restart for probable network reset: 90 seconds 01-06-19 17:35:38 Sev: 4 ISY WS stream 2 closed: None:None 01-06-19 17:35:38 Sev: 4 ISY QH Thread 2 exiting 01-06-19 17:35:39 Sev: 4 ISY Failed Thread Restart 01-06-19 17:35:41 Sev: 4 Thread for: ISY is dead 01-06-19 17:35:41 Sev: 3 Restarting helper thread (3) for: ISY 01-06-19 17:35:41 Sev: 5 ISY Unexpected error on WS stream: ISYUnknown 01-06-19 17:35:41 Sev: 3 ISY: WS stream thread 3 setup 01-06-19 17:35:41 Sev: 3 ISY Delaying Hub restart for probable network reset: 90 seconds 01-06-19 17:35:56 Sev: 3 Entering Maintenance Screen: Maintenance 01-06-19 17:36:00 Sev: 3 Entering Log Screen 01-06-19 17:36:36 Sev: 5 Unknown main event <Event(2-KeyDown {'mod': 0, 'scancode': 28, 'key': 13, 'unicode': '\r'})> 01-06-19 17:36:36 Sev: 5 Unknown main event <Event(3-KeyUp {'mod': 0, 'scancode': 28, 'key': 13})> 01-06-19 17:36:39 Sev: 5 Unknown main event <Event(2-KeyDown {'mod': 0, 'scancode': 1, 'key': 27, 'unicode': '\x1b'})> 01-06-19 17:36:39 Sev: 5 Unknown main event <Event(3-KeyUp {'mod': 0, 'scancode': 1, 'key': 27})> 01-06-19 17:36:50 Sev: 3 Entering Maintenance Screen: Maintenance 01-06-19 17:36:50 Sev: 3 Entering Maintenance Screen: Flags0 01-06-19 17:37:04 Sev: 3 Debug flag ISYdbg = True 01-06-19 17:37:11 Sev: 3 Entering Maintenance Screen: Flags1 01-06-19 17:37:11 Sev: 4 ISY WS stream 3 closed: None:None 01-06-19 17:37:11 Sev: 4 ISY QH Thread 3 exiting 01-06-19 17:37:12 Sev: 4 ISY Failed Thread Restart 01-06-19 17:37:14 Sev: 4 Thread for: ISY is dead 01-06-19 17:37:14 Sev: 3 Restarting helper thread (4) for: ISY 01-06-19 17:37:14 Sev: 5 ISY Unexpected error on WS stream: ISYUnknown 01-06-19 17:37:14 Sev: 3 ISY: WS stream thread 4 setup 01-06-19 17:37:14 Sev: 3 ISY Delaying Hub restart for probable network reset: 90 seconds 01-06-19 17:37:15 Sev: 3 Entering Maintenance Screen: Maintenance 01-06-19 17:37:19 Sev: 3 Entering Log Screen 01-06-19 17:37:40 Sev: 5 Unknown main event <Event(2-KeyDown {'mod': 0, 'scancode': 28, 'key': 13, 'unicode': '\r'})> 01-06-19 17:37:41 Sev: 5 Unknown main event <Event(3-KeyUp {'mod': 0, 'scancode': 28, 'key': 13})> 01-06-19 17:37:41 Sev: 5 Unknown main event <Event(2-KeyDown {'mod': 0, 'scancode': 54, 'key': 303, 'unicode': ''})> 01-06-19 17:37:41 Sev: 5 Unknown main event <Event(3-KeyUp {'mod': 0, 'scancode': 54, 'key': 303})> 01-06-19 17:37:45 Sev: 3 Entering Maintenance Screen: Maintenance 01-06-19 17:37:49 Sev: 3 Exiting Maintenance Screen 01-06-19 17:38:45 Sev: 4 ISY WS stream 4 closed: None:None 01-06-19 17:38:45 Sev: 4 ISY QH Thread 4 exiting 01-06-19 17:38:45 Sev: 4 ISY Failed Thread Restart 01-06-19 17:38:47 Sev: 4 Thread for: ISY is dead 01-06-19 17:38:47 Sev: 3 Restarting helper thread (5) for: ISY 01-06-19 17:38:48 Sev: 5 ISY Unexpected error on WS stream: ISYUnknown 01-06-19 17:38:48 Sev: 3 ISY: WS stream thread 5 setup 01-06-19 17:38:48 Sev: 3 ISY Delaying Hub restart for probable network reset: 90 seconds 01-06-19 17:40:18 Sev: 4 ISY WS stream 5 closed: None:None 01-06-19 17:40:18 Sev: 4 ISY QH Thread 5 exiting 01-06-19 17:40:19 Sev: 4 ISY Failed Thread Restart 01-06-19 17:40:21 Sev: 4 Thread for: ISY is dead
kck Posted January 6, 2019 Author Posted January 6, 2019 (edited) You are definitely not getting a websocket stream from the ISY which is why you get the x-d out screen and no feedback. The touch is sending the on/off blindly which is why you do see the light control. I don't know what those KeyDown/Up events are unless you have somehow chosen to use a strange touchscreen or not spec'd the touchscreen in the setup correctly. If the screen an Adafruit screen or the 7" Foundation design? And did you spec that when installing? Because those events look like they might come from some other touchhandler code. I don't think this would be related to the ISY comm problem but not sure. The comm problem is also odd. Basically, the websocket code is getting an error processing the stream that I don't expect to see - i.e., don't know what it is. The fact that you can turn things on/off suggests that your authorization is ok otherwise I'd ask if your password etc. is correct. My guess is that there is something odd about your ISY url as you spec it. You might try just using http and the actual ip address to start rather than https and see if that helps. You an event set the IP address in the config file to just the IP address with no "http" in front and see if that works. Here's an example of what is in my config file. Quote [ISY] type = ISY address = http://192.168.1.15 user = *** password = *** If this still has problems send me the start of the Console.log file where all the basic info is documented like the screentype and isy address stuff. Edited January 6, 2019 by kck
chrisb5 Posted January 6, 2019 Posted January 6, 2019 The display is the official Raspberry PI 3 7" Touchscreen Display. I selected the 7pi (or whatever that option was) in the list of display choices in the config screen. The screen is showing ok; I selected the default setup and I can click through and select the date/time screen, maintenance screen, etc. My ISY isn't setup to allow non https access - and my address = line in auth.cfg specifies https:// plus the IP address and :PORT (I use a non standard port). At the moment I do have a Dell USB keyboard hooked up to the Pi as well; not sure if that was the source of the keydown/up events?
chrisb5 Posted January 6, 2019 Posted January 6, 2019 Also having trouble grasping the config structures; I'd like a home display with temperature, smoke sensor, and some leak sensor statuses... Then buttons perhaps buttons for each room (roughly grouped by floor if there's enough screen space) that each take me to a secondary screen specific to that room...? I see how you define ISY devices in each cfg file but not clear how you define buttons that take you to another screen? Thanks though! Getting excited about the app!
kck Posted January 7, 2019 Author Posted January 7, 2019 The PORT could be the issue. I haven't tried to support a non-standard specified port although I would have thought it would work. I use a standard Python library for websockets that just takes the url and it looks like the url would just have the port as part of it and so should work. I just tried changing my config files to specify the IP:80 since it uses http on the standard port and that seemed to work just fine as the console came up as expected. I'm pretty sure that others here are using https on its standard port. I suppose that there could be a bug in the library that somehow doesn't like the specified port if it isn't standard but that seems unlikely. If it were possible you might try temporarily using the standard port and dropping the :portnum part of the url to see if that fixes it. If so, I'd have to figure out why the existing library and usage doesn't work. But really, I'm guessing that something else is going on with the comms. Clearly, the direct comms work since you can turn the light on but the websocket stream isn't connecting correctly. There is a debug file that may have a clue in it in your Console directory in a hidden directory names ".HistoryBuffer" If you send me that file (the last one if there are more than one) it might shed some light on the nature of the websocket error. The Dell keyboard probably was the source of the other events so I'd not worry about those. Those key events just got forwarded to the console that didn't know what they were. As to the config structure - the usage document is pretty complete and there are a lot of examples in the example configs directory. However, there is no support for defining a button screen that takes you to another screen like you are describing. The nav keys at the bottom are the console's way of moving between screens. The screen tree model you are describing is a good one - just not the one the console implements. Sorry on that one. (Perhaps a future enhancement.) Unfortunately I have a bit of a time crunch to be useful to you getting your comm issue sorted quickly because I am about to be traveling out of the country for 3 weeks. I'd normally try to figure out the issue and get you a fix right away but I have only tomorrow and still a list of things to do unrelated to the console. So I won't be able to do any significant debugging/fixing right now although if the HistoryBuffer file makes it clear what is going on I may be able to do something quickly. Kevin
chrisb5 Posted January 7, 2019 Posted January 7, 2019 Here's the latest file in that (.HistoryBuffer) directory. Looks like it's creating one of these files every minute... Thanks again! 113-01-06-19 20%3A12%3A29
kck Posted January 7, 2019 Author Posted January 7, 2019 Well the reason the websocket stream isn't opening is given as SSL: CERTIFICATE_VERIFY_FAILED. On the other hand the requests from the request interface seemed to succeed if things get set up and you can turn the light on/off. Unfortunately I'm not particularly up-to-speed on how the SSL should work so I don't know at this point what that means other than it is the response that comes back as an error on the attempt to open the stream. BTW - the reason that it creates one a minute or so is that the console expects that websocket errors generally clear and so waits a bit and then retries to open the stream. You're seeing the history log for every failure.
chrisb5 Posted January 7, 2019 Posted January 7, 2019 It is a self signed certificate so unless the client trusts it I would expect verification to fail I guess.Is there a library name I can research to see about adding it to the trust store that library uses?Sent from my SM-G935V using Tapatalk
kck Posted January 7, 2019 Author Posted January 7, 2019 OK - think I have a workaround for now. It appears that the library I use doesn't pick up the OS certificates by default. I nervous that without a bit more careful testing than I have time for I won't get things correct in getting it to pick them up. However, it looks easy to tell the library to not worry about the cert. So I just pushed a version that does that. Since the console only operates on your internal network this seems a reasonable stopgap. If you grab the new stable version I just pushed to GitHub it should do this. If you want a quicker test then in the file isyeventmonitor.py in the consolestable directory the 4th from bottom line (which is a run_forever call) should be changed to: ws.run_forever(ping_timeout=999,sslopt={"cert_reqs": ssl.CERT_NONE}) and you will need to add an "import ssl" line to the top of the file. The library in question is websocket-client with the code at https://github.com/websocket-client/websocket-client/blob/master/websocket/_http.py The longer term fix appears to be to set an environment variable ( WEBSOCKET_CLIENT_CA_BUNDLE) to point at the OS cert bundle directory and get your self signed cert in there. If you are into Python feel free to explore! If you want to try that you should get rid of the sslopt that I added in the release file and described above. Let me know if the workaround works around. Kevin
Screw Loose Dan Posted January 7, 2019 Posted January 7, 2019 You guys talking about SSL certificate errors, reminded me we had some of this conversation before. I did get it to work (with a trusted "trusted" cert), but ultimately found the softconsole was not very useful. On 1/20/2018 at 1:14 PM, Screw Loose Dan said: Just to follow up, in case anyone else tries it...I got everything working with SSL/HTTPS, but found it extremely slow and cumbersome. It is so slow that I went back to using HTTP. I did some digging around on the forum and the limitation is the ISY hardware. It's documented (see here and here for example) that it takes 4+ seconds to establish an SSL connection (TLS handshake) for rest calls, this is because the amount of computation it takes when using a certificate with 2048 bit encryption. Those using smaller keys might not have (as much) an issue but industry standards now suggest 2048 bits as a minimum. Let me know you you make out.
chrisb5 Posted January 7, 2019 Posted January 7, 2019 Your code fix above did the trick!! I have a functional button on the display now... Will need to learn the config files now so I can have some real fun BTW personally I prefer your (temporary) solution of ignoring certificate issues with SSL certs... I can't imagine many people going to the expense of paying for a commercially signed/trusted server certificate for their home automation controller... I'm sure almost everyone would opt for a self-signed certificate - and the steps to add that self-signed cert to a local trust store on the Pi could be done - but not sure if the effort is worth it. I've had to script such code in bash and perl but never python so not sure I'd be much help there either but if you choose to go that route I will assist in any way I can. *I do use https on my ISY and have for years; true initial connections are slower but it hasn't caused me any problems. And since I manage my own firewall and lots of other servers (and use the Agave app to control my stuff from my phone) having https and decent passwords on my ISY made opening a proxy up in my firewall for access from the outside (for Agave) worthwhile. -Chris B
kck Posted January 7, 2019 Author Posted January 7, 2019 Great - thanks for letting me know. Sorry it was a bit of a hassle to get initially working. I try to test everything I can but some stuff I don't run and so depend on testing blindly with some folks like Dan who do. Given your comment I won't worry about figuring out the alternative approach. Don't think it would add any real security inside most homes anyway. Kevin
kck Posted January 7, 2019 Author Posted January 7, 2019 So I pushed 3.2.1 as the current release. Mostly minor bug fixes and better dealing with V5 ISY Node Server devices that I don't otherwise handle to avoid lots of log entries.
Screw Loose Dan Posted January 7, 2019 Posted January 7, 2019 I just downloaded and started using the latest release. So far no issues. A thought for a feature request...I recently purchased a Google Home Hub display. It amazes me with it's screen dimming. It has an ambient light sensor and gets extremely dim in a dark room, but is still readable. And, obviously it gets brighter as needed. As I type all that out, it doesn't come across as nearly astounding as it actually is, it's really quite remarkable. Google did there homework. One of the thing I've struggled with on the softconsole is finding a sweet spot for the dim settings., especially for the bedroom unit. Having an ambient light sensor would be awesome, but I realize that might open a whole other can of worms. Instead, perhaps a time based feature for dimming? So in addition to DimLevel/BrightLevel, also a DimLevelLate/BrightLevelLate as well as two variables to set the time to use the additional Dim/Bright levels (LateStart/LateEnd). I know, easy for me to suggest, probably not as easy to implement. ? If you have any suggestions on what to look at, I could perhaps poke around. Anyway, you mentioned you would be travelling out of country for a few weeks. Enjoy your trip! Safe travels.
kck Posted January 7, 2019 Author Posted January 7, 2019 (edited) Dan, The light sensor would be nice - of course it would require some hardware added to the Pi. Have to think about that as it would be a bit more than some folks would want to tackle. However, the time based change you can do now pretty much. All the screen general variables are in the store "System" so in particular the DimLevel is referenced as System:DimLevel. So I have a couple of alerts defined in my configs that look like: [Alerts] [[AdjustScreenDim]] Type = Periodic At = 11:00pm Invoke = AssignVar.Assign Parameter = System:DimLevel = 5 [[AdjustScreenDimUp]] Type = Periodic At = 6:00am Invoke = AssignVar.Assign Parameter = System:DimLevel = 50 These do the obvious things. The only issue with them is that since they execute at a specific times so the initial condition after console start will persist until the first time one executes. If you are running MQTT then you can even do the adjustments via that if you publish the right stuff. For example, using mosquito you could run the command: mosquitto_pub -t consoles/rpi-pgaw1/set -m '{"name":"System:DimLevel","value":50}' To change the brightness of that node right now. This is handy if you want to quickly experiment with dim levels - just reissue it changing the value. You could even send this to all your consoles with a single command if you replaced "rpi-pgaw1" in the above with "all". So it wouldn't be too hard to do more sophisticated changes by running scripts from cron or the like. Other system variables are also changeable if that's useful. Do a StoresDump from the debug flags screen and you'll get a file in the Console directory with a full dump of what the stores hold and in principle you could change any of them. The caveat is that things that impact screen colors and the like may have been already used to build templates and so not have any effect, although I think background colors would change. I know key colors wouldn't generally change. Figuring out how to make all those actually take effect dynamically is actually on the longer term todo list so if there are any items that you think would be useful to make sure are actually changeable let me know. Does this handle your immediate desire? Kevin P.S. Going to Antarctica to see the Penguins! Edited January 7, 2019 by kck 1
Screw Loose Dan Posted January 7, 2019 Posted January 7, 2019 That gives me a lot of ideas and thoughts on what to start playing with!! My initial thought is to connect a an ambient light sensor to a GPIO pin on the RPi that is running Polyglot. There is a Polyglot GPIO node server already and I *think* they expose an MQTT service as well...hmmm. Lots to play with. Thanks!!
chrisb5 Posted January 7, 2019 Posted January 7, 2019 Some quick newbie-to-the-console questions: Never dealt with the weather services before and I understand some are going away/unreliable; what do you recommend for a quick/easy free service so I can get a basic weather summary on my console? I'd like to show status of my (Insteon) smoke and CO sensor on the/a screen. They obviously don't warrant a click to turn on/off; I would just like a button or some labelled indicator showing a good/bad or red/green status. Doable? Same request for some Insteon leak sensors. I have several AEOTEC (ZWave) multisensors that report temp/humidity/light & UV level/motion/etc and would like to show at least the temperature (+humidity) of some of these sensors on some screen(s)... How might I go about that? Thanks - and enjoy the penguins!
chrisb5 Posted January 7, 2019 Posted January 7, 2019 Re: the ambient light/console dimming questions; you might consider some of the AEOTEC multisensors if you have any ZWave capability. I have several of them and the ISY (I have the ZWave module on my ISY) discovers them natively and reports (in addition to temp/humidity/motion) the "luminance" level and UV light levels. The sensors can be battery or USB powered and work well on the ISY as triggers for events/programs/etc. They're just under US$50 last I checked. I use them for motion sensing around the house and keeping track of temperatures in several rooms; and even have some rules that know not to turn on devices when it's "already lit" (based on the luminance levels).
kck Posted January 7, 2019 Author Posted January 7, 2019 The console only support APIXU as a weather service. It used to support WeatherUnderground but they went away/pay only. APIXU has free signup for limited but very sufficient calls/day. The VARKEY key type on a keyscreen is the answer for this. It can be used either for display only or for setting variables. Basically it allows you to change the appearance of the key based on an ISY variable value (e.g. red if 5, orange if 1-4, green if 0). It's documented in the usage notes. Again the VARKEY so long as you can get an ISY variable to hold the value. Alternative is to just reference the variable value and show it on a weather or time/temp screen using the store reference syntax. I do that to display the local temperature on my deck along with the general weather I get from an APIXU feed. There are examples of this in the example config files, see e.g., timetempPDX.cfg.
Screw Loose Dan Posted January 7, 2019 Posted January 7, 2019 Kevin - In the install scripts you set up: update-alternatives --install /usr/bin/python python /usr/bin/python3.5 2 update-alternatives --install /usr/bin/python python /usr/bin/python2.7 1 And you mention: Quote To undo this run 'sudo update-alternatives --config python' to select desired alternative I'm trying to run a Polyglot node server Presence-Poly, which installs (via apt) an application "python-bluez". This application won't install/run without python2.7 set as the desired alternative. Will that cause an issue for the softconsole?
kck Posted January 7, 2019 Author Posted January 7, 2019 You could go to the systemctl service file and edit it for the console to use python3 instead of python. That should explicitly run python3 even if you switch the alternative that python maps to. The Python folks keep saying that python 2 gets deprecated at the end of the year but given the number of programs that still seem to require it I'm beginning to doubt that.
chrisb5 Posted January 7, 2019 Posted January 7, 2019 Sorry; but I am not figuring out the weather format config files for a simple APIXU feed? I'd like to change my HomeScreen to a TimeTemp screen and I tried copying one of your examples that looked as close/simple as possible... but the console won't start (keeps failing with an error saying: Sev: 5 Error accessing ScreenParams:store['store']KeyError('store',) -- and then reboots over and over. Most of the samples utilize MQTT which I'm not using... In my main config.txt I added the following: [APIXU] type = WeatherProvider apikey = <myAPIkey> [SpringfieldVA] type = APIXU location = 'Springfield, VA' refresh = 59 And then the screen config file I tried was the following: [Springfield] type= TimeTemp label = Springfield, VA location = 'Springfield, VA' TimeFormat = "%-l:%M:%S%P %a %b %-d", ConditionFields = Sky,Temp, WindDir, WindMPH ConditionFormat = "{d[0]} {d[1]:.0f}","{d[2]} @ {d[3]}" FcstLayout = 2ColHoriz FcstIcon = True CondIcon = True ForecastDays = 10 SkipDays = 0 ForecastFields = Day,Sky,High,Low,WindDir,WindSpd ForecastFormat = "{d[0]} {d[1]}","{d[2]}/{d[3]} {d[4]}@{d[5]}" ClockSize = 35 LocationSize = 0 CondSize = 25, 25,15 FcstSize = 20, 15 CmdKeyCol = blue Which was a blatant copy of one of the sample configs with just the label/location changed. I added the Springfield screen name to the list of screens and set it as the HomeScreenName in the main config.txt and things went badly... ?
Screw Loose Dan Posted January 7, 2019 Posted January 7, 2019 15 minutes ago, kck said: You could go to the systemctl service file and edit it for the console to use python3 instead of python. That should explicitly run python3 even if you switch the alternative that python maps to. The Python folks keep saying that python 2 gets deprecated at the end of the year but given the number of programs that still seem to require it I'm beginning to doubt that. A quick attempt to try that and it failed (ImportError: No module named configobj). I'm guessing you are calling a subprocess. I'm guessing I'd be chasing that for a while... No big deal, I'd rather not spend time trying to figure out how to make softconsole work while running with an older version of python. I'll see what I can do on the other program. And, Pi zero's are cheap... Thanks! Enjoy your trip!
Screw Loose Dan Posted January 7, 2019 Posted January 7, 2019 Chris - I missed your questions. My setup, this information is in auth.cfg (along with the ISY info): [APIXU] type = WeatherProvider apikey = <myAPIkey> Then in a file named weathsources-drc.cfg (name it whatever, just include it on config.txt): [CV] type = APIXU location = 19426 And them my TimeTemp: [TimeTemp] type= TimeTemp label = 'Collegeville', location = CV TimeFormat = "%-l:%M %p %a %b %-d", ConditionFields = Sky,Temp,Humidity, ConditionFormat = "{d[0]} {d[1]:.0f}° {d[2]}%", ForecastDays = 4 FcstLayout = 2ColHoriz ForecastFields = Day,Sky,High,Low ForecastFormat = "{d[0]}","{d[1]}","{d[2]:.0f}°/{d[3]:.0f}°" ClockSize = 60 LocationSize = 0 CondSize = 50, FcstSize = 25, CmdKeyCol = blue BackgroundColor = black CharColor = lightgray Don't know if that's all best practice, but it works for me.
Recommended Posts