Soft Keypadlinc-like Console from Raspberry Pi

[kck]

That's odd - aiohttp is some Python library that the home assistant access module just uses (that module actually  came from the HA folks - I just sucked it in to my download when they decided to stop supporting it directly) and it seems present on all my systems.  I will do a clean build/install at my end to see if something has changed to make it not a part of the standard install.  In the meantime, you might try doing a "sudo pip install aiohttp" to see if you can get it installed manually.  Basically what you are seeing is a Python "compilation" error (it doesn't really compile) where it fails to find a needed library.

By the way, VNC should run fine in normal landscape mode since it doesn't access the desktop on the pi screen.  There is a secondary VNC port that gets set up if you explicitly choose a port at that chosen port -1 that does overlay the pi screen and would be in portrait but you'd have to go out of your way to get to that one and generally I wouldn't/don't use it except for some wierd debug situations.  I do all my normal access into the consoles for maintenance etc. from a PC running VNC viewer from RealVNC and that works great.

[lhmathys]

I found it!

I was receiving a compiler error during the homeassistant install where one of the gcc headers was missing (ffi.h).  I manually installed libffi-dev and libssl-dev and got it to work.

 

Time to start tinkering with it!

[kck]

Great!

By the way - it sounds like you are using HA (only or with ISY?).  Besides me you will probably be the heaviest (only?) user of HA with the console so please do ping me as needed since the code has had a much more restricted use environment with HA than with ISY and likely has issues.  Also, suggestions for improvements regarding HA use with it happily accepted.  I've been doing a bunch of cleanup to the code, much of it related to HA and specifically updating my fairly old version of HA to the near current (.90.2 since I had issues with .91 and zwave).  I'd been sitting on that since I didn't think it impacted anyone else, but I'll push it to github later today and I'd suggest updating to that once I do since it will provide a cleaner baseline and fewer spurious HA log messages.

[lhmathys]

Sorry for the late response - yard work today.  ?

 

I'm not actually using HA, it was just the component that was giving me fits in the installation script.

[lhmathys]

I have another silly question for you...

I can't seem to locate an example of how to integrate the ISY Climate module into the display.  Is there a way to query the ISY994i for weather local information?  If so, could you give a quick example?

[kck]

No silly questions!  Unfortunately, though, I don't use or even have the ISY Climate module so can't really help you on this one.  I use APIXU for my weather (you can get a free low use code from them) and there are examples of its use in the sample config files - basically all the weather data manifests as store variables that can be referenced in screens, alerts, etc.  If the ISY climate module has the ability to set values into ISY variables that would be a way to make them available on the console - don't know if it can do that.  My guess would be no in V4 ISY and possibly yes in V5 since my sense is that the newer firmware has been trying to smooth the differences across variables and device states (e.g., I think you can actually get the temperature as a value from a thermostat in V5 whereas in V4 you cannot, at least in any easy manner).  If you do see something that is missing that I could add (realizing that I don't actually have the module) let me know and I'll take a look.

By the way, still planning an imminent release of an update but when you pointed out the installation of homeassistant in the install scripts, I thought I'd see if I could remove that.  Turns out it was an artifact of how HA used to allow access but that with a small amount of work I can get rid of it for the future since HA changed their stuff a bit.

[kck]

I just pushed a new version as the currentrelease labeled as V3.3.  It makes the screen title stuff discussed above part of the standard release and it pulls Dan's alert also discussed above as part of an install.  Here is the full release description:  (as always, I updated the usage notes to include documentation improvements for all this).

V3.3 is a significant internal cleanup as well as adding a few features. It adds an ability to specify a title for a screen via the keywords - see the usage notes for details. If reduces or downgrades unneeded reporting in the log by changing the severity of many messages from Warning to Info or Detail. This should reduce log clutter. It improves some of the term shortening for weather based on experience with the APIXU weather service. It updates Home Assistant support to be run quietly with versions up to 0.90 by handling some of the newer HA domains that get reported. It also no longer installs any homeassistant libraries as part of the initial install which simplifies/shortens the install process for those who don't use HA.

It now installs Dan's light sensor alert - see his github for details at https://github.com/ScrewLooseDan/softconsole_sensor_alert. Autoversion downloads are now done asyncronously to the console interactions so that the console isn't locked up while the new version is set up - this is mainly an issue on pi0 systems due to their speed making the setup a somewhat long process. Explicit maintenance screen updates are still syncronous.

The release also adds a new alert that some might want to use. I have discovered that there must be some sort of a memory leak in Raspbian desktop since after very long up-times (40+ days) the Raspian X desktop that is created by VNC sessions winds up with huge memory use and eventually will cause the console to lock up. This alert (lxpanelreset) is meant to be run periodically (probably once every few weeks would be enough but I just run it once per day since it is harmless) to do a reset on the lxpanel which frees the memory. You could also do this with crontab but this saved me from needing to edit all my system crontabs.

This version makes a major cleanup to how internal timer work for the console. This should be invisible to all users as it was done to make my maintenance and development easier but if you notice something strange regarding timing do let me know since regressions are always a risk with reworks. It adds a flag on the maintenance debug flag screen to force out a message to clear error indicators of all networked systems if you are using the multi-console error notice stuff. It adds a bit of a failsafe mechanism to catch dead consoles that get hung in a strange state. This doesn't happen often, but I had one system that seemed prone to it (possibly failing hardware) so I added a dead man timer process to restart a console in such a state. Again - this shouldn't be visible to you but . . . If anyone is running the "homerelease" version, there are some bug tracking/logging things enabled in that version - probably a good reason not to run my homerelease version as it is really meant for my houses.

[kck]

I just pushed V3.3.1 to GitHub.  Release note:

V3.3.1 is a minor update primarily to handle a discovery I made that any code that touches the file system on a Pi in any way can cause 
delays of many (up to 10) seconds.  Normal logging activity from the console could therefore cause what seemed to be stalls of the interface.  
This release moves file system interactions to asynchronous threads that can suffer the delays without being noticed.  The release also now 
handles Home Assistant nodes that the console knows about but that are not on-line in HA at the time the console starts.  This can happen on 
systems that cohost both a console and home assistant at power up time because HA can be slower to initialize that the console.  Other minor 
code cleanups and small bug fixes as always.

The discovery about the Pi file system delays may be of interest to other users of Pi systems since it is an OS issue potentially impacting any system that has multiple processes doing a substantial amount of file I/O so I documented this issue (and another Pi item of possible interest related to WiFi dropouts) in a separate thread at:

https://forum.universal-devices.com/topic/26380-couple-of-obscure-but-noteworthy-raspbian-pi-items/

 

[BamBamF16]

Kevin,  I finally got my system put together and softconsole loaded.  However, once the console starts it is no longer responsive to touch.  Any thoughts?  My system is an Rpi0W with a Pimoroni Hyperpixel 4" touchscreen.  I used the pimoroni drivers and touch works fine on the desktop.  However, once softconsole starts, nothing.

One more item.  I can start the console via VNC no problem (that is when touch goes away), but when I try via SSH I get:

pi@SoftConsoleTest:~/consolestable $ sudo python -u console.py
06-26-19 08:08:32 Console ( 1041) starting in /home/pi/consolestable
Traceback (most recent call last):
  File "console.py", line 143, in <module>
    utilities.InitializeEnvironment()
  File "/home/pi/consolestable/utilities.py", line 98, in InitializeEnvironment
    hw.initOS(scrntyp)
  File "/home/pi/consolestable/hw.py", line 131, in initOS
    pygame.display.init()
pygame.error: Unable to open /dev/fb1
06-26-19 08:08:33 Exiting with history trace (0)
mvtops exception: [Errno 2] No such file or directory: '/Current/'
1053(1561554514.1216013): Logger loop ended

One final thing, the console will not auto-start.  During the installation, when the reboot occured after updates the installconsole script did not run after reboot.  I had to manually start the console install. 

 

Thanks.

Jeff

Edited June 26, 2019 by bambamf16

[kck]

That screen is not one that I have support for in the scripts or have every tried so I'm not sure what is going on.  What sort of screen did you tell the install scripts you had?  It looks like /dev/fb1 isn't defined which suggests to me that you chose one of the standard Adafruit screens or other.  If you look in /dev you probably only have a fb0 defined.  You might try installing as a pi7 and see how that works.  It looks like the screen you have is capacitive which should at least make getting this to work seem possible (resistive screens are more of a bear to get right).  Take a look at /sys/class/input and you should see a directory named event*, i.e., event0, event1, etc.  First how many such directories are there?  Hopefully only one, event0.  For each such directory look at its subdirectory named "device" (e.g.,/sys/class/input/event0/device) and cat the file "name" and tell me what you get.  I'm hoping you see "FT5406 memory based driver".  If so the pi7 setting should be close and I may just need to possibly adjust some scaling.  If touch works with that driver then let me know if touched seem to be in the right places and if the screen looks normal.  You might also tell me what the console log says the screen dimensions are if the console starts with these settings.  We can go from there to try to get that screen working for you.  (Since that I can see of the documentation for that screen shows that it is 800x480 I think the pi7 settings may work out of the box.)

 

Kevin

[BamBamF16]

Ok, here goes what answers I can give you:

1.  I setup the screen as "custom"

2.  I have an fbo in /dev

3. The screen "name" is "Goodix Capactivie Touchscreen"

4.  How do I reinstall for a different screen ?  (I can't find installconsole.sh anymore)

5.  The touch works outside of the Console and was calibrated (the latest stretch change to libinput which this screen didn't care for initially).

 

Console.log

[kck]

OK - Custom definitely won't work.  I think pi7 should - you can check by editing the file in your home directory ".Screentype" to be "pi7" instead of custom.  That should solve the output side of things (i.e., the can't find fb1 stuff).  Editing that will avoid rerunning the install script.  (By the way, the installconsole.sh is left in a directory of your home directory called consoleinstallleftovers).  The issue wilh touch is that outside of the console you are running via an Xserver.  The console does direct control of the screen and touch and so isn't using the correct driver info given the different name.  If you are willing to try editing one source file we can see if the simple fix will sort this out.  If not, I'll generate a beta release for you - just slower and you need to download a bunch to go that route.  You will want to go into the consolestable directory and find the file touchhandler.py.  Line 122 of that file defines TOUCHSCREEN_EVDEV_NAME.  You want to edit this so that the string is 'Goodix Capactivie Touchscreen' instead of the 'FT5406 memory based driver'.  I hope that this will let the console fine the touch events for your screen.  If it does, then the only other issue may be the relative orientation of the touch event coordinates versus the actual screen.  I.e., does where you touch correspond to the button drawn under the touch.  It is possible that it might be flipped horizontally or vertically and I'd need to account for that.  If you can try this and let me know what you see that would be easiest.  If you're uncomfortable with editing that py file I can generate a full beta with that change instead.  Once I know what works I can incorporate a permanent change in a next release.

Kevin

[garybixler]

Hi

I thought I would give this soft console a try.  Copied the script (pisetup.sh) onto the boot drive after copying the image to

an SD card for the PI. After starting the script I got this error on line 4.

/boot/pisetup.sh: line 4: syntax error near unexpected token `$'\r''
'boot/pisetup.sh: line 4: `function Get_yn()

I'm not a programmer so I doubt I could go in and alter anything that my be a problem. 

Model 3 B+ PI. Latest raspian.

Thanks for any suggestions. 

Gary

 

 

 

 

[kck]

This one is easy.  You somehow managed to get the file translated to DOS format from its original Unix format.  Windows and Unix use different end of line conventions and it looks like you managed to get the pisetup.sh into Windows format so the shell on Linux couldn't make sense of it.  Things you might have done to cause that are editing it and writing it back (even with no real changes) on Windows using a Windows editor.  You should just copy the version from the GitHub copy and then copy that to the SD card.  The easiest fix for this using the copy of the file you have is to edit it on your pi system.  Run

sudo nano pisetup.sh

on the Pi.  This opens the simple editor on Linux.  You will see a highlighted line that says how many lines it read and then says '(Converted from DOS format)'

Now type ctrl-O and it should say File Name to Write [DOS Format]: pisetup.sh

Now type Alt-M and the 'DOS Format' part should go away.

Now it return to actually write the file back.  

This should fix the line terminators.  The difference in line terminator conventions is one of the more annoying differences between Windows and Unix and something to keep in the back of your mind if you ever see really weird messages from a program that is reading a text file in Linux.  Always worth a check to see if somehow the file got written with the wrong terminators.

Kevin

 

 

 

[larryllix]

I just ran into this running my Python3 genealogy software on a Win 10 system. When the .csv files were opened on my Rpi they caused problems and I realised they had the double EoL terminator from the Windows python3.

A simple utility like "dos2linux" downloaded into the RPi  can fix these problems for files ported over from Windows. I had to make allowances for blank lines in my software to avoid confusing my code in the future.

[garybixler]

Hi Kevin

Thanks for the help. It all went as you said. Had to sudo nano  /boot/pisetup.sh though. 

When I now do the bash /boot/pisetup.sh   pi just goes to a new line. Nothing executes. 

Unless I am missing something.

File now reads:   Read 323 lines (Converted from Mac format) 

Thinks Gary

[BamBamF16]

4 hours ago, kck said:

OK - Custom definitely won't work.  I think pi7 should - you can check by editing the file in your home directory ".Screentype" to be "pi7" instead of custom.  That should solve the output side of things (i.e., the can't find fb1 stuff).  Editing that will avoid rerunning the install script.  (By the way, the installconsole.sh is left in a directory of your home directory called consoleinstallleftovers).  The issue wilh touch is that outside of the console you are running via an Xserver.  The console does direct control of the screen and touch and so isn't using the correct driver info given the different name.  If you are willing to try editing one source file we can see if the simple fix will sort this out.  If not, I'll generate a beta release for you - just slower and you need to download a bunch to go that route.  You will want to go into the consolestable directory and find the file touchhandler.py.  Line 122 of that file defines TOUCHSCREEN_EVDEV_NAME.  You want to edit this so that the string is 'Goodix Capactivie Touchscreen' instead of the 'FT5406 memory based driver'.  I hope that this will let the console fine the touch events for your screen.  If it does, then the only other issue may be the relative orientation of the touch event coordinates versus the actual screen.  I.e., does where you touch correspond to the button drawn under the touch.  It is possible that it might be flipped horizontally or vertically and I'd need to account for that.  If you can try this and let me know what you see that would be easiest.  If you're uncomfortable with editing that py file I can generate a full beta with that change instead.  Once I know what works I can incorporate a permanent change in a next release.

Kevin

Kevin,

I edited the files as directed above (I think I got them correct).  I added them below for review.  However, I get a catastrophic failure and console doesn't launch.  See error:

06-26-19 20:07:00 Console ( 16716) starting in /home/pi/consolestable
OSError: [Errno 22] Invalid argument

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "console.py", line 143, in <module>
    utilities.InitializeEnvironment()
  File "/home/pi/consolestable/utilities.py", line 98, in InitializeEnvironment
    hw.initOS(scrntyp)
  File "/home/pi/consolestable/hw.py", line 139, in initOS
    GoBright(100)
  File "/home/pi/consolestable/hw.py", line 56, in GoBright
    GoDimPi7(level)
  File "/home/pi/consolestable/hw.py", line 148, in GoDimPi7
    f.write(str(level * 255 // 100))
OSError: [Errno 22] Invalid argument
06-26-19 20:07:00 Exiting with history trace (0)
mvtops exception: [Errno 2] No such file or directory: '/Current/'

16756(1561597621.5156422): Logger loop ended

 

Also, what are all of the files in consolestable labeled with date/time.  There are about a thousand of them now and it is making it difficult to "ls" in that directory.

 

Thanks for all your help!

Jeff

touchhandler.py .Screentype

[kck]

@garybixler:  If you do a nano on pisetup it should just say Read 323 lines.  If it also says "Converted from Mac format" then the file is still in a bad format (Mac version of bad versus Windows version of bad).  When you do the write of the file from nano it should only show the name of the file to write - Alt-M when you try to write it out switches between Mac and Unix flavors.  I just reproduced what you see on my system - if the file is in Mac format the pi just returns.  Not sure why as I'm not a Mac person but you definitely have the file format wrong.

[BamBamF16]

@kck,

Switched back to "custom" from "pi7" and console ran (left "Goodix Capacitive Touchscreen" in touchhandler.py).  Still no touch.

Jeff

Edited June 27, 2019 by bambamf16

[garybixler]

Kevin

Thanks a lot for your help. Got it running now. 

Gary

 

[kck]

@bambamf16:  That would have been my next suggestion.  The error you got with pi7 is because apparently your display does dimming in some other way.  But with custom the code will point the display at fb1 which you don't have.  Are you actually seeing a console screen appear?

With the screentype as custom right now you simply aren't getting a touchhandler started in the console which is why you aren't seeing touch working.  Of course, it is possible that even if the handler was getting started it might not work.  Right now the code for the screen and the touch handler are a bit intertwined for historical reasons.  I can probably clean that up but it may take me a day or two since that is old code I haven't looked at in a long time.

If you want to try one more patch at your end, you can go to the file hw.py (not hwa.py) and all the way at the bottom you will find the lines:

def GoDimPi7(level):
   with open('/sys/devices/platform/rpi_backlight/backlight/rpi_backlight/brightness', 'w') as f:
      f.write(str(level * 255 // 100))

You could try deleting the last two of these and replacing them with a single line that has the same indentation as the "with"  that just reads "pass"  Make sure that the indentation is achieved with a tab character not spaces (Python is picky).  This should make brightness setting a no-op and get rid of the error when you set the screen to pi7 and should get you the right touch handler.

 

I have no idea what you are seeing with files that have date/time names.  Do they have any interesting content?  They might be some log files that are getting in the wrong place as a result of some exception reports happening very early in initialization before the structures to get them where they belong are set up.

Kevin

[BamBamF16]

@kck Short of pulling the plug, how do I stop they running console via VNC/SSH?

[kck]

If touch was working you'd just go to maintenance and tell it to shut down.  if it was started as a service automatically, then sudo systemctl stop softconsole should work.  If you started it some other way then do a ps agx | fgrep console and issue a sudo kill -SIGINT on the console processes that you see there.

[BamBamF16]

@kck

Made the change to 'pass'. This led to the following error:

06-26-19 21:36:18 Console ( 2130) starting in /home/pi/consolestable
Traceback (most recent call last):
  File "console.py", line 143, in <module>
    utilities.InitializeEnvironment()
  File "/home/pi/consolestable/utilities.py", line 105, in InitializeEnvironment
    ts = Touchscreen()
  File "/home/pi/consolestable/touchhandler.py", line 137, in __init__
    self._f_device = io.open(self._touch_device(), 'rb', self.EVENT_SIZE)
  File "/home/pi/consolestable/touchhandler.py", line 270, in _touch_device
    raise RuntimeError('Unable to locate touchscreen device')
RuntimeError: Unable to locate touchscreen device
06-26-19 21:36:18 Exiting with history trace (0)
mvtops exception: [Errno 2] No such file or directory: '/Current/'
2192(1561602979.6633866): Logger loop ended

 

I am attaching one of the log files.  looks a lot like the "Exiting with history trace" line.

0-06-25-19 22:38:02

[BamBamF16]

@kck

Ok, I'm an idiot.  Stupid capitalization error.  On line 122 I didn't capitalize the S in TouchScreen.  Now I have a functioning console.  However, the right call it 1/4 to 1/5 of the screen isn't responsive to touch.  Looks like the calibration is a slightly off.  Also, left of the test button still activates the button, even though it shouldn't.  Possibly the touchscreen is "shifted" left relative to the image by actually about 1/8 of the width of the screen.

Jeff

Edited June 27, 2019 by bambamf16
More details