GDavis01 Posted January 26, 2018 Posted January 26, 2018 First an overview on what I have done, and then I will explain how I went about doing each step. Overview This write-up addresses the following devices on my home network: - 27” iMac (mid-2011) running High Sierra - ISY 994i/IR Pro, v.5.0.10 with Portal integration - Mobilinc on an iPhone6S connected via the ISY Portal - Raspberry pi 3 with 32Gb microSD card - Amazon Echo - iPad (version 1) running iRule - Back in 2012, I was intrigued by a thread entitled “ISY has a Voice” by oceanman93, wherein he explained how to set things up using network resources on the ISY to have the Mac speak certain things using the “say” command. (This is an archived topic and can be found at https://forum.universal-devices.com/topic/7717-isy-has-a-voice/). At that time, I got the process working and was using it for things like having our iMac (which is in our kitchen) announce that “Garage door number 1 is still open and should be closed soon”. (note that I originally only had one garage door connected to the ISY via an IOLinc sensor, but I now have all three connected to the ISY using Nodelink with an EnvisaLink card connected to my DSC alarm system. I am now using the alarm sensors to determine the status of the 3 garage doors). In 2016, I read another thread “How to get iRule to communicate with your ISY” started by chrishick (in 2014). This was of interest to me because iRule provides the ability to create a nice interface on an IOS device (in my case, an old iPad) that can communicate with various devices including the ISY and the Raspberry pi. My level of interest was quite high because my wife had been saying for quite some time “Don’t go and die on me, because I won’t know how to turn anything on in this house!” So, I thought that I had found something that would make it easy for someone to control most things in the house. The beauty of iRule is that not only can it control devices, but it also provides the feedback, from the ISY, so that you can see whether a light is on or not, or even what level it is dimmed to. The downside of iRule is that the learning curve is fairly steep particularly when you want to get sophisticated! I have attached a few pictures of my iRule screens from the iPad. Another downside of iRule is that there is a lot of work involved in moving the same setup to a new device with a different resolution (i.e. newer iPad!). Unfortunately, there is not an easy migration process! Another concern I have is that iRule was purchased by another company (Kramer Electronics) about a year ago and it appears that there has been no development since then on the application! With iRule, I currently control all the lights in the house (those with Insteon devices), the blinds (using Somfy Universal Interface RTS Interface II (Somfy Part No. 1810872) and the Autelis Somfy RTS interface… as recommended by stusviews), the sprinkler system, the AC system (Venstar 2491T thermostat), and a Russound C3 system for distributing our outdoor sound. You will probably also notice from the pictures that I have integrated a lot of weather data into the iRule interface! Note that I do not plan to go into any detail, in this thread, on how I set up iRule. If there is interest, I would recommend that you read through chrishick’s thread which explains how to get started with this. The weather data integration into my iRule set up is what lead me down another path and that is what I plan to explain here. As I mentioned above, I wanted a nice user-friendly interface on an old iPad which sits on our kitchen counter and I thought that integrating weather data would be a nice touch. Reading another thread ‘wunderground.com’, I saw a posting by btreinders, wherein he explained how to use a visual basic script to extract weather data from wunderground.com. Several things came out of this… Never having used VB before, I jumped in and figured out how to use his VB scripts, however, after I got them working I concluded that I didn’t like the idea of having to have my PC running 24/7 in order to keep updating the weather! I also felt that I was being constrained with the information I could ultimately pass to iRule, from the ISY, because I could only populate numerical values into ISY variables! How would I get textual conditions to iRule such as ‘Sunny’ or ‘Cloudy’? My solution was to rewrite the script in Python on my Raspberry pi. I had also never used Python before so it was another learning curve! (note that, although I have very limited programming experience, I have found that I can generally cobble things together. As a result, though I can guarantee that the code could be more efficient, it runs quickly and consistently, so I have no need to change it!) As I embarked on my Python journey, I found that iRule seems to works best when reading XML files, like reading XML data from the ISY. However, I also found that iRule is also quite finicky on how it reads XML files, and I quickly realized that simply pulling XML data from wunderground did not work well with iRule! So, I came to the conclusion that the best strategy, using Python, would be to extract the data I wanted from wunderground and create my own XML file, which I would store on the Rpi. In this way, I could create an XML file in the specific format that I needed for iRule. The result is that I now have a Python script, on the Rpi, that runs and recreates an XML file, every 10 minutes, with the latest weather data. Based on the timing I set up in iRule, the iPad application accesses the Rpi (via a Flask server) and reads the XML file. The XML file I create incudes all the weather data that I need, including: - local current temperature from nearby Private Weather Stations (PWS’) (my script allows for data from 3 PWS’s and averages the data) - local airport data for other information, including feels like, windspeed, current conditions description (e.g. ‘snow’), as well as forecast data for the next 3 days (since I don’t live near an airport, I select 2 airports, which I am sort of in the middle of, and average the data. For my cottage weather I simply select the local town report and put it in twice so the averaging works) - textual descriptions (e.g.“Snow and gusty winds this evening will taper off and give way to cloudy skies late. Low -14C. Winds NNW at 30 to 50 km/h. Chance of snow 90%. Snowfall of 2-3cm); and - variables created by the script to use in iRule… (e.g. “if 'Sunny' in xx: weather_icon = 2”… and then in iRule I use weather_icon to choose a ‘Sunny’ icon to display on the iPad) The Python script also passes some of the data I accumulate to the ISY populating integer and state variables. After finishing this iRule setup, I then had the thought that I should try to get the iMac to actually ‘say’ the weather! However, I had to figure out a way to trigger the iMac to start talking, given that I had the data in my XML file, on the Rpi. What I chose to do was to set up an HTTPServer on the Rpi. I modified some code which larryllix provided (in some thread ??) so that the HTTPServer sits and waits for a call from an ISY network resource. On the ISY side, I created a network resource that simply passes a parameter (e.g. “9”) to the Rpi. When the Rpi receives the call, via the HTTPServer, and sees the parameter “9”, it executes another Python script. This other Python script extracts the data I want to “say” from the XML file sitting on the Rpi; then it massages the data so that the iMac can read the data (e.g. it changes “SSE” to “South, SouthEast”) in normal English; and finally sends code to the iMac in the following order: - It gets a shell script, sitting on the iMac, to run. This shell script saves the current sound volume level, of the iMac, to a file on the iMac and then changes the sound level to 18 (which is the level I like). The shell script then pauses iTunes, if it is playing (I do this because we often use the iMac to play music in the kitchen); - It gets a second shell script on the iMac, passing it a parameter. That shell script has the code to have the iMac ‘say’ the parameter. The first parameter passed is "Hey it's Alex". - Then, in succession, it reruns the same shell script passing the following phrases: o “Good morning, today is… (e.g. Monday, January 15, 2018)” (or afternoon or evening) o “The current time is …” o “The current temperature in ‘my home town’ is …Celsius” o “Which feels like …” o Then it ‘says’ the forecast for the day or evening, depending on the time, (e.g. “Snow and gusty winds this evening will taper off and give way to cloudy skies late. Low -14 Celsius. Winds NorthNorthWest at 30 to 50 km/h. Chance of snow 90%. Snowfall of 2 to 3cm) o “Thank you for listening. Have a great day” - Finally, the Python script gets a third shell script, on the iMac, to reset the sound volume to where it was originally (by reading from the saved file) and unpauses iTunes, if it was originally playing. (note that if the iMac had the volume muted originally, it will put it back to that state) My last step in this process was to add a button to the iRule application, on the iPad, that runs an ISY program that invokes a network resource that starts the above process causing the iMac to “say” the weather. As a slight variation on the above, with the acquisition of an Amazon Echo, I have been having fun getting our iMac to talk to Alexa! Using a network resource that sends a different parameter to the HTTPServer, I have it say things like “Alexa, turn on the coffee machine”. (my coffee machine is connected to the ISY using an On/Off Outdoor module 2634-222). Obviously, I could easily tell Alexa myself… but where’s the fun in that? (On Christmas day, I had another ISY program which, using another network resource, would have the iMac tell Alexa to play another Christmas song {Alexa,,,+Syng,+another+Christmas+song}… every once in a while I would use Mobilinc to run the program! Cheesy? Yes, but the kids seemed to like it!!) Now I will try to go through the various steps and explain how I did each step… Please remember that I am not a programmer. I am sure that some of the coding is not the most efficient, however it works and can provide a good basis to work from. Some of my explanations go into a lot of detail, but I thought that more was better than less! (Note that I will be attaching numerous files, in the next few posts, that I make reference to. All of the files have been changed to .txt so that I could attach them! However, I will also attach a text file which explains how to edit each file’s file type and where to copy the files to -- Attached Files_text.txt) 2
GDavis01 Posted January 26, 2018 Author Posted January 26, 2018 Setting up the iMac to “Say” As I mentioned in the Overview, oceanman93’s “ISY has a Voice” thread is where I got the basis for setting this up. His set up was done in 2012 and was based on Mac OSX Lion. I have recently upgraded the iMac to High Sierra and have things working. My information will be based on my current setup but I’m not really sure if there is a big difference! (I also had it running on Yosemite) Step 1 - Start by opening up a terminal session on the mac and type in: say I have something to say o You should hear the Mac say “I have something to say” - If you want to hear it with a different voice type in say -v? to get a list of available voices - Try this say -v Whisper I have something to say Step 2 - Create a shell script on the mac with the say command. I recommend to those not familiar with Linux that you create the shell script within the terminal. When I first started on this I created the script outside the terminal and then copied it in… unfortunately I didn’t realize that I had hidden characters that were messing things up! - Use these commands: o cd /Library/WebServer/CGI-Executables -- brings you to the directory for executable scripts o sudo nano saysomething -- opens up a new file called saysomething in editing mode § type in say -v Vicki "i have something to say" § Then Ctrl-X to exit; ‘Y’ to save the changes; and finally hit enter to accept the file name you initially entered (saysomething) o Now make the file executable with this command: sudo chmod 0755 saysomething o Next, test to see if the file works locally… again in the terminal use this command: ./saysomething # you should hear the Mac say whatever you typed in the file Step 3 - This step is to run the file from the network, I ran into a lot of issues with file permissions when I first started doing this! I have arrived at a method that works but perhaps is not the most efficient manner… someone with a lot more experience with Apache may have a better approach. However, here is what I have done: - Check to see that Apache is working: o Start Apache with the command sudo apachectl start o The command httpd -v will give you the version of Apache you are running - Determine the user name and group: o In the terminal enter /usr/bin/id -- the user is the first name in brackets and the group is the second – you will use this information when editing the apache config file in the next step. - Edit the apache config file… o cd /etc/apache2 -- takes you to the correct directory; o sudo cp httpd.conf httpd.conf.orig -- creates a copy of the original file within the same directory; o sudo nano httpd.conf -- to edit the config file – here are the changes I made § change User _www to User XXX -- XXX is the user name you found in the first brackets above § change Group _www to Group YYY -- YYY is the group name you found in the second brackets above § change #ServerName www.example.com:80 to ServerName localhost § Make sure that the line below is uncommented: ScriptAliasMatch ^/cgi-bin/((?!(?i:webobjects)).*$) "/Library/WebServer/CGI-Executables/$1" § Make sure that these lines are there: <Directory "/Library/WebServer/CGI-Executables"> Options +FollowSymLinks +ExecCGI AllowOverride None Require all granted </Directory> o To save the editing changes, Ctrl-X to exit; ‘Y’ to save the changes; and finally hit enter to accept the file name httpd.conf o Restart Apache after making changes to the config file with the command sudo apachectl restart - Run the script: o Run the script created earlier (saysomething), from a browser on your network with the command (replace ‘'10.0.1.35’ with the network address of your Mac) http://10.0.1.35/cgi-bin/saysomething You should hear the Mac say whatever you typed in the file Step 4 - Now to run the same script from the ISY’s network resources. - The network resource is the really easy part and one can be defined with the following protocol information: GET /cgi-bin/saysomething HTTP/1.1 Host: 10.0.1.35 (replace ’10.0.1.35’ with the network address of your Mac) User-Agent: Mozilla/4.0 Connection: Close Content-Type: application/x-www-form-urlencoded After the network resource is created and saved (remember that you have to save it not only in the tab it is created but also on the main network resource tab. It can then be tested right in the network resource tab at the bottom there is a test button. - Once validated that it is working, it can now be called in an ISY program. Save program and then test by right clicking on program and Run ThenIf No Conditions - (To add one, press 'Schedule' or 'Condition')Then Resource 'SaySomethingNetworkResource'Else No Actions - (To add one, press 'Action') Now, that we have the ISY making the Mac talk, we can move on to something a little more complicated… However, before we do and while still working on the Mac, copy the 3 shell scripts that I have attached over to the Mac. They are saysomethinghttp9, saysomethinghttp9a, saysomethinghttp9b (remember to change the file types – check out the file Attached Files_text.txt) They should be placed in the /Library/WebServer/CGI-Executables directory. The permissions on the files should be changed to 0755 with the command: sudo chmod 0755 saysomethinghttp9 saysomethinghttp9a saysomethinghttp9b Then change the ownership to the same name and group that you found earlier and edited the apache config file (httpd.conf) with the command: sudo chown name:group saysomethinghttp9 saysomethinghttp9a saysomethinghttp9b I will go into more detail on how these 3 scripts are used a little later on and what they do. I am suggesting that they be put there while working on the Mac. My next post will be about setting up the Weather Script on the Raspberry pi3. I am having a problem attaching the 3 shell script files which I had saved as .txt files! Not sure why this is, but I will work on finding a solution... The 3 files aren't needed at this stage. Attached Files_text.txt 2
GDavis01 Posted January 27, 2018 Author Posted January 27, 2018 (edited) Shell Scripts for the Mac As I explained, in my last post, I was struggling with attaching the shell script files to the post! I have come to the conclusion that it is easier to simply print out the content of the 3 scripts here and if you are interested in doing this, then you will have to create the files yourself. Using a terminal session on the Mac go to the correct directory with this command: cd /Library/WebServer/CGI-Executables Create the new file with this command: sudo nano saysomethinghttp9 Then type in the code below, exit and save. Then repeat the process for the other two files Edit: After saving the files take a look at the text file Attached Files_text.txt for notes on dealing with permissions and ownership of the 3 scripts saysomethinghttp9 #!/bin/bash export PATH=/bin:/usr/bin:$PATH echo -e "Content-type: text/html\n" PHRASE=`echo "$QUERY_STRING" | sed -n 's/^.*phrase=\([^&]*\).*$/\1/p' | sed "s/+/ /g" | sed "s/%20/ /g"` cat << junk <html> <head> <title> saying </title> </head> <body> junk #----------------------- sudo say $PHRASE #----------------------- cat << junk </body> </html> junk saysomethinghttp9a #!/bin/bashexport PATH=/bin:/usr/bin:$PATHecho -e "Content-type: text/html\n"cat << junk<html><head><title>saying</title></head><body>junk#-----------------------# NOTE THAT YOU HAVE TO EDIT THE LOCATION WHERE THE 3 FILES ARE STORED# THIS SCRIPT SAVES THE 3 FILES IN "/Users/Sarah/iTuneStates/"# CREATE THE NEW DIRECTORY BEFORE RUNNING THE SCRIPT# this captures the current volume in a variable parmcurrVol=$(osascript -e "get volume settings")echo "Current Volume Setting = $currVol"var2=$( echo $currVol | cut -d":" -f2 )origVol=$( echo $var2 | cut -d"," -f1 )echo $origVolparm="set volume output volume $origVol"echo $parm parm3=$( echo $currVol | cut -d":" -f5 )echo $parm3# following code saves $parm (Volume level) to a filedestfile="/Users/Sarah/iTuneStates/Volume1.txt"echo $parm > $destfile# following code saves $parm3 (Muted - True; or Not Muted - False) to a filedestfile1="/Users/Sarah/iTuneStates/MutedState.txt"echo $parm3 > $destfile1# The next IF is to determine if iTunes is actually running. If running things happen# if it isn't running nothing happensprocess="iTunes"processrunning=$( ps axc | grep "${process}$" )if [ "$processrunning" != "" ]; then /bin/echo "$process IS running, things happen"# following code determines if iTunes is paused, stopped or playing parm4=$(osascript -e 'tell application "iTunes" to get player state') echo $parm4# following code saves $parm4 Player State (playing, stopped or paused) to a file destfile2="/Users/Sarah/iTuneStates/PlayerState.txt" echo $parm4 > $destfile2# the following IF checks if iTunes is playing... if so, it toggles playpause to pause it# I don't need to do this if it's already paused or stopped if [ "$parm4" = "playing" ] then osascript -e 'tell application "iTunes" to playpause' fielse /bin/echo "$process IS NOT running - Nothing happens"fi# the following sets the volume to the lower level for the SAY commandosascript -e "set volume output volume 18"#-----------------------cat << junk</body></html>junk saysomethinghttp9b #!/bin/bashexport PATH=/bin:/usr/bin:$PATHecho -e "Content-type: text/html\n"cat << junk<html><head><title>saying</title></head><body>junk#-----------------------# NOTE THAT YOU HAVE TO EDIT THE LOCATION WHERE THE 3 FILES ARE STORED# THIS SCRIPT READSS THE 3 FILES IN "/Users/Sarah/iTuneStates/"# CREATE THE NEW DIRECTORY BEFORE RUNNING THE SCRIPTS# this reads the file and saves it to the variable $parmfile="/Users/Sarah/iTuneStates/Volume1.txt"echo $fileread -d $'\x04' parm < "$file"echo $parm# this returns the volume to where it was originallyosascript -e "$parm"# this reads the Muted State file and saves it to the variable $parm3file="/Users/Sarah/iTuneStates/MutedState.txt"echo $fileread -d $'\x04' parm3 < "$file"echo $parm3if [ "$parm3" = "true" ]then osascript -e "set volume with output muted"fi# This next section checks to see if iTunes is actually running# if so, and the state saved in the file is "playing" playpause is toggled to restart# if the state was "paused" or "stopped", or iTunes isn’t running, there is nothing to doprocess="iTunes"processrunning=$( ps axc | grep "${process}$" )if [ "$processrunning" != "" ]; then /bin/echo "$process IS running, things happen"# this reads the Player State file and saves it to the variable $parm2 file="/Users/Sarah/iTuneStates/PlayerState.txt" echo $file read -d $'\x04' parm2 < "$file" echo $parm2 if [ "$parm2" = "playing" ] then osascript -e 'tell application "iTunes" to playpause' fielse /bin/echo "$process IS NOT running - Nothing happens"fi#-----------------------cat << junk</body></html>junk Attached Files_text.txt Edited January 27, 2018 by GMD99
GDavis01 Posted January 27, 2018 Author Posted January 27, 2018 Weather Script on the Raspberry pi3 -- Python_Weather.py As I explained in the Overview, I took a lot of the direction for this coding from the Visual Basic script that btreinders wrote. Registering with wunderground.com You will have to register with wunderground.com and get an API key to add into the script. Registration is free as long as you make less than 500 calls per day and less than 10 calls per minute. https://www.wundergr...om/weather/api/ Setting up the Rpi I have been modifying my current installation over the last few years and as such I cannot remember all the steps I took, or the packages (libraries) I downloaded! So, in order to see if I could recreate an environment, on the Rpi, where this whole process works, I started a fresh install, on my Rpi3, using NOOBs v2.4.5 with a new 32Gb microSD card. The new set up is running "Raspbian GNU/Linux 9 (stretch). The good news is that I got it up and running with only a few fixes. Attached is a file RPI_Notes_text.txt with some additional notes on what I did to set up the Rpi, including testing the weather script and other areas. I will not be going into any depth on actually setting up the Rpi in this post. That being said, most of the comments below are based upon my original installation. Here is the operating system that I am running on my Rpi, from the command cat /etc/os-release NAME="Raspbian GNU/Linux" VERSION_ID="8" VERSION="8 (jessie)" ID=raspbian ID_LIKE=debian HOME_URL="http://www.raspbian.org/" SUPPORT_URL="http://www.raspbian.org/RaspbianForums" BUG_REPORT_URL=http://www.raspbian.org/RaspbianBugs I run my Rpi as a headless device, meaning that I don’t have a monitor or keyboard/mouse attached to it. My Rpi only has an ethernet connection and power. I access the Rpi through my home network. I use 3 apps on my PC to control and program the Rpi. They are: - TightVNC Viewer – this app provides a GUI interface that provides, in my opinion, an easier programming environment when compared to using the terminal. I think that this is particularly true when you are creating more complicated scripts. - WinSCP – this app is like Windows Explorer for the Rpi (or for the iMac). I find this very useful for finding files; transferring files between my PC and both the Rpi and the iMac; viewing the content of files on the Rpi; or looking at the permissions of the files on the Rpi. - PuTTY – this app is a very versatile tool for remote access to another computer. I use it for terminal sessions on both the Rpi or the iMac. One thing that you will notice is that some of my Python scripts are written in Python2 and some are written in Python3. This is as a result of my searching the web for ways to do what I want… if I found it in Python2, then that’s how I wrote the script, but if I found it in Python3 then I went that way! The Python_Weather.py script is written in Python 2.7.3. Essentially, this script accesses the wunderground.com site, grabs data from their XML files and creates a new XML file, which is then stored on the Rpi. It also massages some of the data so that my iRule application can read the data correctly. A few notes: - In my setup, the Python scripts reside in the /home/pi directory; - I created a subdirectory home/pi/templates which is where the created XML file is stored. The script will be looking for this directory. - The script includes a lot of explanations of what each section purports to do. So, reading the script code will probably help you better understand what I did; - Living in Canada, my data extraction is in Celsius, and metric (cm’s, mm’s, kph). I have comments in the script in those places where editing would be necessary to change the code to extract Fahrenheit and standard measures (in, mph); - There are numerous lines commented (‘#’ at the beginning of the line) where I am using the ‘print’ command (e.g. # print feelslike.text). I originally used those statements to echo the output to the screen when I was writing the script as a debugging tool. Uncommenting them will echo a lot more to the screen but is useful when trying to debug things. Of course, you will only see the results echoed to the screen when you manually run the script from a PuTTY session or from the GUI interface using TightVNC. This is discussed further on. - You will need to edit the script for the following things: o Put in your wunderground API key o Choose your local airport or airports to get the bulk of the weather data. Note that if there aren’t any airports nearby, you can use the data from a local city that includes the data you need. You will have to play around with wunderground.com and see where you can get your best information. If you are only going to use one airport or city, then enter the same code in both airport ID’s, because the data is averaged within the script. § To find the airport data enter this line is a browser replacing your wunderground key and the airport 4-digit code at the end (note that I inserted a space between 'http' and '://' so that you can see the full address) http ://api.wunderground.com/api/wunderground_key/pws:0/conditions/forecast/alerts/almanac/astronomy/q/CYYZ.xml o Choose Private Weather Stations (PWS) close to your home § Go to the site wunderground.com and enter in the search box your local area § This should bring up a page relatively close by; § Under the name of the location (on the screen) you will see the name of the station it found… if you click on “Change” it will bring up a map and a list of Nearby Weather Stations and their ID (e.g. IONHUNTS4) § This ID is what you have to enter into the script. The script accommodates 3 ID’s and then averages the data. In the script there is an explanation on how to set things up if you only want to use 1 PWS or 2 PWS’s instead of the three. § Note that some PWS’s do not always report! So, I would recommend that you look at their XML data before you make your choice. Enter this line in a browser, replacing the ID at the end. This will show you the XML data for that PWS (note that I inserted a space between 'http' and '://' so that you can see the full address) http ://api.wunderground.com/weatherstation/WXCurrentObXML.asp?ID=IONHUNTS4 § Also note that the script currently only grabs the current temp from the PWS stations because most, I have found, do not report much more with any accuracy or consistency. The rest of the data comes from the airport(s) or local city data. o Enter in your ISY credentials so that data can be moved over to the ISY (near the end of the script). I currently only move 4 items into the ISY as variables, as follows: § The current day’s forecast rain POP %. I use this for programs I have written in the ISY to decide on how much to water my lawn and gardens with my sprinkler system. § I also populate 3 other variables on the ISY for current average temperature for the Airports; the current average temperature for the PWS stations and the variable, calculated in the script, for the difference between the previous 2. I do this because if the difference between the current temps is greater than 8 degrees Celsius then I generate a notification (email) to myself from the ISY. This amount of difference is a good indication that the PWS’s are not reporting properly and that I should check to see if one or more are no longer reporting! o If you want to bring more data over, to the ISY, then copying the code at the bottom and adding more instances of that code is easily done. The only editing is to choose which variable to pass to the ISY (value = ???) and add in the location of the variable on the ISY (e.g. 1/42 means integer variable (1) and the number of the variable within the integer variables “42”; 2/42 means state variable (2) and “42” means the number of the variable within the state variables). Remember to create the variables on the ISY beforehand. o Also note that my script currently populates the following variables on the ISY: integer variables 42, 51, 52 and state variable 22. You should make sure that these aren’t being currently used or change the script accordingly. Testing the script As I mentioned in a post above, I have found that the best way to edit scripts is to use TightVNC Viewer. I will open the GUI interface and start the Programming Python2 environment (Python2 because I know that my script was written in Python2). Then I will open the script (Python_Weather.py), do my edits, save the file and then press F5 to run it. What I like about this method is that, after the script has run, the variables, set in the script, are still in memory, so I can debug issues by doing a ‘print variable’ and see what was set. I have not found a way to do this when running the script from a PuTTY session. However, having said this… for some reason, this script when run from the GUI interface throws out errors when it gets to the coding to transfer data to the ISY! The way around this is to comment out the 10 lines of code at the end of the script that send the data to the ISY and then test the script up to that point. Interestingly, the script, including the transfer to the ISY, does run correctly when run manually from a PuTTY session or if it is set up to run automatically in the background through the crontab file. These things are explained below. I am attaching a copy of the Python script (Python_Weather.py) as well as the XML file (weather.xml) that the script creates. (Once again, remember to change the file types – check out the file Attached Files_text.txt) How and when does the script run I run the script every 10 minutes, which overwrites the XML file I create and updates the 4 variables on the ISY. To do this, I first opened a PuTTY session on the Rpi and logged in (default user id is “pi” and the default password is “raspberry”). Then I edited the crontab file with this command… crontab -e This opens up an editor (nano) which I used on the Mac. Note that if it’s the first time you edit the crontab file, it will ask you which editor you want to use. I like nano so that is my choice. Add in the following line: */10 * * * * sudo python /home/pi/Python_Weather.py Then Ctrl-X, ‘Y’ to save the changes, and ‘Enter’ to accept the name and exit the editor. You should see a message on the screen “crontab: installing new crontab”. Now this script will run every 10 minutes. Note that if you want to run the program manually to see if it works at the command line (in a PuTTY session) type in sudo python /home/pi/Python_Weather.py. This can be done even if the crontab setting is in place. (Note that, for some reason, I have to be in the /home/pi directory to have this command work without errors…) To check to see that things are working, I will often run the script manually so that I don’t have to wait the 10 minutes. First, I open up the Admin Console of the ISY and arbitrarily change the values in the variables that the Python script passes to the ISY. Then, I will run the script from the command line (manually), before going back to check the variables on the ISY to confirm that they changed. Next, I will have previously opened up a WinSCP session logging into the Rpi and navigate to the directory home/pi/templates. I will refresh the page and look at the time of the file which should coincide with the time you ran the script from the command line. This tells me that the Python script ran as expected. After setting up the script to run automatically, I now have the data I need for my iRule application. Note that to allow the iRule application to actually access the data from the XML file on the Rpi, I am running a Flask server, on the Rpi, which hosts the XML file. I don’t plan to explain this any further here, however, there are some notes related to setting up the Flask server in the attached file RPI_Notes_text.txt. The next post will look at how I setup things to get the iMac to ‘say’ the weather… app1c.txt Python_Weather.txt RPI_Notes_text.txt weather.txt Attached Files_text.txt
GDavis01 Posted January 27, 2018 Author Posted January 27, 2018 As I mentioned in the last post, this one is to explain the steps I took to have our iMac ‘say’ the weather. 1 - Set up the HTTPServer on the Rpi -- RunProgramOnRPI.py As I explained in the Overview, this script is a modification of a script that larryllix provided in some thread on this forum. It’s a Python 3 script that starts an HTTPServer when the Rpi boots up. I have attached a copy of the Python3 script that I use – RunProgramOnRPI.py (don’t forget to change the file type) Essentially this script sets up a HTTPServer to run in the background, on the Rpi, which waits for a call from an ISY network resource. The network resource sends the script a parameter which the causes the script to run certain commands. When it sees the number 9 as the parameter, it runs another Python script Python_Weather_Forecast_text.py, which I will explain further down. The network resource that sends the parameter has the following characteristics: GET 9 HTTP/1.1 Host: 10.0.1.33:8111 (change the address to the Rpi’s address) User-Agent: Mozilla/4.0 Connection: Close Content-Type: application/x-www-form-urlencoded The port number (in my case 8111) is set up in the coding of RunProgramOnRPI.py near the bottom of the script. If this port number doesn’t work, edit the script and change it. Note that I use this HTTPServer for several other things depending on which parameter I send from the ISY. As an example, if I send the number 6 as a parameter together with some specific text in the network resource, the HTTPServer saves the text in a file and then calls another Python script which reads that file and causes the iMac to ‘say’ the text. I use this, for example, when I want to say “Garage door number 3 is still open and should be closed soon”. A little more on this further down… You might wonder why I go through a convoluted process of saving the text to a file and calling another script, which eventually calls 3 scripts on the iMac, to get it to ‘say’ the text, when I could simply use a network resource and send the say command with the text directly to the iMac to be ‘said’! It all gets back to the fact that we often use the iMac to play music and the volume level for iTunes music when used for the ‘say’ command is extremely loud. The manner in which I have set it up allows me to pause the music if it is playing and lower the volume for the broadcast. Then to reset the volume after the broadcast and restart iTunes, if it was playing. In the attached script (RunProgramOnRPI.py), I have removed some of the coding, related to other parameters I send to the HTTPServer, in order to keep the script as simple as possible. However, as you will see, the server can be coded to do numerous things using input from the ISY (e.g. alarm sensors) to trigger the iMac to ‘say’ things or run scripts on the Rpi. As an example, I also use this facility to transfer variables on my cottage ISY to my home ISY (I have a similar set up running at my cottage: ISY and Rpi3). If you read the coding of the file, you will see that there are 2 parameters that are coded (6 and 9) and I think the logic for each is self-explanatory when read in conjunction with these posts. The next step in setting up the server is getting it to run automatically and in the background on start-up. Often things that you want to run on start up, on an Rpi, are placed in the rc.local file (/etc/rc.local) however, for some reason, I could not get it to work running from this file! (I already have, in the rc.local file, the NodeLink set up; the command line to run my No-IP DUC file; and the Flask server which I referenced in my last post) Fortunately, there are several different ways to have things run on a reboot… I chose to use systemd. First of all, I created a file, RunProgramOnRPI.service (copy of the file is attached), in the directory /etc/systemd/system. This service, on a reboot of the Rpi, starts up the HTTPServer by running /home/pi/RunProgramOnRPI.py (Python3 script). After placing the RunProgramOnRPI.service file in the correct directory, and before rebooting, use this command to enable the service. sudo systemctl enable RunProgramOnRPI.service Then, reboot the Rpi. The HTTPServer should now be running. Note that before setting up the automatic load via the system service, I found it useful to load the HTTPServer manually by entering this command in a PuTTY session sudo python3 /home/pi/RunProgramOnRPI.py This command simply starts the HTTPServer which sits and waits for calls. Then, using an ISY network resource, send a parameter to the Rpi (one that has been coded into the script) and watch the output in the PuTTY session. This is a good way to debug things. If you want to use the manual method, for debugging purposes at a later date, follow these steps: 1 stop the automatic system service with the command: sudo systemctl disable RunProgramOnRPI.service 2 at this point you could simply reboot the Rpi with the command sudo shutdown -r now and then go to step 6 Or, if you don’t want to reboot, follow the next steps: 3 run this command -- sudo netstat -lepunt (this shows you active servers running on your system) 4 in the column ‘PID/Program name’ find your HTTPServer – it should be something like “751/Python3” (your PID will be different than 751) and the Local Address should match what you have in the Server script (RunProgramOnRPI.py) – Mine is 0.0.0.0:8111 5 then enter the command sudo kill -9 751 (change the value 751 to match your PID number). This stops the server from running. 6 Then enter the command to run the server manually: sudo python3 /home/pi/RunProgramOnRPI.py 7 When you are finished with the manual method, hit Ctrl-C to exit the HTTPServer 8 Then enter the command to re-enable the system service: sudo systemctl enable RunProgramOnRPI.service 9 And finally reboot the Rpi with sudo shutdown -r now. This will start the service when the system is back up and running. 2 – Python Script to Massage the Text to be ‘Said’ -- Python_Weather_Forecast_text.py In order for the iMac to say things properly there is a reasonable amount of massaging that has to be done to the text! For example, this line of forecast taken, from the XML file I created, would not be read correctly by the iMac “Cloudy early with partial clearing expected late. A few flurries or snow showers possible. Low -7C. Winds SW at 15 to 25 km/h” The ‘say’ command doesn’t deal well with spaces or know to say “SouthWest” when it sees “SW” So, I wrote a Python script (Python 2.7.3) Python_Weather_Forecast_text.py which sits on the Rpi in the /home/pi directory. I mentioned this script above as the one that is executed by the HTTPServer (RunProgramOnRPI.py) when it receives the parameter number “9”. This Python script (Python_Weather_Forecast_text.py ) pulls the data from the weather XML file; reformats the data into an iMac friendly format; and then calls 3 shell scripts, which sit on the iMac, causing the iMac to ‘say’ the weather. The script is fairly easy to read and I have comments included which should explain the various steps. The 3 shell scripts on the iMac sit in the /Library/WebServer/CGI-Executables directory. These are the 3 files which I suggested earlier should be copied to the Mac. Here is what they do: "saysomethinghttp9a" - this is the first shell script and it does several things, including: - determines the current sound volume level on the iMac, and stores it in a file; - sets the volume level to the level I like for the broadcast, which is 18 - determines if the iMac has been muted or not, and saves that setting in a 2nd file - determines if the iTunes application is currently running on the iMac; if so, it · determines if iTunes is 'playing', ‘stopped’ or 'paused' on the iMac, and saves that setting in a 3rd file; and · if iTunes is ‘playing’, it toggles the play/pause to pause the music "saysomethinghttp9" - this shell script, which is called numerous times, is the script to actually 'say' phrases "saysomethinghttp9b" - this script is the last one to run, and it also does several things, including: - reads the file with the original sound volume level, and resets the volume to that level - reads the Muted State file, and if the iMac had been muted before the broadcast, then it mutes it - determines if the iTunes application is currently running on the iMac; if so, it · reads the Player State file, and if iTunes was playing before the broadcast, it toggles the play/pause to restart the music Note that saysomethinghttp9a saves 3 files to the directory /Users/Sarah/iTunesStates, and saysomethinghttp9b reads the three files. You have to edit both of those files to change the directory to wherever you want to save the files on your Mac. That is the end of my description on how to get the iMac to ‘say’ the weather. This next section explains how I get the iMac to say whatever phrase I want to write into an ISY network resource. 3 – Python Script to Say Whatever -- Python_text_to_Say.py I made reference to using a network resource on the ISY to send another parameter “6” accompanied by a textual phrase to the HTTPServer to have the iMac ‘say’ the textual phrase. An example of the structure of the network resource is as follows: GET 6/Garage+door+number+2,+is+still+open,+it+should+be+closed+soon HTTP/1.1 Host: 10.0.1.33:8111 (change the address to the Rpi’s address) User-Agent: Mozilla/4.0 Connection: Close Content-Type: application/x-www-form-urlencoded In this example, I am sending the parameter “6” with the text “Garage door number 2 is still open, it should be closed soon”. Note that you need the “+” instead of the spaces. Prior to running this script via the network resource, create the following new directory on the Rpi, as the script will be looking for it: /home/pi/DataFiles. (capitalization is important) When the network resource sends the call to the Rpi, the HTTPServer receives it and if the parameter is “6”, it first, saves the text phrase to a file on the Rpi, and then executes a different Python script than the weather one (Python_text_to_Say.py). This script reads the text from the saved file and executes the same 3 shell scripts, on the iMac, to have it say the text phrase. (I am sure that there is a way to pass the text from the HTTPServer to the Python script, without first saving it to a file, but I ran into issues and finally decided on using a file to store and then extract the data. It might have something to do with the server running Python3 and the script Python2!) I find this facility useful to have the iMac say anything I want simply by writing an ISY network resource… This is what I use to have the iMac tell Alexa what to do! That’s it… Hopefully this is helpful to some of you. RunProgramOnRPI.txt RunProgramOnRPI_service.txt Python_Weather_Forecast_text.txt Python_text_to_Say.txt Attached Files_text.txt
Michel Kohanim Posted January 28, 2018 Posted January 28, 2018 Hi GMD99, THANK YOU! With kind regards, Michel
Recommended Posts