Amazon Echo and ISY

[blueman2]

I have just placed a new version of the AWS_Configuration (version 4.0.0) program on my web site (www.the-gordons.net). It is a major revision and handles the Harmony Hub in addition to the ISY. It is configurable as to the inclusion of the Harmony Hub configuration capability, and the default is not to include it. If you do not have the Harmony Hub and are currently running a version of the Configuration system (AWS_CONFIG.com) I suggest you upgrade to version 4.0.0. Version 4.0.0 has changes to the user interface and better code handling.

 

If you are going to use the Configuration system to handle the Harmony hub, then I strongly suggest you read the documentation included in the Install package. You will also need to be running the BWS Systems Restful Harmony application.

 

I personally do not own a Harmony system as the ISY994 handles all my needs for Home Automation and I use Pronto PRO remotes for my theater and the various TV's around the house. The New version has been tested in an ISY994/Harmony Hub environment and appears to be working correctly. This means I can not be of much help in setting up the basic Harmony system with the Restful Harmony application, but there are several people who have posted here that are capable of assisting in that endeavor.

 

Barry has done an amazing job with his AWS Config tool.  I am running the latest versions of the BWSSystems apps (ha-bridge-0.4.10.jar and restful-harmony-0.1.4.jar) on my Raspberry Pi 2, and running Barry's AWS_Config v4.0.0 tool on my laptop to configure the setup.  The wonderful thing about Barry's tool is that it auto populates everything you will need from your ISY (Devices, Scenes, Programs, Variables) and from your Harmony (Activities, Device Buttons) and has very well thought out logic for how to use those devices and options.  

 

One of the neater things Barry did was to include the option for button pairs.  For example, volume up and volume down.  If you select Volume Up button on a device, it will automatically populate Volume Up for the "ON" command and Volume Down for the "OFF" command.   And since it is configurable, you can put in any pair you want. For me. I use Aspect ratio of 4:3 and 16:9 as a pair.  When I say "Alexa, turn on aspect ratio", my TV goes into 4:3 mode.  When I say "Alexa, turn off aspect ratio" the TV goes back to normal 16:9.  As you can see, this opens the door to a lot of on/off options.  

 

Of course, this program also will allow you to turn on and off any Harmony Activity by simply saying "Alexa, turn on TV" or "Alexa, turn off TV".  

 

Along with the NEST controller program by xKing (also running on my Raspberry Pi 2), I have pretty much all the automation I originally wanted with the Echo.  I am a very happy camper!

Edited November 2, 2015 by blueman2

[Xathros]

I'm really trying to hold out for the Echo support via the UDI Portal and ISY skill, but it's getting harder and harder to resist rebuilding my current setup with the newer tools posted here.  The Harmony integration might just have pushed me over the edge...

 

-Xathros

[barrygordon]

I, although the author of the configuration tool, want to see what Michel et. al. produces as the ISY skill. The UDI team has produced an amazing product in the ISY994. I am just about to start moving my ISY to version 5. I understand that you need to move first to the latest ISY994 4.x version and then to version 5.  I tried this before without that intermediate step and was unsuccessful 

 

Remember the leading edge of technology is the edge of an abyss, implying that technology is an abyss. 

[blueman2]

I'm really trying to hold out for the Echo support via the UDI Portal and ISY skill, but it's getting harder and harder to resist rebuilding my current setup with the newer tools posted here.  The Harmony integration might just have pushed me over the edge...

 

-Xathros

 

I do not view the two as mutually exclusive.  The configuration I am using above is based on "Primary" Echo invocations.  They are all directed to Alexa using the Hue emulator.  This creates some limitations, but also some simplicity.  Especially from a WAF perspective.  Being able to say "Alexa, turn on TV" is easier than "Alexa, tell Izzy to Turn on TV".  Especially since my wife and I are using Alexa constantly for lights, TV, Den Music, whole house FAN, ceiling fan, thermostat temperature, etc.  All of these are done with fewer words because they are all directed to Alexa.  My most common uses are:

 

Alexa, turn on TV

Alexa, set thermostat to 69

Alexa, turn house fan on medium

Alexa, turn on Kitchen Area

Alexa, turn on door chime (this tells the ELK to set door chime to active)

Alexa, turn on night mode (a special program on my ISY that sets everything I need to go to bed, such as checking all door locks, turning off all lights, etc)

Alexa, turn on Den Music (turns on Roku to Pandora station I like, and turns on receiver for sound)

Alexa, turn on Den Loudness (turns up the receiver by 4 volume up button presses)

Alexa, turn on Den Mute (mutes the TV and Receiver in the Den)

 

I doubt I will ever want to change these basic functions over to the ISY skill/portal because I do not want to have to add the "tell Izzy to..." wording to each command.  BUT, there will be things that are too complicated or impossible to do with the direct commands based on the Hue emulator.  That is where the ISY Skill/portal will come in.  

 

So I think both will stay around for some time.  I plan to purchase the ISY skill/portal service as it matures, but also plan to keep my RPi setup as well.  

Edited November 2, 2015 by blueman2

[blueman2]

BTW, one of the things that allows me to use the Hue Emulator with such success and high WAF is that Amazon does a very good job interpreting 'friendly names'.  For example, I have a Hue emulator entries with friendly name "House Fan Low", "House Fan Medium", and "House Fan High".  While you can say "Alexa, turn on House Fan Medium", you can also switch the words around as you wish.  So the following also work:

 

Alexa, turn house fan on medium

Alexa, turn on house fan to medium

 

I am surprised at how well the Echo parses phrases to get to the right result.  Of course, it can also be maddening at times when you have too many devices with the same words in it.  In that case, it comes back with "I do not know which device you mean" or something like that.   

[barrygordon]

I believe someone had posted an utterance generator which takes a basic utterances and generates the other phrases which mean the same thing.  I had just read about it last night.  Saves a lot of time in building the detailed set of utterances which must include all the permutations of a phrase

[barrygordon]

A new user of the AWS_Config system has found a few bugs (non-Harmony related) so I have posted an update to the new version.  The new version (4.0.x) completely replaces the old version and is fully compatible with the older version. The only user interface change required is the additional ini file lines if yoiu wish it to work with the Harmony Hub.

 

As an aside, BWS Systems is just finishing up a new version of the HA Bridge which incorporates all of the code for the Restful-Harmony app so it will be all that is required. Naturally I will ensure that the configuration system works with it. 

[ScottAvery]

Forgive my vanity, but I will quote myself to renew the campaign for "Izzy" as the preferred name:

 

"Come on, it's snazzy!  It feels like the geeky ISY will be coming out as a debutante at the cotillion by just taking off her eyeglasses!  A whole new persona for the great, open consumer market.  Siri and Alexa are all talk; Izzy gets it done."

 

(or, "Izzy goes all the way!")

[btreinders]

Ok, I guess I am confused.  Why does UDI need to make a second name, skill, whatever it is?  Can't they just make Alexa understand things like open and close, start and stop, etc...?  Like Alexa, lock all the doors.    Or even Izzy, lock all the doors.  I thought that was the whole purpose of the portal/skill/whatever.  So to do other things related to what the ISY can do I will have to say Alexa, tell whoever to lock all the doors?  Sigh. The family will laugh and laugh and laugh at me.  More than they already do!    

[barrygordon]

It is an amazon limitation.  To make the ECHO understand the speech they have to have it listening for a specific word known as the wake up word.  Amazon only allows for two wake up words. Alexa or Amazon. To then allow for secondary skills to be developed Amazon decided to have "Alex, tell <skill Identifier> or Alexa, Ask, <skill identifier>.

 

We are discussing the skill identifier here as we have no say regarding  the wake up word. I have discussed with Amazon several alternate approaches but so far they have been intransigent on my suggestions.

 

I would like to see an open wake up word so the user can pick it. I would then like to see a chain of skills and the user be able to order the chain. If a skill on the chain can process the speech, then it does so, if it can't it hands it off to the next skill in the chain. This scheme does require however excellent speech recognition, as it eliminates the "I didn't understand . . . " logic until the last skill in the chain is tried.

 

So as with most things in the world, we live with what we have to work with.

[MWareman]

Remember, as far as I understand there are two APIs UDI is going to be using. The 'Home Control' API and the 'Skill' API.

 

Home control allows a very natural 'Alexa, turn on the kitchen light'. It does not require referencing a skill name, but is limited in vocabulary (by Amazon) to very common commands (on, off, dim, open, close etc...). This is the same API that Hue currently interfaces with I believe... So things that are currently possible to say via the Hue emulator should work this way via ISY once it it approved by Amazon.

 

The Skill API requires the extra 'Alexa, (Tell|Ask) <skill>', but allows the implementer of the skill complete autonomy over the language after the name of the skill. So, things are possible with a skill that are not possible with the home control API, at the sacrifice of having to reference a skill in the verbal command..

 

Both are in progress, I believe.

 

Michael.

[blueman2]

Remember, as far as I understand there are two APIs UDI is going to be using. The 'Home Control' API and the 'Skill' API.

 

Home control allows a very natural 'Alexa, turn on the kitchen light'. It does not require referencing a skill name, but is limited in vocabulary (by Amazon) to very common commands (on, off, dim, open, close etc...). This is the same API that Hue currently interfaces with I believe... So things that are currently possible to say via the Hue emulator should work this way via ISY once it it approved by Amazon.

 

The Skill API requires the extra 'Alexa, (Tell|Ask) <skill>', but allows the implementer of the skill complete autonomy over the language after the name of the skill. So, things are possible with a skill that are not possible with the home control API, at the sacrifice of having to reference a skill in the verbal command..

 

Both are in progress, I believe.

 

Michael.

 

+1  Well put.  

 

In my mixed world, I plan to use my own RPi setup to do the 'home control' portion that has commands going directly to Alexa.  While I have not seen the UDI offerings yet, I am one who likes to tinker and tweak to my own needs and feel better having my own mini-server running in my home and being able to tweak all the settings for integration with new devices (like with the harmony remote and with my pool controller). But, I think I am in the minority in terms of wanting to 'roll my own' for this.    

 

For the Skill portion, where we need to have Alexa 'ask Izzy to...', I will probably use the UDI portal skill.  Depends on how hard it is to create my own skill.  I have started the process of creating my own skill, but it may be more complex than I desire.  For example, I had to sign up as a developer and Amazon could theoretically take away my ability to run my own skill at some point since I am not creating a mass market product with it (I would be the only user!).

Edited November 3, 2015 by blueman2

[btreinders]

Thanks for clearing all that up.  I guess I will use whatever is the least amount of words I need to say for each thing I want to do.  Leave the easy stuff like turning on lights to the HUE emulator (or ISY Home Control API) and use the skill for more complex tasks.  I was under the impression that HUE came up with the control words but I guess that's all Amazon.  Is sure wish they would allow Close the garage door!    Seems strange that open garage door works and close lights works but not close garage door.  Oh well.  Alexa, tell Izzy to close the garage door isn't that bad I guess.  Now that I think about it, I'd rather say Alexa, have the computer close the garage door than another name.

[blueman2]

... sure wish they would allow Close the garage door!    Seems strange that open garage door works and close lights works but not close garage door.  Oh well.  ...

 

Actually, this is smart thinking on Amazon's part.  They tend to do a good job of looking at commands such as Thermostat, Light, Door, etc.  They allow Open Garage Door as a command because that is pretty safe.  But CLOSE garage door?  Pretty dangerous if there is a person or car in the way.  

[mwester]

Oddly enough "Open the Garage Door" should work, but you might have to say "Close the Garage Door light" or "Turn off the Garage Door" to do the opposite.  "Opening" and "closing" lights is an idiom used by some (I've heard that most often used by those for whom English is a second language).

[btreinders]

Actually, this is smart thinking on Amazon's part.  They tend to do a good job of looking at commands such as Thermostat, Light, Door, etc.  They allow Open Garage Door as a command because that is pretty safe.  But CLOSE garage door?  Pretty dangerous if there is a person or car in the way.  

 

That's what door sensors are for are they not?    I also have a camera in the garage and the ISY will not shut it if there is motion and I can see when it's safe as well.  

 

mwester, Yes, turn on close the garage door works fine.  I just don't like having to say that.  Turn off the garage door works too.

[rdavidowski]

Hi Blueman2

 

Do you have the restful-harmony working with Barry's configurator?  I just can not get it working. Alexa always comes back with a "Device is not currently blah blah blah". Can you give some insight on how you configured this to work?

 

Thanks

rdavidowski

[barrygordon]

What versions of the three software systems are you running:

 

The HA Bridge jar ?                       must be 0.4.9 or better

The Restful-Harmony Jar?            must be 0.1.4 or better

the AWS_Configuration.exe ?       must be 4.0.0 or better 

 

If you are running those versions and getting no errors in the AWS_Configuration system, Then please describe your configuration in detail and I will try and help you. Include what you are running the Java stuff on (What Java version) and what type of host (PC MAC RPi).  If on a RPi  then in a terminal do

sudo ps -ef | grep Java

to ensure that the emulators are running. You should see a line for Java and one for each emulator that is running. AWS_Config only runs on windows. and only needs to be running when you are configuring the bridges.  The bridges must always be running 24/7 which is why a lot of us run them on RPi's.

Edited November 3, 2015 by barrygordon

[blueman2]

Sure.  First, are you able to get just the HA-bridge software from BWS working with Barry's config and your echo?  

 

EDIT: Barry and I posted at same time.  Barry's questions above are same as mine.  We will help you get it going.  Just need more info on where you are now.  

Edited November 3, 2015 by blueman2

[rdavidowski]

Sorry Barry and Blueman2

 

I posted this question late last night and went to sleep.  Right now I am at work. But basically I have the latest versions of the 3 needed apps.

 

restful-harmony.0.1.4

Ha-Bridge.0.4.10

Barry's Configurator 4.0.4

 

I have everything working betwween the HA-Bridge and the ISY and Echo with Barrys latest configurator.  Is just that yesterday a introduced the restful-harmony into the mix with mixed results.  I know the resful-harmony.jar is running because I can request all activities through a Web Browser and it does respond with all the info back plus Barrys configurator finds all my activities and devices on the Harmony Hub. Here is the thing.... Responce for the activities is eratic most of the time Alexa responce is it did not find the device but every once in a while it does turn ON the activity.  What it always does is turn Off the activity. All of this is running on a Windows 10 PC.

 

I do have this Harmony stuff working with Alexa through another .exe file that I right now do not remember cus I am at work. The way it works is that I have a web activity app listening for web calls in the windows machine and then that app executes the exe file adding the apropiate parameters to make the Harmony hub execute the activity I called for through alexa. I use the custom setting on Barry's configurator for this. It work 100% everytime. The reason I want to change this to the resful-harmony.jar is because I want to move all of this setup to a Pi in order to not have the windows machine always On.

 

Also I detected that Barry's Configurator version 4.0.4 force closes when I hit custom button and also when I hit the test ON/OFF button for a Harmony activity everything else does execute the Test On/Off.  This did not happened with version previous to 4.0.0.

 

Any suggestion on what Raspberry Pi 2 Kit to buy with the right OS?

 

Thanks

Edited November 3, 2015 by rdavidowski

[barrygordon]

I will look into the force close on the custom button. I need to remove the test on and test off for the harmony hub. I can only do that for custom and ISY entries. I will post here again when I figure out what is wrong.

 

I can not help you with the restful harmony jar as all the configuration program does is set up the configuration. You should be able to test the restful harmony jar independently.

 

Any RPi 2 kit should be good. I generally buy one with just the clear plastic case and possibly a copy of NOOBS which can also be downloaded from the raspberry pi org. Using NOOBS I select the Rapbian OS and configure it. I run the pi wired with DHCP and have my router reserve a static address. As soon as the OS is loaded be sure and do a sudo apt-get update and sudo apt-get upgrade to get all the latest changes.

Edited November 3, 2015 by barrygordon

[bocarob]

 Barry and Blueman2:

 

​Barry I did get the new pi 2 and manage to get all the bridges up and working great harmnoy rest is also working as i can make calls from the web and I get info and action back but when i uses 4.0.4 I get the same problem as above  it force closes and alexa see the device but can not act upon it 

 

here is my config:

[system]

My Browser         = C:\Program Files (x86)\Google\Chrome\application\chrome.exe

My Echo            = echo.amazon.com/#settings/connected-home

Debug              = All                                               ' debug options are any or all of 'All ISY DVC HRM SCK CSTM'

Emulators          = 4, 10   ' first number is the maximum number of emulators to look for, the second the number of seconds to look.

 

[uDI ISY994]

ISY userid         = ********

ISY Password       = p*****

 

[Emulators]                      ' This is an optional section. It's presence precludes a uPnP search

192.168.1.87:8081=Bridge1

192.168.1.87:8082=Bridge2

192.168.1.87:8083=Bridge3

192.168.1.87:8084=Bridge4

 

[Harmony Hub]

Emulator           = 192.168.1.87:8085

Antonyms           = Up/Down, Left/Right, On/Off, Forward/Backward, ToStart/ToEnd, BackToTick/ForwardToTick, FastForward/Rewind, Play/Pause, 16:9/4:3

 

[Custom Endpoints]

 

 

[Device Types]

Controller         = 0 

Dimmer             = 1   

Switch             = 2

Network            = 3

Irrigation         = 4

Climate            = 5

Pool               = 6

Sensors            = 7

Energy             = 9

Shades             = 14

Access             = 15

Security           = 16

X10                = 113

'                         ' If you have other types of Zwave devices that have an On/off operation they may be added here 

[Zwave Categories]

id=0,   name=Uninitialized

id=101, name=Unknown

id=102, name=Alarm

id=103, name=AV Control Point

id=104, name=Binary Sensor

id=105, name=Class A Motor Control

id=106, name=Class B Motor Control

id=107, name=Class C Motor Control

id=108, name=Controller

id=109, name=Dimmer Switch, dim=true

id=110, name=Display

id=111, name=Door Lock

id=112, name=Doorbell

id=113, name=Entry Control

id=114, name=Gateway

id=115, name=Installer Tool

id=116, name=Motor Multiposition

id=117, name=Climate Sensor

id=118, name=Multilevel Sensor

id=119, name=Multilevel Switch

id=120, name=On/Off Power Strip

id=121, name=On/Off Power Switch

id=122, name=On/Off Scene Switch

id=123, name=Open/Close Valve

id=124, name=PC Controller

id=125, name=Remote

id=126, name=Remote Control

id=127, name=AV Remote Control

id=128, name=Simple Remote Control

id=129, name=Repeater

id=130, name=Residential HRV

id=131, name=Satellite Receiver

id=132, name=Satellite Receiver

id=133, name=Scene Controller

id=134, name=Scene Switch

id=135, name=Security Panel

id=136, name=Set-Top Box

id=137, name=Siren

id=138, name=Smoke Alarm

id=139, name=Subsystem Controller

id=140, name=Thermostat

id=141, name=Toggle

id=142, name=Television

id=143, name=Energy Meter

id=144, name=Pulse Meter

id=145, name=Water Meter

id=146, name=Gas Meter

id=147, name=Binary Switch

id=148, name=Binary Alarm

id=149, name=Aux Alarm

id=150, name=CO2 Alarm

id=151, name=CO Alarm

id=152, name=Freeze Alarm

id=153, name=Glass Break Alarm

id=154, name=Heat Alarm

id=155, name=Motion Sensor

id=156, name=Smoke Alarm

id=157, name=Tamper Alarm

id=158, name=Tilt Alarm

id=159, name=Water Alarm

id=160, name=Door/Window Alarm

id=161, name=Test Alarm

id=162, name=Low Battery Alarm

id=163, name=CO End Of Life Alarm

id=164, name=Malfunction Alarm

id=165, name=Heartbeat

id=166, name=Overheat Alarm

id=167, name=Rapid Temp Rise Alarm

id=168, name=Underheat Alarm

id=169, name=Leak Detected Alarm

id=170, name=Level Drop Alarm

id=171, name=Replace Filter Alarm

id=172, name=Intrusion Alarm

id=173, name=Tamper Code Alarm

id=174, name=Hardware Failure Alarm

id=175, name=Software Failure Alarm

id=176, name=Contact Police Alarm

id=177, name=Contact Fire Alarm

id=178, name=Contact Medical Alarm

id=179, name=Wakeup Alarm

id=180, name=Timer

id=181, name=Power Management

id=182, name=Appliance

id=183, name=Home Health

id=184, name=Barrier

 

log file:

 

[main] INFO com.bwssystems.HABridge.HABridge - HA Bridge (v0.4.10) starting setup....

[main] INFO com.bwssystems.HABridge.devicemanagmeent.DeviceResource - HABridge device management service started.... 

[Thread-0] INFO spark.webserver.SparkServer - == Spark has ignited ...

[Thread-0] INFO spark.webserver.SparkServer - >> Listening on 0.0.0.0:8081

2015-11-03 13:25:52.580:INFO:oejs.Server:Thread-0: jetty-9.0.z-SNAPSHOT

2015-11-03 13:25:53.428:INFO:oejs.ServerConnector:Thread-0: Started ServerConnector@188115a{HTTP/1.1}{0.0.0.0:8081}

[main] INFO com.bwssystems.HABridge.hue.HueMulator - Hue emulator service started....

[main] INFO com.bwssystems.HABridge.upnp.UpnpSettingsResource - Hue description service started....

[main] INFO com.bwssystems.HABridge.upnp.UpnpListener - UPNP Discovery Listener starting....

[main] INFO com.bwssystems.HABridge.upnp.UpnpListener - UPNP Discovery Listener running and ready....

[qtp15447091-17] ERROR com.bwssystems.HABridge.hue.HueMulator - Error on calling url to change device state: http://192.168.1.87:8085/harmony/start

[qtp15447091-21]

[qtp15447091-21] ERROR com.bwssystems.HABridge.hue.HueMulator - Error on calling url to change device state: http://192.168.1.87:8085/harmony/start

[qtp15447091-21]

[qtp15447091-12] ERROR com.bwssystems.HABridge.hue.HueMulator - Error on calling url to change device state: http://192.168.1.87:8085/harmony/start

[qtp15447091-21]

[qtp15447091-21] ERROR com.bwssystems.HABridge.hue.HueMulator - Error on calling url to change device state: http://192.168.1.87:8085/harmony/start

[qtp15447091-21]

 

 

I also can also run test as the program closes with no error

 

Many Thanks 

 

Robert

 

 

Edited November 3, 2015 by bocarob

[barrygordon]

I need a copy of the AWS Config log (AWS_Config.log) located in the same directory as the executable aws_config.exe.  Send it to me at my email address barry@the-gordons.net 

[blueman2]

Sorry Barry and Blueman2

 

... Here is the thing.... Responce for the activities is eratic most of the time Alexa responce is it did not find the device but every once in a while it does turn ON the activity.  What it always does is turn Off the activity. ...

 

I too have had a lot of issues with Alexa not clearly understanding my ON command when referring to Harmony events.  She hears my command, says "OK", but when I look at History in the Echo web page, she thinks I said OFF.  That is why nothing appears to happen.  And why OFF seems to always work.   Apparently, the Echo is 'OFF' happy.  I have found that when I stand closer to the Echo, it works 100%.  

 

But it sounds like your issue is different in that she does not find the device at all?  

 

Let me do some more testing on my system today.  I was using my own custom created commands for harmony devices but have now completely migrated them to commands created by Barry's tool.  I will see if I can duplicate your issue.  

Edited November 3, 2015 by blueman2

[barrygordon]

I have just updated the AWS_Configuration system on my web site to version 4.0.10. This update is mostly bug fixes.  If you had any errors or issues running on version in the 4.0.x series before 4.0.10 I strongly suggest that you upgrade to 4.0.10. If the errors are still there send me a log file (Aws_Config.log)

 

If you do not have a Harmony hub do not use the ini file that is in the install package of version 4.0.10 as it turns on the Harmony hub logic. The Harmony hub logic is turned on the ini file by the presence of a section named "[Harmony Hub]" and the line items within that section.  If you are sending me a log file (AWS_config.log) please ensure  that the debug option is the system section says "All" (without the quotes).

 

Barry