REST Subscriptions

[tazman]

I might have jumped to soon but I moved all my Insteon product over to the Polisy. I'm using Housepanel with my ISY to display the status of everything in my house. It seems with Polisy Housepanel does not receive the updated state of the devices but I can still control the devices. Does anybody know if REST Subscriptions should be working the same? When I do the /rest/subscriptions it shows housepanel connected and marked yes for reusing. Anyone have any ideas?

[Techman]

The ISY to Polisy conversion is still in Alpha development so there's a good possibility that there's bugs and issues that need to be resolved.

[tazman]

6 minutes ago, Techman said:

The ISY to Polisy conversion is still in Alpha development so there's a good possibility that there's bugs and issues that need to be resolved.

I understand and I am just putting my experience out there. It seems the Insteon part is working well and the supporting features are what need to be worked on and I hope my observations help find some of the bugs to get a good finished product.

[MrBill]

11 minutes ago, tazman said:

I understand and I am just putting my experience out there. It seems the Insteon part is working well and the supporting features are what need to be worked on and I hope my observations help find some of the bugs to get a good finished product.

Normally I'd say @kewashi could probably identify the issue the fastest even if it's something to communicate to UDI rather than adapt housepanel, but he seems to have vanished https://github.com/kewashi Jan 26th 2021 is the last time he touched any code.

[tazman]

23 minutes ago, MrBill said:

Normally I'd say @kewashi could probably identify the issue the fastest even if it's something to communicate to UDI rather than adapt housepanel, but he seems to have vanished https://github.com/kewashi Jan 26th 2021 is the last time he touched any code.

Yes kewashi would be the man but for it to be working then I switched over to ISY on the Polisy and it does not work seems to me something is not right now or it might just be that it's different now and without kewashi changing something it will not work. I jus figured since we are in testing phase it is best for this to be known.

[Javi]

Hi @tazman,

The Polisy does accept both old and new SOAP bodies.  The client code for the UD Mobile apps is still using the original SOAP requests and is working.

I ran a test with web sockets (ws://192.168.1.160:8080/rest/subscribe) and it works on both ISY and Polisy. Is the port missing? If your running on the same box it may need localhost.

[tazman]

3 hours ago, Javi said:

Hi @tazman,

The Polisy does accept both old and new SOAP bodies.  The client code for the UD Mobile apps is still using the original SOAP requests and is working.

I ran a test with web sockets (ws://192.168.1.160:8080/rest/subscribe) and it works on both ISY and Polisy. Is the port missing? If your running on the same box it may need localhost.

Unfortunately I'm not fluent in these technologies but I managed to get my system going with help from this site. I can see that housepanel makes it's subscription and it says it's marked as reusing the connection. I can control the devices using housepanel but it does not update the state in housepanel when it is operated outside of housepanel after housepanel was started. Housepanel is running on a Pi and worked fine with ISY994. I appreciate everyone's help with this.

[Javi]

Does HP have PORT field?  I took a quick look at the source (hpserver.js lines 8119 and 8399) and it does not appear the PORT is injected.  ISY uses port 80 Polisy uses 8080 for local connections.  If there is not a port field try adding the IP address with the port i.e. 192.168.1.20:8080  where "192.168.1.20" is the IP of Polisy and ":8080" is the port.  If that does not work you may be able to hard code this somewhere below.

hpserver.js Line starting 8119

// if user provides hub access info, use it
                // for ISY hubs we know the endpoint as /rest so use it
                if ( body.hubType==="ISY" ) {
                    body.userAccess = body.clientId + ":" + body.clientSecret;
                    body.userEndpt = hub["hubHost"] + "/rest";
                    hub["userAccess"] = body.userAccess;
                    hub["userEndpt"] = body.userEndpt;
                }

hpserver.js Line staring 8399

  var hubhost = hub["hubEndpt"];
            if ( hubhost.startsWith("https://") ) {
                wshost = "wss://" + hubhost.substr(8);
            } else if ( hubhost.startsWith("http://") ) {
                wshost = "ws://" + hubhost.substr(7);
            }
        }

        // set up socket for ISY hub if one is there
        if ( wshost ) {
            var wsclient = new webSocketClient();
            var buff = Buffer.from(hub["hubAccess"]);
            var base64 = buff.toString('base64');
            var origin = "com.universal-devices.websockets.isy";
            var header = {"Authorization": "Basic " + base64, "Sec-WebSocket-Protocol": "ISYSUB",  
                        "Sec-WebSocket-Version": "13", "Origin": "com.universal-devices.websockets.isy"};
            wshost = wshost + "/subscribe";

 

[tazman]

@JaviI put the port number after the ip:8080 and it grabs the devices and I can add them and control them but they do not show in housepanel when they are operated by outside sources.

[Javi]

@tazman,

Below the lines quoted above are console.log error messages.  Can you open the the Chrome dev tools (google console.log) to see if an error is presented. Basically this is opening dev tools, clicking Console tab, then reload.

For process of elimination we have an Android dev app for Websocket logs, if you have an Android device it will log output for a WS connection, this will eliminate server side issue. 

[tazman]

7 hours ago, Javi said:

@tazman,

Below the lines quoted above are console.log error messages.  Can you open the the Chrome dev tools (google console.log) to see if an error is presented. Basically this is opening dev tools, clicking Console tab, then reload.

For process of elimination we have an Android dev app for Websocket logs, if you have an Android device it will log output for a WS connection, this will eliminate server side issue. 

Thank you @JaviI can check the Chrome log when I get done with work this afternoon.

I do have an android phone I would just need direction on what app you are referring to.

[Javi]

2 hours ago, tazman said:

Thank you @JaviI can check the Chrome log when I get done with work this afternoon.

I do have an android phone I would just need direction on what app you are referring to.

Added java source, apk build, and installation instructions for Sample Dev App to GitHub.

https://github.com/UniversalDevicesInc/UDWebSocketExample-Android

[tazman]

@JaviI was unable to install the app on my phone but I could try other phones if it will help.

I downloaded a new version of housepanel and started from scratch to make sure it was "clean". I put one of my Insteon devise back to ISY994 and the status works for that device I can control the one on Polisy but does not change status when operated from outside unless I refresh.

The housepane log had this in it when starting

V2.418 on 11/2/2021  6:50:16 PM  Connection failure to ISY socket:  Error: Expected a Sec-WebSocket-Protocol header.  wshost:  ws://192.168.0.30:8080/rest/subscribe  header:  {
  
  'Sec-WebSocket-Protocol': 'ISYSUB',
  'Sec-WebSocket-Version': '13',
  Origin: 'com.universal-devices.websockets.isy'

Then looking at the developer tool it flashed this error so quick I had to video it and take a screen shot from the video

Edited November 3, 2021 by tazman

[Javi]

13 hours ago, tazman said:

@JaviI was unable to install the app on my phone but I could try other phones if it will help.

 

Some file managers do not allow install or require a setting to allow install of non store apks. But based on your screen grab I do not think it is an issue with Polisy.  This was just to verify that the Polisy's websocket is working.

13 hours ago, tazman said:

 

The housepane log had this in it when starting

V2.418 on 11/2/2021  6:50:16 PM  Connection failure to ISY socket:  Error: Expected a Sec-WebSocket-Protocol header.  wshost:  ws://192.168.0.30:8080/rest/subscribe  header:  {
  [Redacted]
  'Sec-WebSocket-Protocol': 'ISYSUB',
  'Sec-WebSocket-Version': '13',
  Origin: 'com.universal-devices.websockets.isy'

Then looking at the developer tool it flashed this error so quick I had to video it and take a screen shot from the video

Looking at Console it looks as if HP is injecting an incorrect port as both 1380 and 3080 are listed and it does not match the HP log of 8080.  I searched his source and found both hard-coded and default 3080 port values but am unsure how HP is pulling 1380 although it may just be misrepresented in the log.  

[tazman]

Well thanks for trying @Javi I guess I will try @kewashi but he does not seem to be around and this is beyond my abilities. It just seems really weird that it works with 994 but not Polisy.

[Javi]

@tazman,

You could try hard coding the webSocketUrl  in housepanel.js. change line 495 

from:

wsSocket = new WebSocket(webSocketUrl);

to:

wsSocket = new WebSocket("192.168.0.24:8080");

 

This is right below the console message:

console.log("Creating webSocket for: ", webSocketUrl);

which is showing the incorrect port. If it works you will have a reproducible fix if someone wants to correct the error.

[tazman]

Thanks @Javi I will look into that but probably will not be until the weekend. I will post if I get it working.

[BamBamF16]

@tazman

Were you ever able to solve this?  I am getting the same websocket error with @kck softconsole.  I can control devices, but status is not correct.  Getting a header error as well.

 

Thanks, Jeff

[tazman]

3 minutes ago, bambamf16 said:

@tazman

Were you ever able to solve this?  I am getting the same websocket error with @kck softconsole.  I can control devices, but status is not correct.  Getting a header error as well.

 

Thanks, Jeff

No unfortunately and I don't have the technical ability to figure it out and it seems the person who wrote the program I was using is not around anymore.

[kck]

@Javi

While the fix may mean a change to the app (in my case the console) the issue is clearly caused by some change in how Polisy works versus how previous versions work.  So I need to find out what has changed about subscribing to the websocket stream in the newer versions.  Any help would be really appreciated since clearly the OP's app and my app work fine with the older versions.

[Javi]

Hi @kck,

Looking at the OP's browser logs from house panel it is showing that it is using an incorrect port "3080" instead of "8080". While we do not develop House Panel, I took time to looked at the code and found the log line corresponding to the browser log. So until someone makes the change in House Panel, I have to operate under the assumption that the log is correct, and there is an bug in the House panel program causing incorrect port injection with ports other than "80".

I believe PyISY is using websockets for Home Assistant and we have not had any issues reported with regards to Home Assistant connections. The open source test app referenced earlier in the tread also works on both ISY and Polisy.

Is the app you are using open source? Can you verify the port is correct?

[kck]

I am the author of this app.  It uses a common Python library (websocket-client) to subscribe to the stream.  It is trying to open 

ws://10.0.1.71:8080/rest/subscribe

so as I understand it that would be the correct port for Polisy.  This has always worked on the 994 but here it appears that there is something awry with the headers.  Authentication seems to be correct because a direct call to get the nodes from the Polisy gets good data - it is just subscribing to the websocket stream that seems to have  broken.  As I said, I'm fine having to adjust my code but at the moment I don't know what changed between previous version and this current stuff.

 

ETA: it is all open source and can be found at https://github.com/kevinkahn/softconsole/tree/master/hubs/isy in the isyeventmonitor.py module around line 475.

Edited February 2, 2022 by kck

[tazman]

Well with housepanel it gets the devices from ISY and the proper state. I can turn them on and of and if I refresh the browser the state changes properly. The only thing is when the state changes it is not pushed to housepanel.

[kck]

3 minutes ago, tazman said:

Well with housepanel it gets the devices from ISY and the proper state. I can turn them on and of and if I refresh the browser the state changes properly. The only thing is when the state changes it is not pushed to housepanel.

That's basically the same behavior that I see.  Any app of this sort (mine or apparently HP) wants to monitor the stream for device state changes because that way it sees the changes no matter their source.  On the other hand, to make a state change it uses a direct rest call.  So you get what we are seeing - actions work but the result isn't registered back at the app because it can't see the stream of events.  It sounds like in the case of HP the author gets the initial state of the devices by a direct query which is why a refresh of the app gets the correct data.  In my case, I don't do this because I know that the websocket stream will give me all that state info, so I just use the stream for the initial case as well as the updates.

[Javi]

Hi @kck, I'll take a look when I get some free time,  but if I had to guess I would say the library being used filters out the port. I've seen this many times with ws and mqtt libraries.

 

@tazman, this is because the subscription socket is not connected as shown in the logs. In your case this is due to incorrect port.

Here are the headers and code used in the example app which works with both ISY and Polisy:

String credentials = Credentials.basic(credential.userName, credential.password);
OkHttpClient client = new OkHttpClient();

String url = "ws://";
        url = url + credential.ipAddress;
        if (credential.port != null){
            url = url + ":" + credential.port;
        }
        url = url + "/rest/subscribe";
        Request request = new Request.Builder()
                .url(url)
                .header("Authorization", credentials)
                .addHeader("Sec-WebSocket-Protocol", "ISYSUB")
                .addHeader("Sec-WebSocket-Version", "13")
                .addHeader("Origin", "com.universal-devices.websockets.isy")
                .build();
       webSocket = client.newWebSocket(request, listener);