REST Subscriptions

[kck]

@javi

Appreciate you looking when you can.  Don't think it is the library suppressing the port because essentially the same code flow exists in my console for the case of a Home Assistant hub which also uses websocket streams for updates.  It runs its subscription on port 8123 which is what my code passes in to that type of hub and that works fine.  So it seems that an actual port number embedded in the url stream works fine to direct where the subscription happens.  My guess is that some other header requirement subtly changed which is what is rendering my call (and the one in the HousePanel) invalid.  I'm going to keep trying to isolate it if bambamf16 can get me some debug streams.  I, unfortunately, don't have a Polisy I can test against.  Thanks.

[Javi]

@kck,

I would not link this to the other issue as the OP has screenshots of chrome logs are showing an incorrect port for HP.  A quick search of the issues on python library may have a clue; due to the IPV6 address?

https://github.com/websocket-client/websocket-client/issues/775

[tazman]

3 hours ago, Javi said:

Hi @kck,

Looking at the OP's browser logs from house panel it is showing that it is using an incorrect port "3080" instead of "8080". While we do not develop House Panel, I took time to looked at the code and found the log line corresponding to the browser log. So until someone makes the change in House Panel, I have to operate under the assumption that the log is correct, and there is an bug in the House panel program causing incorrect port injection with ports other than "80".

I believe PyISY is using websockets for Home Assistant and we have not had any issues reported with regards to Home Assistant connections. The open source test app referenced earlier in the tread also works on both ISY and Polisy.

Is the app you are using open source? Can you verify the port is correct?

@Javithe port 3080 is what I use in a Web browser to connect to housepanel

[Javi]

On 11/3/2021 at 7:37 AM, Javi said:

Some file managers do not allow install or require a setting to allow install of non store apks. But based on your screen grab I do not think it is an issue with Polisy.  This was just to verify that the Polisy's websocket is working.

Looking at Console it looks as if HP is injecting an incorrect port as both 1380 and 3080 are listed and it does not match the HP log of 8080.  I searched his source and found both hard-coded and default 3080 port values but am unsure how HP is pulling 1380 although it may just be misrepresented in the log.  

Meant to say 1380, from your screen shot

[tazman]

This is what is in my hmoption.config file which lists the isy information

"hubs": [
   {
    "hubType": "ISY",
    "hubHost": "http://192.168.0.30:8080",
    "clientId": "editusername",
    "clientSecret": "editpassword",
    "userAccess": "editusername:editpassword",
    "userEndpt": "http://192.168.0.30:8080/rest",
    "hubName": "ISY",
    "hubId": "isy01",
    "hubAccess": "editusername:editpassword",
    "hubEndpt": "http://192.168.0.30:8080/rest",
    "hubTimer": "0"

Is there something I can change in here that might make it work?

[Javi]

1 minute ago, tazman said:

This is what is in my hmoption.config file which lists the isy information

"hubs": [
   {
    "hubType": "ISY",
    "hubHost": "http://192.168.0.30:8080",
    "clientId": "editusername",
    "clientSecret": "editpassword",
    "userAccess": "editusername:editpassword",
    "userEndpt": "http://192.168.0.30:8080/rest",
    "hubName": "ISY",
    "hubId": "isy01",
    "hubAccess": "editusername:editpassword",
    "hubEndpt": "http://192.168.0.30:8080/rest",
    "hubTimer": "0"

Is there something I can change in here that might make it work?

You would have to change the source as there is a bug causing 8080 to change to 1380.  When I was looking at the source there was code to change ports for some of the other supported hubs. This is likley catching 8080 and converting it to 1380.  You could hard code the web socket url as mentioned here: https://forum.universal-devices.com/topic/34002-rest-subscriptions/?do=findComment&comment=321014

 

 

[tazman]

Unfortunately that did not work @Javi and it made things weird that items in housepanel would not show a status change when I clicked on them until I refreshed the browser.

[tazman]

This is what it shows in the error log

V2.418 on 2/1/2022  6:33:23 PM  Connection failure to ISY socket:  Error: Expected a Sec-WebSocket-Protocol header.  wshost:  ws://192.168.0.30:8080/rest/subscribe  header:  {
  Authorization:
  'Sec-WebSocket-Protocol': 'ISYSUB',
  'Sec-WebSocket-Version': '13',
  Origin: 'com.universal-devices.websockets.isy'
}

[tazman]

I went into an older version of housepanel. The log file looks like this with the old 994

7/27/2020  4:08:52 PM  Loading  1  hubs.
7/27/2020  4:08:53 PM  HousePanel Node.js Server is running on port:  3080
7/27/2020  4:08:53 PM  webSocket Server is listening on port:  1380
7/27/2020  4:08:53 PM  Success connecting to ISY socket. Listening for messages...
7/27/2020  4:09:06 PM  
****************************************************************
Serving page at:  http://192.168.0.40:3080
****************************************************************
7/27/2020  4:09:06 PM  Displaying main HousePanel web page:  http://192.168.0.40:3080
7/27/2020  4:09:06 PM  
****************************************************************
Serving page at:  http://192.168.0.40:3080
****************************************************************
7/27/2020  4:09:06 PM  Displaying main HousePanel web page:  http://192.168.0.40:3080
7/27/2020  4:09:08 PM  Connection accepted. Client #0 host=::ffff:192.168.0.40  Client count:  1
Pushing client #0
7/27/2020  4:09:27 PM  Peer:  ::ffff:192.168.0.40  disconnected. for:  1001  desc:  Remote peer is going away

 

All I did was change the IP address for ISY to the Polisy one and I get this and have the same behavior of not updating devices state

 

2/1/2022  7:00:48 PM  Loading  1  hubs.
2/1/2022  7:00:49 PM  HousePanel Node.js Server is running on port:  3080
2/1/2022  7:00:49 PM  webSocket Server is listening on port:  1380
2/1/2022  7:00:49 PM  Connection failure to ISY socket:  Error: Expected a Sec-WebSocket-Protocol header.  wshost:  ws://192.168.0.30:8080/rest/subscribe  header:  {
  Authorization:
  'Sec-WebSocket-Protocol': 'ISYSUB',
  'Sec-WebSocket-Version': '13',
  Origin: 'com.universal-devices.websockets.isy'
}

[Javi]

 @tazman,  Thanks for being persistent.

All libraries I use can handle the error, but I could replicate in Postman.  It appears not all headers are returned, so removing the following header should fix the issue temporarily.

"'Sec-WebSocket-Protocol'", "ISYSUB"

 

[tazman]

@Javi I tried but I don't know what to change because it would not load with what I did. If it is something that will eventually work I can wait. I found ISYSUB 2 places in the hpserver.js file. If I just remove the firs bold it still worked the same with the below error

V2.418 on 2/1/2022  8:27:04 PM  Connection failure to ISY socket:  Error: Expected a Sec-WebSocket-Protocol header.  wshost:  ws://192.168.0.30:8080/rest/subscribe  header:  {

 

 var header = {"Authorization": "Basic " + base64, "Sec-WebSocket-Protocol": "ISYSUB",  
                        "Sec-WebSocket-Version": "13", "Origin": "com.universal-devices.websockets.isy"};
            wshost = wshost + "/subscribe";

and

      });

            wsclient.connect(wshost, "ISYSUB", origin, header);
        }

[Javi]

I'll report tomorrow. As for making the change yourself, it looks this would require finding the wsclient.connect function and removing the second parameter and any references.

[tazman]

Thank you @Javi I'm trying not to be to much of a pain that is why I left it go until others brought it back. I really don't know programing and I don't think UDI should have to support other people's software but I appreciate all you have done so far to help!

[kck]

@Javi

Your last exchange with tazman provided the pointer to fix my version of this issue.  I was specifying subprotocol ISYSUB because at some point in the past this was required.  My guess it that you guys moved to a different websocket library at your end that doesn't need or handle that subprotocol specification and that this resulted in the failure.  I deleted specifying the subprotocol and things are now working as they should.  At least this thread should provide informal documentation should anyone else trip over this, but you might want to stick a note someplace in the documentation that this may have changed.  Thanks for the help.

Kevin

[tazman]

@JaviBelow is the main area that looks to me it defines the websocket. Would you be able to tell me what to change to make it work?

 // make websocket connection to any ISY hub
    // unlike ST and HE below, communication from ISY happens over a real webSocket
    var wshost;
    for (var h in GLB.options.config["hubs"]) {
        var hub = GLB.options.config["hubs"][h];
        wshost = false;
        if ( hub["hubType"]==="ISY" && hub["hubEndpt"] && hub["hubAccess"] ) {

            var hubhost = hub["hubEndpt"];
            if ( hubhost.startsWith("https://") ) {
                wshost = "wss://" + hubhost.substr(8);
            } else if ( hubhost.startsWith("http://") ) {
                wshost = "ws://" + hubhost.substr(7);
            }
        }

        // set up socket for ISY hub if one is there
        if ( wshost ) {
            var wsclient = new webSocketClient();
            var buff = Buffer.from(hub["hubAccess"]);
            var base64 = buff.toString('base64');
            var origin = "com.universal-devices.websockets.isy";
            var header = {"Authorization": "Basic " + base64, "Sec-WebSocket-Protocol": "ISYSUB",  
                        "Sec-WebSocket-Version": "13", "Origin": "com.universal-devices.websockets.isy"};
            wshost = wshost + "/subscribe";

            wsclient.on("connectFailed", function(err) {
                console.log( (ddbg()), "Connection failure to ISY socket: ", err.toString(), " wshost: ", wshost, " header: ", header);
            });

            wsclient.on("connect", function(connection) {
                console.log( (ddbg()), "Success connecting to ISY socket. Listening for messages...");

                // handle incoming state messages from ISY
                // this will be ignored if the node isn't in our list
                connection.on("message", function(msg) {
                    if ( msg.type==="utf8" ) {
                        processIsyMessage(msg.utf8Data);
                    }
                });
            
                connection.on("error", function(err) {
                    console.log( (ddbg()), "Connection error to ISY socket: ", err.toString());
                });
            
                connection.on("close", function() {
                    console.log( (ddbg()), "Connection closed to ISY socket");
                });
            
            });

            wsclient.connect(wshost, "ISYSUB", origin, header);
        }
    }

}

 

[Javi]

Hi @tazman

this object:  var wsclient = new webSocketClient();

calls this function: wsclient.connect(wshost, "ISYSUB", origin, header);

so find where webSocketClient() is defined and it should have a function called connect

the connect function accepts the parameters (wshost, "ISYSUB", origin, header)

If you find the connect function post it and I'll take a look.

@kck ,  good to hear.  We are looking into why this was removed, so it may be restored.  I ran tests on both ISY and Polisy and the server (ISY) will work as intended without the sub protocol header on both platforms. So it all depends on if the client/ws-library is checking for a sub protocol. There is only one protocol, and we have no plans to add another. It should be safe to remove this header if the client or library is not checking for this value in the event it is returned in a later firmware update.

[tazman]

@Javi I'm not sure what you mean with the connect function but I don't see anything in the file I cut the information from and posted above. I'm going to attach 2 files that seem like they could be relevant and if they are not it I guess I will just go back to giving up.

websocket.js WebSocketConnection.js

[Javi]

@tazman,

Node.js is not my strong point so this is just a guess.

Remove the protocol here:

 var header = {"Authorization": "Basic " + base64, "Sec-WebSocket-Protocol": "ISYSUB",  
                        "Sec-WebSocket-Version": "13", "Origin": "com.universal-devices.websockets.isy"};

change to:

 var header = {"Authorization": "Basic " + base64, "Sec-WebSocket-Version": "13", "Origin": "com.universal-devices.websockets.isy"};

Then remove the protocol(s) here also

 wsclient.connect(wshost, "ISYSUB", origin, header);

This may require a couple of tries such as:

 wsclient.connect(wshost, "", origin, header);

or

 wsclient.connect(wshost, , origin, header);

or

 wsclient.connect(wshost, null, origin, header);

[tazman]

@Javi Your the man THANK YOU!! I appreciate the time you put in to help me. Just so others know if they come looking "wsclient.connect(wshost, null, origin, header); " was what made it work for me. I'm not back to where I was with houspanel I only have 2 Items working but the fact that they update makes me think I can get back to where I was before.

 

[tazman]

@JaviI hate to do this but it looks like IOP 5.4 made your fix stop working again! Any idea what changed?

[Javi]

1 minute ago, tazman said:

@JaviI hate to do this but it looks like IOP 5.4 made your fix stop working again! Any idea what changed?

I believe the missing protocol was fixed. So the original version should work...

[tazman]

@Javiyou are correct going back to the original version fixed it Thank You.