Serial communication?

[rickerdo]

Any plans to add bi-directional communication via a serial port? The goal would be for the ISY to send status changes out and accept commands in through the serial port.

 

It's troublesome and overkill to have to use a protocol like SOAP (HTTP, authentication, libraries in a programming language) to send a simple on or off command that could have been easily sent via an ASCII command through a serial port. Just about every home automation product has a serial port for this function: receivers, alarm panels, A/V switchers, irrigation controllers, etc... Even something as simple as the Arduino has a bootloader that allows the USB (or serial) port to server two functions. The port is used for flashing the firmware, or when a serial connection is detected, the port becomes a serial port. I can't imaging that the ISY can't be configured to operate the same way.

[Michel Kohanim]

Hello rickerdo,

 

You do not have to use SOAP for simple functions. You can always use REST: http://forum.universal-devices.com/viewtopic.php?t=2252 .

 

We do not have any plans to support serial communications to ISY. ISY was designed and optimized for network operations and not serial communications.

 

With kind regards,

Michel

[rickerdo]

Hello rickerdo,

 

You do not have to use SOAP for simple functions. You can always use REST: http://forum.universal-devices.com/viewtopic.php?t=2252 .

 

We do not have any plans to support serial communications to ISY. ISY was designed and optimized for network operations and not serial communications.

 

With kind regards,

Michel

 

Michel,

 

I am aware of REST, but again, HTTP is a bit heavy for something as simple as

.

 

Don't get me wrong, I love the ISY and ease of link mgmt, programming, etc... Just seems a bit odd that a device with a serial port cannot be used for serial communications. Even a virtual com port (socket based) would be a nice compromise. If the ISY could act like an Ethernet based serial adapter and allow a virtual com port redirector on a PC to connect to it, then the ISY would "appear" to have a serial connection to the PC. This is similar to how the GC-100 and Lantronics X-ports work. Could something like that be done in lieu of a directly connected serial port?

 

Thanks.

[Michel Kohanim]

Hi rickerdo,

 

Unfortunately not since with HTTP we get security. If I understand it correctly, you want to be able to send raw data to ISY over a TCP connection. Obviously, this is doable but, unlike GC/LanTronics, ISY does a lot of other things that require security and without HTTP headers, it's very difficult to support security rules.

 

With kind regards,

Michel

[rickerdo]

Hi rickerdo,

 

Unfortunately not since with HTTP we get security. If I understand it correctly, you want to be able to send raw data to ISY over a TCP connection. Obviously, this is doable but, unlike GC/LanTronics, ISY does a lot of other things that require security and without HTTP headers, it's very difficult to support security rules.

 

With kind regards,

Michel

 

Michel,

 

I'm looking for a way of sending Insteon commands to the ISY via a serial port on a PC. The ISY should also be able to sent any status changes back to a PC via the same serial port. Since UDI will not support using the existing on-board serial port of the ISY for this purpose, I figured a virtual serial port on a PC that connects to the ISY via Ethernet would be a decent compromise. Basically it would be a socket on the ISY that listens on a given port (10001 is common for this sort of thing).

 

I realize the XPorts are capable of a lot more than what I mentioned, but it was just an example of how a virtual serial port on a PC can be used for sending and receiving ASCII based commands to/from a network enabled device. Writing code to communicate with a serial port is much easier than having to setup something like REST. It also allows for compatibility with software that does not support a network based protocol like REST.

 

Rick

[Michel Kohanim]

Hi Rick,

 

May I humbly ask why you would need serial communications to ISY? I understand SOAP is too much but REST is not that bad since you will be sending ASCII command with the addition of HTTP headers (which include authorization header).

 

You can also use telnet to send HTTP commands to ISY's HTTP port.

 

With kind regards,

Michel

[rickerdo]

Hi Rick,

 

May I humbly ask why you would need serial communications to ISY? I understand SOAP is too much but REST is not that bad since you will be sending ASCII command with the addition of HTTP headers (which include authorization header).

 

You can also use telnet to send HTTP commands to ISY's HTTP port.

 

With kind regards,

Michel

 

Michel,

 

It's more for ease of use with automation software. Every application I've looked at fully supports serial port communication. It's a bit of a standard and I really feel that it's the ISY's biggest weakness. Sure, one could write plugins, or shop around for something that supports the ISY out of the box, but that's either too complicated, or too few and far between.

 

Thanks,

Rick

[Michel Kohanim]

Hi Rick,

 

Thanks so very much for the feedback. Frankly, this is the first time we are given this requirement and I do understand where you are coming from.

 

Unfortunately we never designed ISY to be accessible serially so I cannot even estimate how long it would take and what's involved.

 

With kind regards,

Michel

[rickerdo]

Any chance the SeriaLinc will be fully supported? That would be a great compromise!

 

Thanks,

Rick

[Michel Kohanim]

Hi Rick,

 

Based on what I understand you want to accomplish, SeriaLinc will not be a good solution. There's basically no way to send a serial command to SeriaLinc, convert to INSTEON, receive by ISY and then do anything useful with it.

 

What is your automation software that you are trying to integrate with?

 

With kind regards,

Michel

[rickerdo]

Hi Rick,

 

Based on what I understand you want to accomplish, SeriaLinc will not be a good solution. There's basically no way to send a serial command to SeriaLinc, convert to INSTEON, receive by ISY and then do anything useful with it.

 

What is your automation software that you are trying to integrate with?

 

With kind regards,

Michel

 

Michel,

 

Unless I completely misread the docs for the SeriaLinc, this is exactly what it was designed for. It accepts ASCII commands from it's serial interface and converts those commands to Insteon commands based off of the uploaded CSV values. Insteon status changes are also received and sent out via RS-232 in ASCII based off of the uploaded CSV values. The only reason I don't own one yet is because the ISY will not accept commands from it to execute programs. Commands from the SeriaLinc can only be sent to devices that are directly linked to it. Perhaps this is what you meant when you said this cannot be done? If this is what you mean, I know that and it's why I'm asking for it to be considered for implementation.

 

I'm currently using Misterhouse. It's great with serial communications. Its Insteon support is rather cumbersome at best. Using it with the ISY is almost impossible without writing something network based. That gets back into the issues I mentioned in my first post. Again, a lot of heavy lifting for some simple commands.

 

I am dumbfounded by this request being so misunderstood. Based on the number of reads this posting has gotten, I would assume a lot of people are interested in this functionality. Again, just about every HA device is controllable via RS-232 (except the ISY). The Ethernet based interface is GREAT for mgmt/configuration of the ISY, but RS-232 should be available for commands and status changes. Keep in mind, not every controller is PC based and capable of running software. A lot of [embedded] controllers use simple ASCII commands for communication. I know this request isn't difficult to implement because I program micro-controllers myself. Adding RS-232 is trivial.

 

Thanks again for hearing me out!

 

Rick

[tpolito]

I too am interested in this. I posted a message a little while ago along these similar lines. I currently use HAL2000 to control my Insteon devices, Ocelot, and HAI Omni Alarm/Thermostats. It is a great product, but the Insteon support is lacking. I purchased an ISY to handle the Insteon portion of it (haven't taken it out of the box yet), but I want HAL to keep control of the Ocelot and Omni. I was hoping HAL could communicate with the ISY over a serial link as well to initate lighting scenes based on actions from my Ocelot or Omni. I would be ok with using http requests, but I don't know anything about them, and wouldn't even know where to begin. USing serial commands seems easier, but still more robust than using X10 to communicate between these devices and the ISY, as was suggested to me.

[Michel Kohanim]

Hello Rick,

 

I was under the impression that you wanted to control all facets of ISY with the serial interface. It seems that all you want to do is to control INSTOEN devices.

 

As far as this being trivial, that's quite subjective and does not take into account our architecture and business priorities:

 

As I mentioned before, ISY was designed as a network appliance. ISY is based on a multi-tasking operating system and we have optimized each task's stack specifically for the purposes it serves. HTTP stack takes 16K while serial task uses 2K. So, we would be losing 14K for something that is at best requested in less than 0.01% of all the other requested features. On top of that, we would have to add locks and semaphores for Serial task so that it would not cause any interference with everything else ISY is doing (processing programs and user requests via HTTP). Perhaps trivial from a coding perspective but not trivial with respect to testing and regression testing.

 

All this said, I think the SeriaLinc approach might be easier but even that depends on what you are after and how much development effort you are willing to put into it.

 

For instance, SerialLinc is like a PLM so it would have to be included in all the scenes for us to be able to figure out what happened (highly suspect because it never worked before). Furthermore, if you do anything to a scene, you would have to do all the accounting on your end to figure out the status of devices because they are not reported. And, if we can't figure out what happened, then ISY's stauts will be out of synch with your real device status.

 

So, if status feedback and ISY being in synch with your network is important, then I do not think SeriaLinc will do the job.

 

Is MisterHouse based on Linux? Does it support Java or does it have Java hooks? Perhaps we can get a MisterHouse installed and create a plugin for it.

 

I am so very sorry for being so difficult ...

 

With kind regards,

Michel

[io_guy]

I am dumbfounded by this request being so misunderstood. Based on the number of reads this posting has gotten, I would assume a lot of people are interested in this functionality. Again, just about every HA device is controllable via RS-232 (except the ISY). The Ethernet based interface is GREAT for mgmt/configuration of the ISY, but RS-232 should be available for commands and status changes. Keep in mind, not every controller is PC based and capable of running software. A lot of [embedded] controllers use simple ASCII commands for communication. I know this request isn't difficult to implement because I program micro-controllers myself. Adding RS-232 is trivial.

Rick

 

Rick,

I'm having a hard time understanding exactly what you're looking for here as well. Are you looking to basically have all insteon status change information broadcasted off the ISY's serial port?

 

Full two-way communication is definitely not trivial since it requires processing of retrieved data on both sides. Right now the ISY does none, but will eventually (from what I've read) handle this via a 2-way network module.

 

I have 7 serial devices that I currently link to my ISY. I use serial-ethernet servers to do this. The 4-port Qualtech unit can be found used on eBay pretty cheap sometimes.

These devices range from simply turning a stereo on/off, to interfacing with a DSC alarm system or Honeywell thermostat using custom software on a Windows computer.

I know this is not what you're asking for, but even with serial comm on both sides, you'd need to write code to handle event changes/control.

 

Can you give a more specific example of what you're looking to do?

There's been a lot of custom work done by people here in the forums, maybe someone has an acceptable solution.

 

Right now I run 5 custom Windows applications to interface with serial devices. I've always been debating writing a single more generic serial link program instead (runs on Windows and links serial devices to program flags or variables on the ISY via REST), maybe it could fit in here somewhere.

[rickerdo]

Michel,

 

Again, I appreciate your time. I hope that fact hasn't been lost in all of this. The ISY is an excellent product for what it does. I just feel it's missing that last bit of icing on the top. Perhaps more like the cherry?

 

Thanks for offering to look at Misterhouse for me. In all honesty that would only address one application for a small percentage of your user base. What happens when/if I or others shift to a different app? Let's face it, not everyone runs Windows for automation in order to use what's currently available for controlling the ISY. Yes, I use Linux. Let's not get into a flame war about Windows/Linux. The bottom line is that ISY users are forced into using Windows for controlling the ISY for anything outside of what the ISY can do. This eliminates all of the Mac (Indigo), Linux (Misterhouse or even basic scripting), and hardware based (Control4, Crestron, AMX, etc...) users. So from a business standpoint, this feature will only open more doors.

 

As for "On top of that, we would have to add locks and semaphores for Serial task so that it would not cause any interference with everything else ISY is doing", you could add a switch to enable just Serial, or just HTTP. The extra resources needed could be recovered by removing all of the other "gadgety" type of things like: Brultech Electricity Module, WeatherBug Module, A10/X10 Experimental Support, Portal Integration - Check-it.ca, Portal Integration - GreenNet.com. After all, the ISY is a lighting controller, not an automation controller. It's not robust enough to be a stand alone home automation controller. And yet it can only be integrated with a limited number of home automation controllers through proprietary interfaces. EVERY home automation app/controller supports custom serial interfacing using raw ASCII.

 

 

io_guy,

 

Think of something like:

PC -> ISY: PorchLight,On

PC -> ISY: MediaroomScene,On

PC -> ISY: WatchMovieProgram,RunThen

 

PC -> ISY: GetStatus,PorchLight

ISY -> PC: PorchLightStatus,On

 

PC -> ISY: GetStatus,MediaRoomLights

ISY -> PC: MediaRoomLights,50 (50 as in 50%)

 

My favorite:

ISY -> PC: GarageDoorOpenerButton1,Pressed (garage door receiver connected through EZIO)

 

Hope that helps to clarify.

 

Rick

[io_guy]

Think of something like:

PC -> ISY: PorchLight,On

PC -> ISY: MediaroomScene,On

PC -> ISY: WatchMovieProgram,RunThen

 

PC -> ISY: GetStatus,PorchLight

ISY -> PC: PorchLightStatus,On

 

PC -> ISY: GetStatus,MediaRoomLights

ISY -> PC: MediaRoomLights,50 (50 as in 50%)

 

My favorite:

ISY -> PC: GarageDoorOpenerButton1,Pressed (garage door receiver connected through EZIO)

Rick

 

It seems like you're talking about PC serial interface talking to the ISY, not various external devices.

This would be much easier to handle through simple REST commands, which could be initiated through any number of programming languages, automation programs, or even VBS.

 

I'm still a little confused on how Serial would make this easier.

Whether it's TCP, RS-232 or something else, you'd still need custom code to process events/commands.

[bpwwer]

Rick,

 

From my linux machine I issue commands like:

 

./isy_control -a -c

./isy_control -a "xx xx xx x" -c DON

 

This is a very simple C program that uses the REST interface to send commands to the ISY. I use it to turn an appliance link on/off from within a cron job.

 

It doesn't do any status reporting. Getting status back is a bit more challenging. You need either a separate thread listening for the response or would have to poll via the REST interface.

 

I'd have to clean it up a bit since it has my ISY username/password and IP address embedded in the code, but if it would help, I can post it.

[rickerdo]

Rick,

 

From my linux machine I issue commands like:

 

./isy_control -a -c

./isy_control -a "xx xx xx x" -c DON

 

This is a very simple C program that uses the REST interface to send commands to the ISY. I use it to turn an appliance link on/off from within a cron job.

 

It doesn't do any status reporting. Getting status back is a bit more challenging. You need either a separate thread listening for the response or would have to poll via the REST interface.

 

I'd have to clean it up a bit since it has my ISY username/password and IP address embedded in the code, but if it would help, I can post it.

 

Bob,

 

Thanks, but you've made my point for me. "It doesn't do any status reporting. Getting status back is a bit more challenging. You need either a separate thread listening for the response or would have to poll via the REST interface."

 

Let's compare...

Current solution:

Misterhouse(perl)->fork off to external command (and wait for command to finish)->C app to HTTP->IP stack (ARP, packetize data, route if needed, etc...)->Ethernet->ISY(IP stack of ISY, etc...)

 

My solution:

Misterhouse->Serial port->ISY

Homeseer->Serial port->ISY

Control4->Serial port->ISY

Indigo->Serial port->ISY

Arduino(home brew)->Serial port->ISY

AMX->Serial port->ISY

Crestron->Serial port->ISY

(insert anything beyond what you already know about here)->Serial port->ISY

ISY->Serial port of (insert anything beyond what you already know about here)

 

With RS-232, it's open and ANYTHING can communicate bidirectionally with the ISY. Again, because EVERY HA controller has built-in RS-232 capabilities.

 

My issue, that everyone seems to not get, is not whether anything else is possible. From a technical standpoint, just about anything is possible with enough code/additional hardware. My issue is more along the lines of flexibility and ease of use with as little overhead as possible. In a word... flexibility.

[Michel Kohanim]

Hi Rick,

 

Thank you for the detailed response.

 

First of all, I am not going to discuss how this can be done in our code base (with or without a switch, gadgets or no gadgets, etc.) since I do not think you have enough information about our code base and architecture to make such assessments/assertions/statements.

 

To me, at the end of the day, everything boils down to ROI: the level of confidence we have to generate revenue for a given project and the costs/efforts involved in development. As of right now, and based on our history for the past 5 years, I do not see any revenues generated by supporting this feature. As a matter of fact, I do think we are going to lose time/resources/focus for a case not congruent with our technology vision.

 

I am going to think a little more about this issue. Perhaps we could have a serial to ISY box.

 

Thanks so very much for your feedback.

 

With kind regards,

Michel

[bpwwer]

Rick,

 

I do understand what you're asking for and believe there would be some value in having a serial interface to the ISY. However, as Michel has stated, he doesn't see a business case for it at this time.

 

I was trying to point out that it may be possible to have alternatives. And I agree that creating a "glue" program is not the most efficient way to interface devices.

 

The ISY provides a network interface and an API for interacting with the device over that interface. I believe you can substitute "ethernet" for "serial port" in your list and it still holds true. You seem to be confusing the physical communication layer with the API. Even if the ISY did provide a RS-232 interface, it is still a complicated device and for other devices to communicate with it would require someone to write "driver" software. Could there be a "dumbed down" interface to the ISY? Yes, but it would probably look like the PLM interface. Even interfacing directly to a PLM is non-trivial.

 

Why do you assume that attaching to a device via RS-232 would have less overhead and be easier to use? That seems like an unproven assumption. I actually find it easier and more flexible to attach to network based devices these days. Dealing with the all the stop-bits and handshaking required to make RS-232 work is a pain.

 

One of the primary factors for the non-synchronous nature of the ISY is the Insteon protocol itself. Because the devices are not "hard wired" you can get out-of-sequence responses to commands. This wouldn't change if using an RS-232 interface to the ISY. The ISY could try to hide this from the attached clients, but that would require additional complexities in the ISY's software and probably end up providing a sub-optimal user experience.

 

It sounds like the best solution would be to have someone create an interface to the ISY for MisterHouse. I have not used MisterHouse and I'm not familiar with it's internals so I don't know what it would take to do this. I am willing to provide help though.

[rickerdo]

Michel,

 

Do you mean something a long the lines of this - http://www.elkproducts.com/_webapp_2981 ... _Converter ? I think it's just a re-branded http://www.gridconnect.com/net232.html which is just a nicely packaged Lantronix Xport (I know cause I took one apart and re-purposed the Xport from it).

 

Anyway, I completely understand the importance of ROI. If nothing else I hope I've provided some food for thought. Like I said, for me at least, it's the Achilles heel of the ISY. And I REALLY don't want to go back to a software based controller because the ISY is great for mgmt!

 

Your support is second to none. Keep up the good work!

[Michel Kohanim]

Hello Rick,

 

No, those devices are quite dumb; they convert serial stream to network. That's not what you want. What you want is something that you can develop code for and that can keep/maintain state information without which all of this would be meaningless. Kind of like a very cheap nano computer.

 

Thanks so very much for your kind words. Again, I am so very sorry that we cannot implement this feature at the moment.

 

With kind regards,

Michel

[Harold]

If I might throw my $0.02 in. I posted a while back about the limited appear to joe sixpack because of having to learn a lot of details about systems and a programming language that to Joe is not real intuitive. There are vastly more Joes than "other" and they buy home automation software packages. I believe the point here is if the isy could interface to the already present RS232 embedded in the control packages and/or specialized hardware sold with a package. There are an awful lot of these. And at least some of those packages have spent major effort on making Joe a friendly user interface. And using the support they already have for a serial port should make the support of an isy/Insteon network simpler and less expensive for them. These other devices and software are oriented toward powerline devices and would have to add Ethernet to utilize an isy. That is not really trivial.

 

I have not had the isy installed for some time and I am just now reinstalling. The process is reminding me of just how complex setting this thing up can be. I do not use DHCP (probably most Joes do). How do I find out/assign an IP address - serial interface (granted as currently implemented, not what is being asked for). Joes don't do IP addresses or finding one when they don't know where it is. Or removing a security certificate or clearing Java cache, and so on.

 

I have a Joe for a brother and know many others. If they were presented with programming an isy, their eyes would glaze over and they would buy something else. My brother is incapable of using your system. Trust me on this. I have been dealing with him for a very long time. I built him a CPM machine using a SBC our computer club designed. He never used it. Ever. Why? Because he simply could understand the concept of files and directories and moving between them. He could not master the task of replacing the 8" diskette in a drive when he wanted to run a program or access data. He wanted an Atari (or Commodore, I don't remember) because he went to a friend's house and "HE COULD USE IT). He now has a nice big Apple running Windows (I don't know why) and he still can't understand. His computer was a mess because he would download zip files of programs and never be able to figure how to find them on his PC, and if he happened to run across them he could not install anything from them. Utterly unable to understand that programs can't simply move or how any of that works. Re-purchased software he had downloaded because he did not understand he had them in files as zip, or worse self extracting compressed, or RAR. And he has an Insteon installation. Uses a package from someone and a serial interface to a relatively simplecontroller.

 

You might want to think about your ROI if your device could be sold with all of these additional deceives and software packages. You will immediately be salable with pretty much any of the existing hardware and software for home automation. And these are people who are spending probably big bucks on this software and specialized hardware. And you would have to nothing beyond implementing the serial port and publishing your specs. This could potentially provide a parallel or migration path for users of totally different automation components and Insteon. The possibilities are significant and would translate to isy sales.

 

My thinking is that it would greatly expand your market with what I have to believe is a relatively low investment. I said relative there. Absolute cost is of course a function of what you have to do in comparison to the costs of other software you are embedding, priorities, and budget. And I suspect that this new user base is probably less interested in some of the add-on modules so some memory should be available. You already have the physical port and enough software to hold a conversation with a terminal implemented.

 

Just a thought - wordy, but a thought.

[Michel Kohanim]

Hello Harold,

 

Thanks so very much for your most valuable feedback.

 

I totally understand what you are saying but we have to make a distinction between Joe who has a hard time configuring ISY and Joe the super developer/hacker. For your information, ISY has already been successfully integrated with many user friendlier automation devices using our existing developer kits. Here's a list to name a few:

1. HomeSeer

2. MainLobby

3. CQC

4. mControl

5. Microsoft Media Center

6. Crestron

 

And many others. So, I really do not see having a network API for Joe the Super Developer to be a barrier to entry.

 

As far as the other Joe, you are 100% correct and, although we very much wish to have as many customers as possible, but we cannot address the needs of every Joe. For instance, we are frequently asked for variables ... variables make things even much more complicated and difficult but they are NEEDED since ISY cannot go to the next level without it.

 

With kind regards,

Michel

[Harold]

OK