Export sort / filter

[gregoryx]

I read the posts. I know export sort/filter has been requested. And responded to that this is still a new product. And that UD doesn't want to do something unique for Elk or Insteon since there are other protocols and products. I read all that. I get it... sort of.

 

Still... ISY is creating that silly A01 nomenclature in the ELK-ID and ELK-ID5 fields. And the order is honked up. What order does it follow? The order things were entered or created in ISY? Why not do two simple things for the export:

1- make it ordered based on alphabetical? That way, we can affect it with letters and numbers at least;

2- make a set-able point (like G01 or whatever we choose) that the scenes start to export at - again, in alphabetical order - so that they don't change.

 

Anyone using Elk rules for this stuff is bound to run into a major PITA pretty quick if anything changes.

 

Do-able fixes?

 

If not, how 'bout some way to save the XML from Excel so I can at least tune it in a readable format, then save it for import? Excel 2007 does a nice job making the XML readable; but it doesn't know how to save it back to the appropriate format.

 

Thanks for considering!

[Michel Kohanim]

Hello gregoryx,

 

The silly nomenclature you are referring to is precisely how ELK represents lighting devices in M1. Therefore, we cannot do anything about the nomenclature (which is X10 based).

 

As far as sorting, we had to come up with an algorithm that converts node indeces to ELK X10 and back. As such, any type of sorting, cripples the algorithm.

 

The ultimate solution would be to have a configurable mapping table to translate between ELK X10 based nomenclature and ISY nodes. This would basically slow ELK processing since every request and response have to go through a mapping/translation routine (right now, it's completely mathematical). But, this is what we envision for the future.

 

With kind regards,

Michel

 

I read the posts. I know export sort/filter has been requested. And responded to that this is still a new product. And that UD doesn't want to do something unique for Elk or Insteon since there are other protocols and products. I read all that. I get it... sort of.

 

Still... ISY is creating that silly A01 nomenclature in the ELK-ID and ELK-ID5 fields. And the order is honked up. What order does it follow? The order things were entered or created in ISY? Why not do two simple things for the export:

1- make it ordered based on alphabetical? That way, we can affect it with letters and numbers at least;

2- make a set-able point (like G01 or whatever we choose) that the scenes start to export at - again, in alphabetical order - so that they don't change.

 

Anyone using Elk rules for this stuff is bound to run into a major PITA pretty quick if anything changes.

 

Do-able fixes?

 

If not, how 'bout some way to save the XML from Excel so I can at least tune it in a readable format, then save it for import? Excel 2007 does a nice job making the XML readable; but it doesn't know how to save it back to the appropriate format.

 

Thanks for considering!

[gregoryx]

Thanks for replying - and so fast - Michel.

 

I recognize the X10 nomenclature. I still think it's a very silly thing to use as the index for a modern lighting system. I meant it as no shot at you; I know it was Elk's choice; I've thought it was silly since first dealing with it when Elk first started supporting Insteon. It's still silly.

 

I understand your explanation of how they would have to map it if they chose to do something different... and the processing it would take. Frankly, I think they'd have been better off inventing their own addressing rather than look like X10 because it causes conflicts when one is using actual X10 and another lighting protocol... but that's their thing.

 

So... back to the part that pertains to UD/ISY: help me understand why ISY does not export in any logical order OR have an option to run the table / translation on the export before sending it to Elk. A mapping routine could be built by UD and it could even be a separate program than the ISY program. Have you seen the mess the current method creates? 200 lighting entries get exported in apparently random order. If you can't change it IN ISY, then what about an option to change it before it goes into Elk?

 

I've found a couple of XML programs that handle the file nicely. I can edit it no problem; but I don't have a way to do the logic to re-sort / re-name / re-number against some other table. I'm no programmer.

 

Is the request making sense? Is this stuff that no-one else needs? If not, how are they handling lights in the Elk? Just not much in use? I can see that happening, but it seems a bummer when we're this close.

 

On a happy note, I love being able to easily integrate the two. Don't take me wrong: the ISY has my Elk and Insteon working better together than any time in the last two years (or whatever it's been since I first put Insteon on the Elk).

[Michel Kohanim]

Hello gregoryx,

 

Thanks so very much for the explanation and the feedback. Herein lies the problem with sorting in the current environment:

1. ISY assigns unique IDs for each node (device/scene) from 1 to 1024

2. ELK uses A[1-16] to P[1-16]

 

The easiest and most tactical solution without incurring a lot of change on either party was to have a mathematical model such that A1 = 0 (all on/off), and then A2 = 2, and so on and so forth.

 

As you can see, any type of sorting, will break this fragile and silly mapping.

 

We do envision that we will have a mapping within ISY that can map entries based on user input.

 

With kind regards,

Michel

 

Thanks for replying - and so fast - Michel.

 

I recognize the X10 nomenclature. I still think it's a very silly thing to use as the index for a modern lighting system. I meant it as no shot at you; I know it was Elk's choice; I've thought it was silly since first dealing with it when Elk first started supporting Insteon. It's still silly.

 

I understand your explanation of how they would have to map it if they chose to do something different... and the processing it would take. Frankly, I think they'd have been better off inventing their own addressing rather than look like X10 because it causes conflicts when one is using actual X10 and another lighting protocol... but that's their thing.

 

So... back to the part that pertains to UD/ISY: help me understand why ISY does not export in any logical order OR have an option to run the table / translation on the export before sending it to Elk. A mapping routine could be built by UD and it could even be a separate program than the ISY program. Have you seen the mess the current method creates? 200 lighting entries get exported in apparently random order. If you can't change it IN ISY, then what about an option to change it before it goes into Elk?

 

I've found a couple of XML programs that handle the file nicely. I can edit it no problem; but I don't have a way to do the logic to re-sort / re-name / re-number against some other table. I'm no programmer.

 

Is the request making sense? Is this stuff that no-one else needs? If not, how are they handling lights in the Elk? Just not much in use? I can see that happening, but it seems a bummer when we're this close.

 

On a happy note, I love being able to easily integrate the two. Don't take me wrong: the ISY has my Elk and Insteon working better together than any time in the last two years (or whatever it's been since I first put Insteon on the Elk).

[gregoryx]

Thanks very much for sharing the detail of the issue with me. I understand the challenge better now. I presume that the internal ID remains constant when one does an "exchange" of a device; hence the importance of the unique internal ID indexing the unique controller ID (and nothing else, such as the device ID).

 

Hmmm... three thoughts: 1) user access to determine internal number; or, 2) build a mapping table into the export to XML process; or 3) at least a "starting point" for the scenes so that they're not co-mingled with the devices.

 

 

1) can you give us access to change the internal mapping of the 0-1024? If we could do that, I could do what I used to do with the HomeSeer X10 model: reserve blocks for certain groups so that they remained in order or could more easily be re-ordered.

 

For example, I'd do something like:

1-512 for devices and 513-1024 for scenes

then reserve blocks as I see fit:

20 for the kitchen

60 for the outside area

20 for the living room

etc.

 

In HomeSeer, it was more like:

K for kitchen

L for livingroom

M, N, O for outside

etc.

 

 

or

2) I know it'd suck to have to add something to the XML output process... but could you spare the storage memory for another table of two fields and 1024 entries? Is this problem common to other lighting solutions you'll be supporting? and/or other controllers? If so, the problem will just get more annoying as people get larger networks of devices. It sounds like this is what you're planning to do eventually.

 

 

or (even temporarily)

3) put a "start of scene indices" marker that could come in at 512 or whatever (user settable or not - I don't care initially... not until it's done and then I'll complain about it ). That would be a helpful version of 1) at least.

 

 

Thanks so much for listening!

[Michel Kohanim]

Hello gregoryx,

 

My pleasure!

 

Unfortunately, the mapping is much more rudimentary and thus replacing devices "may" break the mapping. The index is a unique internal ID for our database. Very similar to a primary key in a database with the exception that we can reuse indeces when/if they become available.

 

Since I really do not want to have yet another tactical solution, I envision having a mapping table between ELK addresses and ISY node addresses.

In the code, we do have functions that retrieve nodes by address (i.e. 4 A5 B2 1, etc.). For scenes, the addresses are completely unique and random. So, all we need is that mapping table which, although not very hard, but it may have performance impact on the rest of the system. As such, we have to design it correctly.

 

Btw, the Vista file dialog is already done and should be available in the next release.

 

With kind regards,

Michel

[gregoryx]

Cool. Thanks for the detailed explanation. Good luck on the coding!