Back Fill Program - How To?

[Teken]

I wanted to know if there is any way or method to take a network resource and back fill missing data to a cloud service? In this case its to Smart Energy Groups (SEG) which allows the public to use the great site and push their various data to this cloud hosted service.

 

In this thread I detailed the steps of how to get my Autelis Bridge 1 Wire using the ISY Series Controller as a bridge to send its temperature / humidity readings to SEG.

 

My goal now is to back fill missing minutes, hours, days, weeks, of data from my system. I am just not clear how to go about this and thought the smart people on this forum would know how.

 

Essentially as I understand it each data packet is time stamped. Using the time stamp a person could ultimately reseed missing data (gap fill) for any stream in question.

 

The related thread is here: http://forum.universal-devices.com/topic/16095-autelis-bridge-pushing-data-to-smart-energy-groups-seg-guide/

[Xathros]

Seg will accept a CSV file with data easily from what I read.  As long as you have the data and format it correctly, no problem putting it into the charts.

 

Select a stream, click the exchange tab:

 

 

This is not something that would be done with a network resource.

 

-Xathros

Edited May 27, 2015 by Xathros

[Teken]

Yes I am familiar with doing the above. I was hoping some kind of program could be written to detect a drop in data transmission then invoke a network resource to back fill the gap in data points.

 

As an aside are you seeing any gaps in your 1 wire data in the charts? I am unsure if this is because the room it's monitoring is so stable there is no update to the state var and hence the gap?

 

 

Ideals are peaceful - History is violent

[Xathros]

Yes I am familiar with doing the above. I was hoping some kind of program could be written to detect a drop in data transmission then invoke a network resource to back fill the gap in data points.

 

As an aside are you seeing any gaps in your 1 wire data in the charts? I am unsure if this is because the room it's monitoring is so stable there is no update to the state var and hence the gap?

 

 

Ideals are peaceful - History is violent

 

The ISY could certainly post back dated data to SEG with a network resource assuming we can get date and time values that SEG will accept.  That said, I'm not sure how to detect the gap, determine whats missing, then look up and post the old data.  This is not an ideal thing to attempt with the ISY's event driven logic.

 

As for my setup, I'm not seeing any gaps in data however I am seeing bad (out of range - low values) values coming from my CAI Board on a regular basis.  Getting closer to pulling the trigger on the Autelis box.

 

-Xathros

Edited May 29, 2015 by Xathros

[Teken]

Not seeing any gaps in data however I am seeing bad (out of range - low values) values coming from my CAI Board on a regular basis.  Getting closer to pulling the trigger on the Autelis box.

 

-Xathros

 

Well, the Autelis Bridge has been a most excellent product for me and I know quite a few people bought it after they saw my review and install guide.

 

With the push of development I urged the maker to do it has become the cornerstone of my environmental monitoring. One thing to note about these 1 wire sensors is don't get caught buying the cheapest units you can find.

 

I have experienced a few units that came from another vendor and it caused random errors to a complete shut down of the system. I have ten more sensors enroute to replace the three junk ones I purchased last year.

 

Its important to note these out of band errors will not show up until a few moments or hours of actual use. If you decide to move forward on the Autelis Bridge please do reference me.

Edited May 29, 2015 by Teken

[Xathros]

One thing to note about these 1 wire sensors is don't get caught buying the cheapest units you can find.

 

I have experienced a few units that came from another vendor and it caused random errors to a complete shut down of the system. I have ten more sensors enroute to replace the three junk ones I purchased last year.

 

Its important to note these out of band errors will not show up until a few moments or hours of actual use. If you decide to move forward on the Autelis Bridge please do reference me.

You can count on it!

 

Could you provide a link to your sensor model and source?

 

-Xathros

[Teken]

You can count on it!

 

Could you provide a link to your sensor model and source?

 

-Xathros

 

I will send you a PM or indicate it here once I test them to be fully operational on my unit. I don't like to over generalize but the 1 wire sensors that seemed to give me problems are the round tip shaped units. Opposed to the flat tip with (matted finish) units I installed more than three years ago which are going strong.

 

I will post up a picture in a minute so you can see the difference and the wire colors. Which in themselves is no guarantee of quality by any stretch of the imagination.

Edited May 29, 2015 by Teken

[Teken]

These are the 1 wire sensors that have caused me untold grief and out of band readings.

 

Of the ten 3-4 have right out of the hole when enrolled either had insane high values or they took the entire 1 wire network down.

 

 

These units came with the red, black, white wires. Unlike the red, black, yellow that have worked perfectly fine since 2010.

 

 

This unit also had the rounded tip as seen here.

 

 

Flat tipped units working like a champ.

 

 

 

Ideals are peaceful - History is violent

Edited May 29, 2015 by Teken

[Teken]

I wanted to update this thread that Xathros has once again come to the rescue: http://forum.universal-devices.com/topic/16095-autelis-bridge-pushing-data-to-smart-energy-groups-seg-guide/

 

It should be noted this is not a back fill solution as it is a method to insure data continues to flow to SEG which avoids a gap in the charting data. If done properly there will not be a need to back fill the SEG data via the ISY Series Controller.