Home

Welcome! Log in or Join us

Forums SEGmeter Aggregations stopped working?

Subscribe to Aggregations stopped working? 24 posts, 4 voices

Reply to topic
 

 
Icon_missing_medium
Feb 9, 2015 10:52pm
48 posts

Hi Sam,

 

My aggregations seem to have stopped a few days ago. If I check the raw plots the data is there. No-one else seems to have reported this, so maybe it is just me.

 

Can you please check and re-enable the aggregations on my power streams?

 

Thanks,

 

Grant

 

 
Teles2012img_5162_medium
Feb 10, 2015 11:22pm
19 posts

yeah - reporting has been real spotty for me as well.

I thought it was my system, but it appears that Sam is doing some heavy lifting on the server or ??

I kept rebooting my system, but after every reboot I only get 5 46 sec apart data points from my segmeter streams and then nothing unless I reboot again.

The segboxvtwo keeps reporting every 30minutes, so not sure where the issue(s) lie...

 

What are your specific symptoms?

 

 
Icon_missing_medium
Feb 11, 2015 1:02am
48 posts

I was seeing the raw data logged to SEG every 50 secs for wh, but the charts that show kwh at 30 min and daily intervals had not been showing anything since last Saturday (4 days ago). So the raw data is being captured as expected, but I suspect the aggregation job that produces the daily and 30 min summaries had stopped for some reason.

Aggregations now seem to have updated, but I'm now offline (different issue I haven't resolved yet).

 
Samotage_300_medium
Feb 11, 2015 1:04am
555 posts

Hey All,

There have been a bunch of big updates, and some issues seem to be in play.  I have an idea what they are and working on resolving them.

I will likely kick of a re-agg for the last two weeks for all streams when done.

Thanks for your patience!

Sam, @samotage

 
Tony1_medium
Feb 11, 2015 1:07am
169 posts

Sam looks like everybody is off line

 
Samotage_300_medium
Feb 11, 2015 1:32am
555 posts

Well, that sounded a little boring... so I fixed it!

Apologies there, a new environment was installed this morning, and it seems the API had some fail.

All should be better, let me know if not.  (Note queued up API requests are being run now)

Sam, @samotage

 
Teles2012img_5162_medium
Feb 11, 2015 8:00am
19 posts

I'm still having issues with my system reporting Sam.

segboxvtwo reports new data every 4 min as expected - Good

stream listing screen show incorrect "online" uptime for segboxvtwo - Strange.

stream listing screen show correct "online" uptime for segmeter - Good.

raw non-calculated streams only report the first x5  46sec tics after a reboot of the segbox - bad.
(I've stoped the rebooting as to keep any data that could be in nonV memory...)

calculated stream data is quite delayed (several hours) - bad.

 

Please post if you do any more updates - Thanks, Randy

 
Samotage_300_medium
Feb 11, 2015 8:31am
555 posts

Hey Randy,

Strange...  I will keep an eye on it.

Sam, @samotage

 
Teles2012img_5162_medium
Feb 11, 2015 6:12pm
19 posts

more data on strangeness:

raw non-calculated streams --- report the first x5  46sec tics after a reboot of the segbox - bad.
THEN repeat with x5 more 46sec data points again every 1.5hrs or so.
(ex: https://smartenergygroups.com/streams/10655
I've stoped the rebooting as to keep any data that could be in nonV memory...)

calculated streams are reporting x1 data point every 1.5hrs
(ex: https://smartenergygroups.com/streams/31321)

stream listing screen show incorrect "online" uptime for segboxvtwo - CONTINUED Strange.
stream listing screen does not show any "online" uptime for segmeter - UPDATED bad.
( https://smartenergygroups.com/streams )

 
Tony1_medium
Feb 17, 2015 4:51pm
169 posts

Sam SEG still having problems? The past couple of day you had a banner up stating some problems. Banner is gone, but I'm not seeing any updated data in one of my streams. The last time data was update was on Sunday, about 11:21 est. But data is being sent.

 

Aleerose Aleerose > Solar Energy True RMS

 
 
 
 

 

Point Date
Value
 
 
Sun, 15 February 2015 11:21:09
25.47
delete point
edit stream point
Sun, 15 February 2015 11:20:08
24.98
delete point
edit stream point
Sun, 15 February 2015 11:19:09
24.58
delete point
edit stream point
Sun, 15 February 2015 11:18:08
23.57
delete point
edit stream point
Sun, 15 February 2015 11:17:13
23.96
delete point
edit stream point
Sun, 15 February 2015 11:16:08
22.31
delete point
edit stream point
Sun, 15 February 2015 11:15:08
23.8
delete point
edit stream point
Sun, 15 February 2015 11:14:09
22.88
delete point
edit stream point
Sun, 15 February 2015 11:13:09
21.41
delete point
edit stream point
Sun, 15 February 2015 11:12:11
16.5
delete point
edit stream point
Sun, 15 February 2015 11:11:09
18.39
delete point
edit stream point
Sun, 15 February 2015 11:10:09
19.96
delete point
edit stream point
Sun, 15 February 2015 11:09:09
20.52
delete point
edit stream point
Sun, 15 February 2015 11:08:08
19.19
delete point
edit stream point
Sun, 15 February 2015 11:07:19
15.91
delete point
edit stream point
Sun, 15 February 2015 11:06:08
15.67
delete point
edit stream point
Sun, 15 February 2015 11:05:09
15.6
delete point
edit stream point
Sun, 15 February 2015 11:04:09
15.83
delete point
edit stream point
Sun, 15 February 2015 11:03:09
15.09
delete point
edit stream point
Sun, 15 February 2015 11:02:08
14.22
delete point
edit stream point
Sun, 15 February 2015 11:01:08
13.6
delete point
edit stream point
Sun, 15 February 2015 11:00:09
14.07
delete point
edit stream point
Sun, 15 February 2015 10:59:09
13.84
delete point
edit stream point
Sun, 15 February 2015 10:58:09
13.0
delete point
edit stream point
Sun, 15 February 2015 10:57:16
12.08
delete point
edit stream point
Sun, 15 February 2015 10:56:09
12.27
delete point
edit stream point
Sun, 15 February 2015 10:55:08
12.6
delete point
edit stream point
Sun, 15 February 2015 10:54:09
12.11
delete point
edit stream point
Sun, 15 February 2015 10:53:09
11.98
delete point
edit stream point
Sun, 15 February 2015 10:52:10
12.63
delete point
edit stream point
Sun, 15 February 2015 10:51:09
12.04
delete point
edit stream point
Sun, 15 February 2015 10:50:09
12.67
delete point
edit stream point
Sun, 15 February 2015 10:49:09
12.59
delete point
edit stream point
Sun, 15 February 2015 10:48:09
10.73
delete point
edit stream point
Sun, 15 February 2015 10:47:10
12.74
delete point
edit stream point
Sun, 15 February 2015 10:46:08
14.58
delete point
edit stream point
Sun, 15 February 2015 10:45:08
13.13
delete point
edit stream point
Sun, 15 February 2015 10:44:09
14.25
delete point
edit stream point
Sun, 15 February 2015 10:43:08
15.46
delete point
edit stream point
Sun, 15 February 2015 10:42:09
14.56
delete point
edit stream point
Sun, 15 February 2015 10:41:08
14.88
delete point
edit stream point
Sun, 15 February 2015 10:40:09
14.79
delete point
edit stream point
Sun, 15 February 2015 10:39:09
14.89
delete point
edit stream point
Sun, 15 February 2015 10:38:08
14.1
delete point
edit stream point
Sun, 15 February 2015 10:37:11
13.53
delete point
edit stream point
Sun, 15 February 2015 10:36:09
12.22
delete point
edit stream point
Sun, 15 February 2015 10:35:07
11.6
delete point
edit stream point
Sun, 15 February 2015 10:34:09
11.61
delete point
edit stream point
Sun, 15 February 2015 10:33:08
12.5
delete point
edit stream point
Sun, 15 February 2015 10:32:15
12.55
delete point
edit stream point
Sun, 15 February 2015 10:31:08
12.45
delete point
edit stream point
Sun, 15 February 2015 10:30:10
12.46
delete point
edit stream point
Sun, 15 February 2015 10:29:09
12.01
delete point
edit stream point
Sun, 15 February 2015 10:28:08
11.33
delete point
edit stream point
Sun, 15 February 2015 10:27:11
10.56
delete point
edit stream point
Sun, 15 February 2015 10:26:09
10.91
delete point
edit stream point
Sun, 15 February 2015 10:25:08
10.25
delete point
edit stream point
Sun, 15 February 2015 10:24:08
10.76
delete point
edit stream point
Sun, 15 February 2015 10:23:08
10.27
delete point
edit stream point
Sun, 15 February 2015 10:22:12
10.07
delete point
edit stream point
Sun, 15 February 2015 10:21:09
9.85
delete point
edit stream point
Sun, 15 February 2015 10:20:08
10.28
delete point
edit stream point
Sun, 15 February 2015 10:19:09
10.3
delete point
edit stream point
Sun, 15 February 2015 10:18:08
10.5
delete point
edit stream point
Sun, 15 February 2015 10:17:15
10.24
delete point
edit stream point
Sun, 15 February 2015 10:16:09
10.57
delete point
edit stream point
Sun, 15 February 2015 10:15:08
10.69
delete point
edit stream point
Sun, 15 February 2015 10:14:08
11.22
delete point
edit stream point
Sun, 15 February 2015 10:13:08
12.25
delete point
edit stream point
Sun, 15 February 2015 10:12:13
12.74
delete point
edit stream point
Sun, 15 February 2015 10:11:08
11.76
delete point
edit stream point
Sun, 15 February 2015 10:10:09
12.85
delete point
edit stream point
Sun, 15 February 2015 10:09:09
13.2
delete point
edit stream point
Sun, 15 February 2015 10:08:10
14.06
delete point
edit stream point
Sun, 15 February 2015 10:07:10
11.14
delete point
edit stream point
Sun, 15 February 2015 10:06:08
10.35
delete point
edit stream point
Sun, 15 February 2015 10:05:08
11.47
delete point
edit stream point
Sun, 15 February 2015 10:04:08
9.37
delete point
edit stream point
Sun, 15 February 2015 10:03:08
10.86
delete point
edit stream point
Sun, 15 February 2015 10:02:09
11.32
delete point
edit stream point
Sun, 15 February 2015 10:01:08
11.02
delete point
edit stream point
Sun, 15 February 2015 10:00:09
11.34
delete point
edit stream point
Sun, 15 February 2015 09:59:08
10.9
delete point
edit stream point
Sun, 15 February 2015 09:58:09
9.77
delete point
edit stream point
Sun, 15 February 2015 09:57:13
10.11
delete point
edit stream point
Sun, 15 February 2015 09:56:08
10.34
delete point
edit stream point
Sun, 15 February 2015 09:55:08
10.27
delete point
edit stream point
 
Samotage_300_medium
Feb 19, 2015 12:38am
555 posts

Hey Tony,

Things are pretty much back to normal now, which is a win.

I've just had a look at this stream for your solar true energy, and it seems that SEG isn't getting any data for it's stream, e_33  this would suggest why there is no data for this stream, whilst there is data for other streams on this device.

Let me know what you find out.

Sam, @samotage

 
Icon_missing_medium
Feb 19, 2015 2:03am
48 posts

My aggregations are still about a week behind - latest data is still showing as the 12th Feb 2015.

 
Samotage_300_medium
Feb 19, 2015 2:36am
555 posts

Hey Grant,

You seem to have had some dud API requests that held things back.  I cleared them for you (which you can do yourself by going to tools -> api trace) and all should be well now :D

Sam, @samotage

 
Tony1_medium
Feb 19, 2015 2:48am
169 posts

Sam, I chose the wrong weekend to do firmware upgrades on my GEM and Dashbox. The assumption on my part that it was a SEG issue because of your notice that was posted at the top of the dashboards. It seems that after the upgrades my p33 stream was not being sent after the upgrade. This stream is created via the Brultech devices when the channel is a green device, aka solar panel. I ran a discovery and it is now being sent out as p34. I have posted a question on the Brultech forum about this issue. I might be asking you to merge the data from stream p33 into the p34 stream to preserve the historical data once Brultech gets back to me.

 
Icon_missing_medium
Feb 19, 2015 3:57am
48 posts

thanks Sam.

 

I know I have an issue every now and then with the API but haven't been able to nail down where it is coming from yet. A power cycle usually fixes it, but not this time as other streams seems to be happily logging away.

 

Can you tell why the API requests were dud?

 

I didn't know the API trace could be used to clear issues. I'll have to check it out.

 
Samotage_300_medium
Feb 19, 2015 6:04am
555 posts

Some of them had zero content.

 
Icon_missing_medium
Feb 19, 2015 6:35am
48 posts

do you mean no payload? I would have liked to see that, and don't know how that was possible.

 

I presume the Clear All button on the trace is what you used to clear the blocking error. I thought it just cleared the display so that another trace could be taken.

Question for you: As I indicated, every so often I get my custom SEG 'offline' but the arduino is still looping through the code as the watchdog is not triggered, but the API post fails for some reason. Any chance you could log permanently the last good and bad API requests to make it easier to track why this happening. I can go for weeks sometimes before I see this, then all of a sudden it happens and a power cycle is the only way to fix it. I should check the API response code I suppose, but it still doesn't help me find the problem.

2nd Question: it may not be possible, but in the above situation, can I click a button on the SEG website and detect that in the SEG to trigger a reboot? What do I need to have on the SEG end for this to work? I am only running code on an arduino - I don't have a 'server' component outside the SEG (like in a WRT type router).

 

Thanks,

 
Samotage_300_medium
Feb 19, 2015 7:01am
555 posts

Hey Grant, 

What version of the Arduino codes are you running?  There was an early version that would run for 3 weeks or so then die, but this was fixed in March 2010 or so.

Sam.

 
Icon_missing_medium
Feb 19, 2015 10:46pm
48 posts

Sam,

My code is very custom, but from memory loosely based around v18 I think.

I am really just exploiting the API as I didn't use a lot of your code that caters for different current clamps.

My design is based on yours and also some influence from the UK OpenEnergyMonitor.

It is probably a string handling/memory issue I suspect, but I've never been able to catch the offending API request to see what is going on.

Thanks,

Grant

 

 
Samotage_300_medium
Feb 19, 2015 11:47pm
555 posts

Hey Grant,

1.8 should be ok.

Are you handling the millis() rollover?  The long int value resets at about the time period you mention.

Unfortunately, a lot of Arduino code debug needs to be done in the mind...

Sam, @samotage

 
Icon_missing_medium
Feb 26, 2015 4:25am
48 posts

Hi Sam,

Aggregations stopped again. I ran a trace for a few minutes then hit 'Clear All'. The API trace looked normal for the few minutes it traced, but aggregations now seem OK.

I would have thought Clear All just cleared the trace display, but it also seems to 'fix things up'. I can't figure out the connection between a possible blocking of the aggregations from a bad API call some time ago, to a more recent trace and clearing of the trace. If the aggregation can't proceed because of the bad API call some hours after the event, then I would like to know what is blocking it as apparently the aggregations process knows about it. Still trying to find my mysterious bug, so any chance of somehow reporting on the aggregation block and showing what it is?

Thanks,

Grant

 
Samotage_300_medium
Feb 26, 2015 4:52am
555 posts

The aggregator robot looks for non-complete API requests, which are automatically saved when they fail.  This prevents aggregation fails.

Sam, @samotage

 
Icon_missing_medium
Feb 26, 2015 10:55pm
48 posts

OK, given that the aggregation robot knows about the incomplete API and to not run aggregations, then can I ask for a new feature to notify and report on incomplete API requests (the data seems to be already captured and available). 

If the icomplete/erroneous request can be seen, maybe I can work out where my problem is.

Obviously the 'Clear All' button on the API trace does more than just clear the page of the captured trace information as it also seems to clear the incomplete API as well. Maybe some more help/tooltips here might also be useful to understanding this function.

Food for thought?

Thanks.

 
Tony1_medium
Feb 27, 2015 6:50pm
169 posts

Sam the folks from Brultech got back to me on the issue that I posted in this thread a week or so ago, They stated my issue was caused by "Likely a DashBox bug that was incorrectly labeling the channel number previously. Maybe you can get Sam to combine the streams to the p_34."

 

Would it be possible to migrate the old p and e 33 into the new p and e 34? I would like to preseve the historical data from the old p&e 33.

 

Thanks

 

Log in to reply to this topic

Forums SEGmeter Aggregations stopped working?

copyright © 2008 Smart Energy Groups pty ltd