Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor GMC data processing #444

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

t-pi
Copy link
Contributor

@t-pi t-pi commented Feb 6, 2022

Refactor different functions for GMC processing into single one.

  • DISPLAYREFRESH --> HEARTBEAT as base processing interval (default 10s)
  • Other events snap to this heartbeat interval
  • Improved handling of future additions (e.g. Telegram signaling) as separate event in single location
  • Removed MINCOUNT as trigger for heartbeat.

@ThomasWaldmann
Copy link
Contributor

diff isn't easy to review, would've been better done in multiple commits with special focus in each.

@t-pi
Copy link
Contributor Author

t-pi commented Feb 6, 2022

Sorry, was struggling a bit to get it working - and did not want to commit some kind of intermediate failing status

@ThomasWaldmann
Copy link
Contributor

ThomasWaldmann commented Feb 6, 2022

the current code mix is not pretty (unrelated to this PR), maybe we could rather have something like this:

  • keep some global global gm counts + thp historical values
  • have some code that ONLY updates it with new data (not calling any other functions)
  • have some code that computes measurements / averages / min / max from the historical data (could be also kept in globals), not calling other functions
  • have some code that sends / alarm-checks measurements

we could do fancy graphical stuff if we had some arrays with gm counts, thp for the last N secs / mins / hours.

guess we'ld need some "ring buffers" to easily keep / update historic values in a limited memory.

@t-pi
Copy link
Contributor Author

t-pi commented Feb 6, 2022

Agree about the current code mix. But I guess that ship has sailed...

Regarding the variables, at the moment we keep the main counters in the main loop and with this PR the specific historic states as needed per tracked event in the process_GMC function - at least a little bit cleaner than before :)

IMO a more general approach to keep historic data reaches beyond this PR, as several things have to be considered: values to be kept, frequency, aggregation, memory to be spent - and not least why would we do this.

Here I was aiming to prepare a better handling of the Telegram (and other) messages which are not necessarily in sync with the sensor.community etc. updates.

@t-pi
Copy link
Contributor Author

t-pi commented Feb 10, 2022

@ThomasWaldmann would it be possible to merge this PR? I would like to rebase Telegram on it and add MQTT as well as BME680 AQI, too.
Or else, what needs to be done prior to a merge?
Thanks!

Copy link
Contributor

@ThomasWaldmann ThomasWaldmann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

well, as i already said:

i can't really review this, too many changes mixed together.
i commented some stuff i noticed.

maybe someone else can or you just make sure yourself that everything is behaving and computing correctly.

Comment on lines +125 to +144
struct GM_State {
unsigned long timestamp;
unsigned long counts;
unsigned long last_count_timestamp;
unsigned long hv_pulses;
};

// Events
const byte HEARTBEAT = 0;
const byte MEASUREMENT = 1;
const byte ONE_MINUTE = 2;

const int savedstates_count = 3;

static GM_State saved_state[savedstates_count];
long event_interval[savedstates_count] = {
HEARTBEAT_INTERVAL * 1000, // Basic heartbeat interval
MEASUREMENT_INTERVAL * 1000, // Send measurements to server interval
60 * 1000 // 60 sec
};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

somehow i find the names confusing.

Comment on lines -271 to 285
read_GMC(&gm_counts, &gm_count_timestamp, &gm_count_time_between);

read_THP(current_ms, &have_thp, &temperature, &humidity, &pressure);

read_hv(&hv_error, &hv_pulses);
set_status(STATUS_HV, hv_error ? ST_HV_ERROR : ST_HV_OK);

update_ble_status();
int wifi_status = update_wifi_status();
setup_ntp(wifi_status);

update_ble_status();

// do any other periodic updates for uplinks
poll_transmission();
read_hv(&hv_error, &hv_pulses);
set_status(STATUS_HV, hv_error ? ST_HV_ERROR : ST_HV_OK);

publish(current_ms, gm_counts, gm_count_timestamp, hv_pulses, temperature, humidity, pressure);
read_GMC(&gm_counts, &gm_count_timestamp, &gm_count_time_between);
read_THP(current_ms, &have_thp, &temperature, &humidity, &pressure);

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why did you change order here?

Comment on lines +238 to +239
default:
break;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

something missing here?


const int savedstates_count = 3;

static GM_State saved_state[savedstates_count];
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are the array members in a defined state now, like e.g. is it documented that this is all-zero?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants