You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently, the CU stores its json results in sqlite or postgres. This is not optimal for large results especially when getting a list. Also when booting up a new CU and using it for the same processes, unless it is using postgres it will not have access to the previously cached results and will have to evaluate them.
Solution
Create 2 environment variables EVALUATION_RESULTS_DIR and EVALUATION_RESULTS_BUCKET they will default to not being set and if they are set the feature is enabled.
Create a hydrator worker modelled off of the evaluation worker. and compose it into ao-evaluation.js.
ao-evaluation.js will ask hydrator for eval results, hydrator will then use EVALUATION_RESULTS_DIR and EVALUATION_RESULTS_BUCKET to try and retrieve cached results. ao-evaluation will use these in output.
Current persistance in ao-evaluation will remain the same the hydrator will just hydrate and flush from the persistance in ao-evaluation.js
EVALUATION_RESULTS_BUCKET will be a remote storage system like s3.
hydrator will flush a single process or all processes (compose the single process function into the many one) upon receiving a signal.
The CU will signal the hydrator upon receiving its own signal to flush a single or many processes.
The text was updated successfully, but these errors were encountered:
Background
Currently, the CU stores its json results in sqlite or postgres. This is not optimal for large results especially when getting a list. Also when booting up a new CU and using it for the same processes, unless it is using postgres it will not have access to the previously cached results and will have to evaluate them.
Solution
The text was updated successfully, but these errors were encountered: