[AnomalyDetection] Add base classes and specifiable protocol

[shunping]

This is the first part of the code push for anomaly detection transform.

[shunping]

r: @damccorm

[github-actions]

Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control. If you'd like to restart, comment assign set of reviewers

[codecov]

Codecov Report

Attention: Patch coverage is 91.16022% with 16 lines in your changes missing coverage. Please review.

Project coverage is 59.12%. Comparing base (5aae10d) to head (bc6fa8f).
Report is 1 commits behind head on master.

Files with missing lines	Patch %	Lines
sdks/python/apache_beam/ml/anomaly/base.py	83.60%	10 Missing ⚠️
sdks/python/apache_beam/ml/anomaly/specifiable.py	95.00%	6 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##             master   #33845      +/-   ##
============================================
+ Coverage     59.08%   59.12%   +0.03%     
  Complexity     3237     3237              
============================================
  Files          1156     1158       +2     
  Lines        176907   177113     +206     
  Branches       3391     3391              
============================================
+ Hits         104532   104715     +183     
- Misses        69008    69031      +23     
  Partials       3367     3367              

Flag	Coverage Δ
python	81.25% <91.16%> (+0.02%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[damccorm]

Mostly LGTM, but had some questions. In general, docs would be really helpful here. I really like the concept

[damccorm]

Maybe I'm not understanding this well, but could we just do something a bit simpler like:

def new_getattr(self, name):
   if not self._initialized:
      original_init()
      self._initialized = True
   return orig_getattr(self, name)

if we keep track of the original getattr function in orig_getattr

[shunping]

The general idea is like this, but there are some edge cases we need to avoid. Otherwise, we will end up with an infinite loop.

[damccorm]

Why do we start with these registered? Would it be cleaner to just register them on import?

[shunping]

These are the accepted subspaces for the known specifiable, but not the actual specifiable classes. The purpose is to avoid using a global space for every specifiable classes.

I think they should be predefined before the true registration takes place. WDYT?

[damccorm]

The purpose is to avoid using a global space for every specifiable classes.

I think maybe this is the piece I'm missing - why is this helpful? Aka, why is it helpful to have a nested KNOWN_SPECIFIABLE[subspace][spec_type] dict instead of just KNOWN_SPECIFIABLE[spec_type]. I can't really think of scenarios this enables/makes easier

[shunping]

There is a concern about keeping everything in a global namespace from the design doc.
https://docs.google.com/document/d/1tE8lz9U_vjlNn2H7t-GRrs3vfhQ5UuCgWiHXCRHRPns/edit?disco=AAABaZ332MQ

I think the primary goal here is to avoid naming conflict, but I am open to any suggestion.

[damccorm]

Ok, I can live with this I think, lets just make sure:

a) the user has an easy path to create this object without specifying a subspace (e.g. the mainline case where conflicts are not really an issue)
b) it is really clear what a user should do if there's a conflict

[shunping]

(a) Yes, I think the subspace is transparent to users. Users don't need to specify subspace when they use the decorator to augment their class.

(b) When there is a conflict, users can choose to specify a different spec_type when calling the decorator. I will make sure I have added that clearly in the docstring of the decorator.

On a second thought, I think it may be helpful to turn KNOWN_SPECIFIABLE into a module private variable, so users will know they are not supposed to manage it directly.

[shunping]

@damccorm, I've added docstrings and refactored some code for readability. PTAL. Thanks!

[damccorm]

The purpose is to avoid using a global space for every specifiable classes.

I think maybe this is the piece I'm missing - why is this helpful? Aka, why is it helpful to have a nested KNOWN_SPECIFIABLE[subspace][spec_type] dict instead of just KNOWN_SPECIFIABLE[spec_type]. I can't really think of scenarios this enables/makes easier

[damccorm]

Thanks for the updates. I just have one more thought which includes an API change - other than that we can take this and add in things as needed in the future

[damccorm]

Thanks!