Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"In Development" Flag #2

Closed
pnadolny13 opened this issue Nov 8, 2021 · 22 comments
Closed

"In Development" Flag #2

pnadolny13 opened this issue Nov 8, 2021 · 22 comments

Comments

@pnadolny13
Copy link
Contributor

Theres already this issue to document quality expectations of connectors being migrated to MeltanoLabs #1 but we also need to document a process for connectors that were developed in MeltanoLabs (right now theres a lot of these because of tap-toberfest).

There should be a way to tell users that new tap/target is not production ready yet even though its hosted in MeltanoLabs. From an outside perspective users probably assume that connectors in this repo have been given our stamp of approval but many of the new connectors developed in MeltanoLabs are still experiemental and under development which could be misleading.

There needs to be a way for us to flag that a new connector is beta/experimental/in development/etc. and that users should help raise bugs to get it to a production state.

Right now theres a few connectors that use the convention of adding a note to the description but we should have a consistent way to do this and document it somewhere so users are aware of it.

https://github.com/MeltanoLabs/target-yaml
https://github.com/MeltanoLabs/tap-google-analytics

Action Items:

@pnadolny13
Copy link
Contributor Author

pnadolny13 commented Nov 8, 2021

@tayloramurphy @aaronsteers What do you think? Do you have an opinion about what label we should give these - beta/experimental/under development. I'd vote for beta. Or maybe even a finer grain? like:

In Development: Just created, still developing, single developer that might be pushing directly to main/master, low chance that it works off the shelf.
Beta: Functioning, PRs for changes, probably buggy but works.
None aka GA: runs in prod, hardened

@aaronsteers
Copy link

Not necessarily to put a vote in for the names yet, but I do I think "Beta" and "Under Development" should be different grains.

  • Unknown
  • Under Development
  • Beta
  • Stable

And perhaps separately or as requirements for "Stable":

  • CI Tests
  • PyPi / published status
  • ...

@pnadolny13
Copy link
Contributor Author

@aaronsteers I agree with that. What were you thinking for the Unknown bucket? I'm not sure we need that since everything in MeltanoLabs should have gone through some sort of review as it was added.

I'm good with the naming - Under Development/Beta/Stable

@aaronsteers
Copy link

@pnadolny13 - I think the benefit of Unknown bucket is just that it makes a built-in burndown list. I think functionally, it should probably carry the same warnings as "Under Development" and could be phased out once everything has been reviewed and/or after a certain amount of time has passed.

@tayloramurphy
Copy link
Contributor

@pnadolny13 I think we can make custom badges on shields.io. I don't think they would show up in the list, but we could have them as part of the readme.

@pnadolny13
Copy link
Contributor Author

@pnadolny13 - I think the benefit of Unknown bucket is just that it makes a built-in burndown list. I think functionally, it should probably carry the same warnings as "Under Development" and could be phased out once everything has been reviewed and/or after a certain amount of time has passed.

@aaronsteers ah ok that makes sense. So we can set everything to Unknown to start then work up the hierarchy from there to make sure theyre all appropriately labeled.

@pnadolny13
Copy link
Contributor Author

pnadolny13 commented Nov 10, 2021

@pnadolny13 I think we can make custom badges on shields.io. I don't think they would show up in the list, but we could have them as part of the readme.

@tayloramurphy yeah good idea. We can add those to the top of the READMEs. Looks like we can have something like this:

development_status
development_status
development_status
development_status

@aaronsteers
Copy link

@pnadolny13 - LOVE it! ❤️

@pnadolny13
Copy link
Contributor Author

pnadolny13 commented Nov 10, 2021

These are the statuses as I see it right now:

tap-github - Beta
tap-dbt - Beta
tap-stackexchange (default) - Beta
tap-athena (default) - UnderDevelopment
tap-google-analytics - Beta
tap-gitlab (default) - Beta
target-athena (default) - Beta
tap-slack - Unknown
target-sqlite (default) - Beta
target-yaml (default) - Beta
tap-csv (default) - Beta
tap-peloton (default) - Beta
target-csv - Beta
tap-sqlalchemy - Unknown

Let me know if you disagree with any of those.

@tayloramurphy
Copy link
Contributor

@pnadolny13 I think tap-dbt is pretty Stable. @edgarrmondragon thoughts?

tap-peloton is probably beta. it uses the SDK and relies on an working library underneath.

Also, I'd recommend adding spaces in between the words if we could. so Development Status: In Progress or something like that.

@edgarrmondragon
Copy link
Member

@pnadolny13 @tayloramurphy

I think tap-dbt is pretty Stable. @edgarrmondragon thoughts?

I think it's stable though it may be accumulating bugs 😬. I recall a user (Stephen Bailey, maybe?) saying the other day they identified a type mismatch in some streams.

@pnadolny13
Copy link
Contributor Author

@pnadolny13 I think tap-dbt is pretty Stable. @edgarrmondragon thoughts?

tap-peloton is probably beta. it uses the SDK and relies on an working library underneath.

Also, I'd recommend adding spaces in between the words if we could. so Development Status: In Progress or something like that.

@tayloramurphy sounds good I updated my previous comments to include spaces and your suggestions for tap-dbt and tap-peloton.

@edgarrmondragon
Copy link
Member

edgarrmondragon commented Nov 10, 2021

@pnadolny13 @tayloramurphy would it be helpful to have those statuses as metadata in the Hub? It could still be made into a badge through a JSON endpoint in the Hub and that way they'd be in-sync.

@pnadolny13
Copy link
Contributor Author

@edgarrmondragon interesting idea! Its sort of starting to remind me of https://gitlab.com/meltano/meltano/-/issues/2829 again. We have also have maintenance status (Unknown, Active, Unresponsive) which is kind of similar but still different.

@tayloramurphy I'm also hesitant to give anything in MeltanoLabs a "Stable" badge until we finish this issue and document what that means.

@edgarrmondragon
Copy link
Member

@pnadolny13 either that or if we want to allow the dev keep all that metadata in the repo, I think somewhere we talked about having a file like meltano.hub.yaml that gets scraped along with the other repo data.

@aaronsteers
Copy link

aaronsteers commented Nov 10, 2021

@pnadolny13 - Regarding your comment:

We have also have maintenance status (Unknown, Active, Unresponsive) which is kind of similar but still different.

I think perhaps we should consider merging "Maintenance Status" and "Development Status":

What if we expand "Maintenance Status" to something like this:

maintenance_status
maintenance_status
maintenance_status
maintenance_status
maintenance_status

cc @tayloramurphy, @edgarrmondragon

@tayloramurphy
Copy link
Contributor

@aaronsteers I like it 👍

@pnadolny13
Copy link
Contributor Author

@edgarrmondragon I hadnt heard about the meltano.hub.yaml, do you have an issue you can link me to? I also like the idea of having a hub endpoint for these repos to reference so the status is only stored in one place. Although we'd need to implement that new endpoint in the hub first.

@aaronsteers I like that break down! I can create an issue to update the hub definitions with those statuses and render them as badges instead of text ("Maintenance Status: Active") on the front end.

Unknown - maintenance_status
Active - maintenance_status
Unresponsive - maintenance_status

@edgarrmondragon
Copy link
Member

edgarrmondragon commented Nov 12, 2021

@pnadolny13

@edgarrmondragon I hadnt heard about the meltano.hub.yaml, do you have an issue you can link me to? I also like the idea of having a hub endpoint for these repos to reference so the status is only stored in one place. Although we'd need to implement that new endpoint in the hub first.

I couldn't the find exact mention of such file but there's a .singer with similar intentions discussed in https://gitlab.com/meltano/sdk/-/issues/255.

@aaronsteers
Copy link

@pnadolny13 - re:

I can create an issue to update the hub definitions with those statuses and render them as badges instead of text ("Maintenance Status: Active") on the front end.

I like this path forward. I think it's worth looking at other text renderings and evaluating them as shields also. Generally speaking, I think it's a nice touch to have consistent bade-like rendering on fields/metrics where it makes sense. (Probably it's possible to go "too far" with badges but I don't fee like we're there yet. 😅

@pnadolny13
Copy link
Contributor Author

@aaronsteers I created an issue for these two issue for maintenance status and other general badges.
https://gitlab.com/meltano/hub/-/issues/162
https://gitlab.com/meltano/hub/-/issues/163

@aaronsteers
Copy link

Nice! Thanks, @pnadolny13 !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants