Releases v0.7.1
Notable Highlights
Lineage Visualization
Pipelines and Tasks, Flows and Jobs
Airflow Lineage
Editable Field Descriptions
Nested Schema Viz
Search Improvements
datahub CLI
Official PyPi packages
Production-quality Helm scripts
New Integrations
Officially-supported Sources: Airflow, AWS Glue, dbt, Druid, Superset, MongoDB, Oracle
Changelog
#2440 @dexter-mh-lee feat(k8s): Move helm charts out of contrib#2397 @gabe-lyons feat(lineage): implement support for datasets, charts and dashboards downstream lineage fetching in a generic way#2434 @adriaanslechten feat(ingest) LDAP groups ingestion#2438 @hsheth2 fix(ingest): use entrypoints lib instead of pkg_resources#2425 @gabe-lyons feat(ingest): adding superset ingestion source#2433 @topwebtek7 fix(react): fix lineage sidebar buttons#2436 @hsheth2 fix(ingest): support custom snowflake types#2419 @topwebtek7 feat(react): add dataJob, dataFlow entity pages, refactor with fragments#2418 @frsann Fix(search): fix datajob and dataflow search mappings#2429 @hsheth2 fix(ingest): fix chart type enum serialization and add tests for rest emitter#2431 @shirshanka docs: Update agenda for Apr 23 townhall#2427 @hsheth2 fix(ingest): ensure upstreams in airflow lineage emission are entities#2426 @hsheth2 fix(ingest): include database info for snowflake#2424 @hsheth2 feat: add s3 data platform and logo#2423 @topwebtek7 feat(react): schema visualization add support for nested structs#2422 @topwebtek7 fix(react): lineage sidebar buttons should refer to the selected entity#2421 @dexter-mh-lee fix(kafka-setup): Fix start script for kafka setup#2417 @topwebtek7 feat(react): update dataset entity default svg icon#2411 @thomasplarsson feature(ingestion): Adding the concept of transformers#2415 @dexter-mh-lee fix(k8s): Add credentials to kafka-setup job and clean up#2412 @hsheth2 feat(ingest): add Kafka-based emitter example#2413 @gabe-lyons fix(lineage): allow lineage viz to handle circular dependencies#2414 @dexter-mh-lee fix(kafka-setup): Add the correct context to the git workflow for pushing kafka-setup image#2403 @hsheth2 fix(ingest): bump avro-gen3#2406 @topwebtek7 feat(react): use default entity icon if lineageentity has no icon#2408 @hsheth2 fix(ingest): properly handle fieldDiscriminator with restli#2409 @hsheth2 fix(ingest): add sqlalchemy extra#2398 @G-nther feat(kafka-setup): add option for SSL and topic partition config via environment#2404 @dexter-mh-lee feat(k8s): add extraEnvs to setup jobs#2407 @topwebtek7 feat(react): add footer buttons in lineage sidebar#2405 @thomasplarsson feature(ingestion): Make origin/fabric_type configurable#2384 @topwebtek7 feat(react): add padding between tags and description on datasets profile page#2396 @gabe-lyons feat(sample): adding sample mces for dataflows and datajobs#2400 @hsheth2 fix(ingest): streamline codegen init methods#2382 @topwebtek7 feat(react): update schema table to have fixed description column, set line break with max description width#2402 @dexter-mh-lee fix: Fix env variable setup for kafka, mysql-setup docker containers#2401 @hsheth2 fix(ingest): add db name to postgres URNs#2393 @hsheth2 fix(ingest): enable mypy disallow_incomplete_defs and disallow_untyped_decorators#2395 @gabe-lyons fix(react): fix access to pictureLink in charts and dashboards#2399 @gabe-lyons fix(tags): check description existence on tags#2383 @topwebtek7 feat(react): fix long descriptions overflow issue in lineage side panel#2392 @hsheth2 refactor(ingest): update test harness to use a compose file per test#2391 @topwebtek7 feat(react): fix browse link of last breadcrumb linked to unknown page#2385 @dexter-mh-lee feat(mysql-setup): Add the ability to specify database name for mysql-setup#2389 @hsheth2 feat(ingest): add generic sqlalchemy source#2390 @dexter-mh-lee feat(k8s): Add ability to add service accounts to setup jobs#2387 @dexter-mh-lee fix(kafka-topic-convention): Fix DAOs that do not refer to TopicConvention#2386 @dexter-mh-lee feat(index): Add index naming convention for elasticsearch#2388 @hsheth2 fix(ingest): report correct version status in dev mode#2368 @hsheth2 feat(ingest): add Airflow lineage backend#2380 @OddCN fix(docs): fix config example for file sink#2362 @dexter-mh-lee feat(k8s): Update pods with correct probes and remove unnecessary dependencies#2372 @thomasplarsson fix(ingestion): dont crash on non-RecordSchema topics#2360 @hsheth2 docs(ingestion): remove outdated data-source-onboarding.md docs#2376 @topwebtek7 feat(react): hide Owned By label in card if no owners#2373 @shubham49 fix(react): ownership rendering#2377 @topwebtek7 feat(react): add null state indicator in user profile when no entities#2379 @topwebtek7 feat(react): update avatar to use initial if no image, refactor all avatars with custom one#2369 @gabe-lyons feat(lineage): support arbitrary entity types in lineage viz#2364 @thomasplarsson fix(ingestion): Support mapping from avro "boolean" and "map" types t…#2343 @thomasplarsson fix(ingestion): properly detect optional fields in avro schemas#2370 @topwebtek7 feat(react): add empty state UI for browse when no entities#2242 @frsann feat(datajob): Datajob graphql query#2367 @topwebtek7 feat(react): add dropdown menu links, menu styling, removed warnings#2365 @frsann chore(dependabot): Update pyyaml version#2366 @topwebtek7 feat(react): add icons on entities, updated styling in LineageViz#2351 @hsheth2 fix(ingest): add test for avro serialization and deserialization#2361 @hsheth2 feat(cli): Add support for checking docker memory usage#2358 @topwebtek7 feat(react): original description shows in edit modal even when the description has been updated#2357 @gabe-lyons feat(react): improving error logging on dataset entity#2356 @dexter-mh-lee fix(elasticsearch): Fix inconsistencies between documents and elasticsearch mappings#2359 @hsheth2 fix(ingest): support python3 -m datahub#2353 @hsheth2 chore(ingest): remove unused plugin_requirements.txt file#2352 @hsheth2 fix(ingest): bump pybigquery version#2350 @hsheth2 fix(ingest): support datahub --version#2349 @gabe-lyons feat(lineage): improve lineage re-focus experience#2341 @frsann feat(tags): Add tag graph builder#2348 @jjoyce0510 fix(Ember App): Allow ember build (disabled by default)#2345 @hsheth2 fix(cli): add --verbose flag for datahub check plugins#2346 @gabe-lyons fix(lineage): add upstream arrows back in#2347 @hsheth2 feat(ingest): add Oracle db support#2336 @topwebtek7 feat(react): add description edit behavior along with modal#2340 @gabe-lyons feat(lineage): adding ghost edges indicating hidden dependencies#2331 @hsheth2 feat(ingest): start airflow integration + metadata builders#2339 @hsheth2 fix(ingest): add support for database and table patterns to glue source#2338 @hsheth2 fix(docker): remove restart: always from docker-compose for consistency#2335 @gabe-lyons feat(lineage): adding directionality to lineage edges to make the visualization more clear#2337 @gabe-lyons fix(lineage): fixing lineage layout bugs#2319 @amonkhouse feat(ingest): adding support for AWS Glue#2312 @shakti-garg feat(es-setup): add logic in elasticsearch setup to compare-and-update index if already exists#2333 @gabe-lyons feat(lineage): expandable lineage visualization for dataset <> dataset lineage#2332 @hsheth2 docs: add wolt logo to frontpage#2315 @grantatspothero feat(ingest): adds experimental support for ingesting Looker metadata#2330 @luck02 fix(test): dbt-manifest files#2329 @topwebtek7 feat(react): moving filter panel from modal to drawer#2328 @hsheth2 build: remove deprecated ember app from build#2327 @hsheth2 feat(ingest): verify dynamic registry types at runtime#2316 @joemirizio feat(ingest): dynamically register plugins#2325 @hsheth2 fix(ingest): remove outdated metadata-ingestion scripts#2313 @shakti-garg fix(k8s): make es-setup job parameters more contextual#2322 @gabe-lyons docs(theme): making yarn start instructions more explicit#2317 @hsheth2 doc: update slack links to https#2324 @frsann fix(datajob): Fix URI templates for datajob and dataflow#2320 @frsann fix(tags): Support creating tags with MCE#2323 @arunvasudevan fix(docs): Update metadata-serving.md#2318 @dexter-mh-lee fix(docker): Fix issue in gms start.sh#2321 @shirshanka docs: Update next townhall details, fixup links and misc docs#2251 @bernardino feat(Kubernetes): Add JMX exporter containers to all DataHub components#2308 @dexter-mh-lee fix(search): Fix styling for column match snippet#2302 @shakti-garg feat(k8s): Add k8s hook in datahub helm chart for setting up elasticsearch#2298 @dexter-mh-lee feat(docker): Add dockerfile for initializing an existing mysql server#2297 @shakti-garg feat(kafka-config): add variable KAFKA_CONSUMER_GROUP_ID to override group id value for kafka consumer#2303 @frsann feat(elasticsearch): Elasticsearch simple auth over https support#2306 @hsheth2 feat(ingest): support environment variables in recipes#2305 @hsheth2 fix(ingest): deal with breaking change from confluent_kafka#2304 @hsheth2 fix(cli): validate only required containers#2295 @hsheth2 feat: datahub check local-docker#2296 @gabe-lyons fix(docs): fix index builder link#2287 @gabe-lyons fix(react): improving layout of no data state#2290 @hsheth2 fix(ingest): resolve array serialization bug#2284 @pedro93 fix(ingest): Add druid-specific identification logic#2291 @dexter-mh-lee feat(healthcheck): Add Healthcheck servlet for GMS#2289 @hsheth2 feat(ingest): MongoDB ingestion source#2286 @dexter-mh-lee feat(search-by-field): Add the ability to search for field names#2288 @dexter-mh-lee feat(docker): Add healthchecks to datahub docker images#2283 @hsheth2 fix: remain as root until final datahub-frontend build stage#2267 @thomasplarsson feat(react): enable the react frontend to use SSL when talking with GMS#2277 @gabe-lyons feat(react): updating entity layouts to be friendly to smaller viewport sizes#2279 @hsheth2 fix(ingest): use custom pybigquery ref to get descriptions#2278 @hsheth2 docs(ingest): add python versions badge#2247 @luck02 feat(ingest): Add DBT source to metadata-ingestion#2276 @gabe-lyons feat(react): update search suggestion styling#2274 @gabe-lyons fix(react): fix randomizing suggestions on autocorrect#2250 @saxo-lalrishav feat : custom menu options in the main dropdown menu#2268 @hsheth2 fix(docker): use local datahub-frontend in dev.sh#2270 @gabe-lyons feat(react): show search suggestions#2269 @hsheth2 docs: update readme for v0.7.0 release#2264 @pedro93 feat(react): add druid logo#2266 @gabe-lyons fix: fixing tag formatting issues
You can’t perform that action at this time.
