- PRIME Public Health Data Infrastructure (PHDI)
General disclaimer This repository was created for use by CDC programs to collaborate on public health related projects in support of the CDC mission. GitHub is not hosted by the CDC, but is a third party website used by CDC and its partners to share information and collaborate on software. CDC use of GitHub does not imply an endorsement of any one particular service, product, or enterprise.
This repository is a part of the CDC/USDS PHDI project and has two components:
- PHDI Python Package
- PHDI containerized Building Blocks
The PHDI Python package contains source code for a platform to help public health authorities (PHAs) ingest and report on public health data. This platform is composed of Building Blocks, which are modular software tools that, when composed together, can improve data quality and reduce data cleaning workloads by providing analysis-ready data to downstream public health surveillance systems and other analytical and reporting applications.
PHDI contains:
- Our SDK — the Python library containing Building Block source code
- Containerized web services exposing Building Block functionality as HTTP endpoints
- Repository
- User guide (under Building Blocks)
- Cloud Starter Kit — Repositories that implement a complete cloud-based pipeline composed of Building Blocks
Current public health systems that digest, analyze, and respond to data are siloed. Lacking access to actionable data, our national, as well as state, local, and territorial infrastructure, isn’t pandemic-ready. Our objective is to help the CDC best support PHAs in moving towards a modern public health data infrastructure. See our public website for more details.
PHDI is a sibling project to PRIME ReportStream, which focuses on improving the delivery of COVID-19 test data to public health departments, and PRIME SimpleReport, which provides a better way for organizations and testing facilities to report COVID-19 rapid tests to public health departments.
In order to use the PHDI library, you need Python 3.9 or higher and pip python package manager (or any python package manager).
To install using pip:
pip install phdi
Here are the current tools that PHDI offers:
- Containerized Building Blocks
- Alerts - Provides the ability to send alerts via SMS, Slack, or Microsoft Teams
- FHIR Converter - Enables conversion of health data from legacy formats (e.g., HL7 version 2, CCDA) to FHIR, a standard for health care data exchange
- Data Ingestion - Includes the entire pipeline of Building Blocks below
- Harmonization - Standardizes input data (e.g., patient names and phone numbers) to streamline the process of cleaning data and improve data quality
- Geospatial - Provides a common interface for obtaining precise geographic locations based on street addresses from input data
- Linkage - Assigns a common identifier to patient records in order to link and deduplicate patient records seen across data contributors
- Transport - Offers functionality for reading and writing data from storage resources (e.g,. FHIR servers)
- Message Parser - Extracts desired fields from a given message
- Tabulation - Extracts data from a FHIR server, converts it to a tabular representation, and stores it to a user-defined tabular storage file type (e.g., Parquet or CSV)
- Record Linkage - Links new health care messages to existing records if a connection exists
- Validation - Checks whether health care messages are in the proper format and contain user-defined fields of interest
- Implementation Support - Resources to help users implement PHDI tools to manage their data and analysis workflows
The source code is hosted on GitHub at: https://github.com/CDCgov/phdi.
The latest released version is available at the Python Package Index (PyPI).
Python modules
pip install phdi
Containerized services
build from source
build Docker locally
pull down Docker images from GitHub
PHDI documentation is currently hosted on GitHub Pages: https://cdcgov.github.io/phdi/
There, you can find our:
- SDK API reference documentation
- User guide for containerized Building Blocks
We mapped the rootnames of the PHDI database to nicknames produced by the aggregation and synthesis of open source work from a number of projects. While we do not employ the packages and wrappers used by the various projects (merely their open source data), we wish to give credit to their various works building collections of nickname mappings. These projects are:
- Secure Enterprise Master Patient Index, based on OpenEMPI, conducted by Vanderbilt University
- Curated Nicknames, scraped from genealogy webpages and run by Old Dominion University Web Science and Digital Libraries Research Group
- Simple Public Domain Nickname Mappings, hand collected using various sources
- Lingua En Nickname, collected from a series of GenWeb projects
- diminutives.db, compiled via a nickname extract using Wikipedia and Wiktionary
This repository constitutes a work of the United States Government and is not subject to domestic copyright protection under 17 USC § 105. This repository is in the public domain within the United States, and copyright and related rights in the work worldwide are waived through the CC0 1.0 Universal public domain dedication. All contributions to this repository will be released under the CC0 dedication. By submitting a pull request you are agreeing to comply with this waiver of copyright interest.
This project is in the public domain within the United States, and copyright and related rights in the work worldwide are waived through the CC0 1.0 Universal public domain dedication. All contributions to this project will be released under the CC0 dedication. By submitting a pull request or issue, you are agreeing to comply with this waiver of copyright interest and acknowledge that you have no expectation of payment, unless pursuant to an existing contract or agreement.
This repository contains only non-sensitive, publicly available data and information. All material and community participation is covered by the Disclaimer and Code of Conduct. For more information about CDC's privacy policy, please visit http://www.cdc.gov/other/privacy.html.
Anyone is encouraged to contribute to the repository by forking and submitting a pull request. (If you are new to GitHub, you might start with a basic tutorial.) By contributing to this project, you grant a world-wide, royalty-free, perpetual, irrevocable, non-exclusive, transferable license to all users under the terms of the Apache Software License v2 or later.
All comments, messages, pull requests, and other submissions received through CDC including this GitHub page may be subject to applicable federal law, including but not limited to the Federal Records Act, and may be archived. Learn more at http://www.cdc.gov/other/privacy.html.
See CONTRIBUTING.md for more information.
This repository is not a source of government records, but is a copy to increase collaboration and collaborative potential. All government records will be published through the CDC web site.
Please refer to CDC's Template Repository for more information about contributing to this repository, public domain notices and disclaimers, and code of conduct.