Skip to content

Latest commit

 

History

History
58 lines (38 loc) · 3.61 KB

ETHICS.md

File metadata and controls

58 lines (38 loc) · 3.61 KB

Ethical Considerations in Web Scraping

Overview

This document outlines the ethical considerations involved in the web scraping activities related to the News Summarizer project and the measures taken to address them.

Table of Contents

Purpose of Data Collection

  • Educational Intent: The primary purpose of collecting data through web scraping in this project is for educational and research purposes. It demonstrates the practical application of web scraping techniques.
  • Value Addition: The project aims to provide users with easy access to the latest news headlines and summaries, potentially saving time and offering insights.

Compliance with Website Policies

  • Terms of Service: We have reviewed the BBC's Terms of Use to ensure that our scraping activities do not violate any terms.
  • Permissible Access: The scraper accesses only publicly available data that does not require authentication or bypass any access restrictions.

Respecting Robots.txt

  • Robots.txt File: The scraper checks the BBC's robots.txt file to identify which parts of the website are allowed to be scraped.
  • Compliance: The scraper is configured to avoid any directories or pages that are disallowed in the robots.txt file.

Collection Practices

  • Limiting Requests: The scraper is designed to minimize the number of requests sent to the server to avoid any potential disruption or overloading.
  • No Automated Data Harvesting: The script does not perform continuous or automated scraping over extended periods.
  • No Bypassing Security Measures: The scraper does not attempt to access password-protected areas, hidden content, or use methods to bypass security measures.

Data Handling and Privacy

  • No Personal Data Collection: The scraper does not collect any personal information or Personally Identifiable Information (PII) from users.
  • Secure Storage: Any data collected is stored securely and is not uploaded to public repositories. Sensitive data is included in the .gitignore file.
  • Anonymity: The scraper does not track or log any user interactions beyond the scope of the news data being collected.

Data Usage

  • Non-Commercial Use: The data collected is strictly for educational and research purposes and will not be used commercially or redistributed.
  • Transparency: Users of the project are informed about the purpose of data collection and how the data can be used.

Legal Considerations

  • Copyright Compliance: The scraper only collects brief excerpts (headlines and summaries) that are likely permissible under fair use for educational purposes.
  • Intellectual Property Rights: We acknowledge that the content belongs to the BBC, and we do not claim any ownership over the scraped data.
  • No Redistribution: The project does not redistribute the scraped content in any form that would infringe on intellectual property rights.

Conclusion

Ethical considerations are central to the implementation of this web scraping project. By adhering to website policies, respecting the robots.txt guidelines, and ensuring responsible data handling and usage, we aim to conduct our activities in an ethical and lawful manner.