Code

Fix socket error: other side closed #20

Code

Merged

rael346 merged 14 commits intomainsandboxnu/major-scraper:mainfrom fix-socket-dropsandboxnu/major-scraper:fix-socket-dropCopy head branch name to clipboardMar 1, 2024

+8,601-1,718

Commits

Commits on Feb 15, 2024

• fix: new fetch html logic for socket closed error

  rael346

  committed
  dc32274

  Copy full SHA for dc32274

• refactor: new fetch html logic for scrape major link stage

  rael346

  committed
  66efc99

  Copy full SHA for 66efc99

• refactor: new fetch html logic for classify stage

  rael346

  committed
  e7a4409

  Copy full SHA for e7a4409

• feat: cli is the new frontend

  rael346

  committed
  092463a

  Copy full SHA for 092463a

Commits on Feb 21, 2024

• WIP: add tokenize and parse stage to new CLI

  rael346

  committed
  c0fb990

  Copy full SHA for c0fb990

Commits on Feb 22, 2024

• fix: account for dev url when scraping major links

  rael346

  committed
  e7e1b6e

  Copy full SHA for e7e1b6e

• fix: get concentration html locally in tokenize stage

  rael346

  committed
  c889973

  Copy full SHA for c889973

Commits on Feb 23, 2024

• refactor: cli is the new frontend part 2

  rael346

  committed
  5cc5491

  Copy full SHA for 5cc5491

• feat: scrape current year instead of changing constant

  rael346

  committed
  6a51169

  Copy full SHA for 6a51169

• refactor: remove unused code + renaming stuff for consistency

  rael346

  committed
  0088ce9

  Copy full SHA for 0088ce9

• test: fix tokenize test for business major with separate concentration

  rael346

  committed
  c477779

  Copy full SHA for c477779

• refactor: remove unused dependency

  rael346

  committed
  df10740

  Copy full SHA for df10740

• refactor: move major2 types to global type folder + change imports to absolute for consistency

  rael346

  committed
  cd1b95c

  Copy full SHA for cd1b95c

Commits on Feb 25, 2024

• refactor: abstract phase logging for consistency between phases

  rael346

  committed
  1b6c621

  Copy full SHA for 1b6c621