Skip to content

Scraper uses Raw Inmate Data to determine which Booking Ids to check for #444

@nwinklareth

Description

@nwinklareth

This is part of #395.

The first version of the Next NG version of the Scraper queries the V1 database to find out which booking_ids to scrape from the Sheriff's website. This dependency needs to be brokern. Currently the scraper needs to know the following set of bookings ids:

  • the set of active booking ids from the last scrape - this is the last known population
  • the set of potential booking ids for the days since the last scrape - this is the potential new booked inmates
  • the set of discharged booking ids for the last N days - this helps to find inmates that were marked as discharged, as no repsonse was received when they were scraped often due to network or system problems.
  • the set of not found booking ids for the last N days - this helps to find inmates that for one or reason or not were not found in earlier scrape, oftem due to network or system problems, howver may also be due to input issues at the Sheriff's site..

All of these booking ids can be determined from the raw Inmate data. For a discussion on how to do so is found on the Wiki page: [Notes On Working With Raw Inmate Data}(https://github.com/sc3/cookcountyjail/wiki/Notes-On-Working-With-Raw-Inmate-Data).

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions