Historical GTFS Occupancy Data Effort

Purpose

Expose historical passenger loading to transit riders in lieu of real-time passenger counts with the hopes of:

1. Highlighting particularly overcrowded trips
2. providing insights into when successive trips have capacity

Scope

This effort is divided into 4 tasks:

1. Analysis and preparation of historical data
2. Mapping of historical data to existing/current GTFS Trips
3. Incorporation of historical data into existing systems
4. Display of historical data in mobile applications

1. Analysis and Preparation

Discussions remains around the specifics of the data integration:

1. at what level will the data be aggregated two? Trip? Stop?

• Stop is preferred, agreed to start with this

2. on what interval will the data be updated?

• 6 month implied, will re-visit

3. How will this information be incorporated into Step 2.

• Read on

2. Mapping of historical data to existing/current GTFS Trips

This task encompasses transformation of the data from "Analysis and Preparation" into a standard format, with GTFS-ride being proposed. In doing so, the historical trips need to be mapped to the current trips such that the historical counts can be carried over.

3. Incorporation of data into existing systems

Several tasks need to occur here:

1. add support to onebusaway-gtfs for the proposed extensions/format above
2. add support to onebusaway-transit-data-federation-builder (bundle builder) to load the above data and store as part of the bundle
3. pass this new data through the onebusaway-transit-data-federation (transit data service) into the OneBusAway APIs and SIRI interfaces

4. Display of Historical Data

The OneBusAway mobile developer leads have been engaged in this effort and are willing to take on design and development implications of this effort, but are awaiting specification of the tasks above.

Implementation

1. Analysis and Preparation

• suggest a level of aggregation
• agree on csv format
• agree on update interval
• confirm owner of this task over long term

2. Mapping of historical data to existing/current GTFS Trips

• map above CSV to GTFS-ride
• specify which GTFS-ride features will be supported
• create module that loads CSV and existing agency data and adds GTFS-ride spec

3. Incorporation of data into existing systems

• Add GTFS-ride support to onebusaway-gtfs / onebusaway-gtfs-modules so that the data may be loaded
• Add GTFS-ride support to onebusaway-transit-data-federation-builder so that the data may be indexed
• Add GTFS-ride support to onebusaway-transit-data-federation and onebusaway-api-webapp so that the data may be accessed/exposed

4. Display

• provide anticipated API (OneBusAway/SIRI) to mobile developers
• facilitate follow on design sessions
• assist with technical problems as they arise
• create test environment for development
• create a plan for release/deploy

Specification

1. Analysis

CSV format: (tentative agreement to supply fields similar to trip_capacity.txt in combination with ridership.txt)

Summary:

• ridership.txt is closest to our intention
• if totals (not averages) are included, also need trip_capacity.txt to determine averages

2. Mapping of historical data to existing/current GTFS Trips

Merge Algorithm

* identify service type:  weekday/saturday/sunday/holiday/reduced service day
* identify which subset of these are supported
* for a given CSV trip
** match to a service type
** find route of given trip
** find closest start time to given trip
*** verify same number of stops in CSV and target route
*** verify stop lists are the same in CSV and target route
** assign CSV trip data to selected GTFS trip
** remove CSV trip from future consideration

3. Incorporation of data into existing systems

OBA API

EDIT - Draft for comment is in this Google Doc

Provide additional parameter and response item to trips-for-route API (also returned in any response that includes the <tripDetails/> element):

Request:

/api/where/trips-for-route/1_44.xml?key=<key>&includeEstimates=true 

Response: (both xml and json supported)

<tripId>1_12540399</tripId>
<serviceDate>1271401200000</serviceDate>
<frequency>...</frequency>
<status>...</status>
<schedule>...</schedule>
<estimatedRealtime>
  <stopOccupancies>
    <tripStopOccupancy>
      <stopId>1_18085</stopId>
      <occupancy>{"full" | "seatsAvailable" | "standingAvailable"}</occupancy>
    </tripStopOccupancy>
    <tripStopOccupancy>...</tripStopOccupancy>
  </stopOccupancies>
</estimatedRealtime>

SIRI API

Siri Element EstimatedTimetableDelivery is analogous to VehicleMonitoringDelivery and supported since SIRI v1.2:

Request:

/siri/estimated-timetable.json?key=<key>&OperatorRef=1&LineRef=4&StopRef=431
where any combination of OperatorRef, LineRef, and StopRef is allowed

Response: (both xml and json supported) Occupancy: {"full" | "seatsAvailable" | "standingAvailable" }

<Siri>
  <ServiceDelivery>
    <EstimatedTimetableDelivery version="1.3">
      <ResponseTimestamp>2001-12-17T09:30:47-05:00</ResponseTimestamp>
      <EstimatedJourneyVersionFrame>
        <EstimatedVehicleJourney>
          <LineRef>1_100224</LineRef>
          <PublishedLineName>44</PublishedLineName>
          <Monitored>true</Monitored>
          <!-- we could optionally set this flag below to indicate it is not a strong prediction -->
          <PredictionInaccurate>true</PredictionInaccurate><!-- Whether the prediction for the journey is considered to be of a useful accuracy or not. Default is false. -->
          <!-- this field below is optional, we could omit it as to not contradict the stop level occupancy -->
          <Occupancy>full</Occupancy> <!-- trip level occupancy -->
          <EstimatedCalls>
            <EstimatedCall>
            <StopPointRef>1_18085</StopPointRef>
            <Occupancy>full</Occupancy> <!-- stop level occupancy -->
            </EstimatedCall>
            <EstimatedCall>...</EstimatedCall>
          <EstimatedCalls>
        </EstimatedVehicleJourney>
      </EstimatedJourneyVersionFrame>
    </EstimatedTimetableDelivery>
  </ServiceDelivery>
</Siri>

4. Display of Historical Data

• TODO: discuss testing servers, integration needs, schedule