Data structures: mtldp.meta

mtldp.meta (repository mtldp-meta-utils) contains the data structures shared by every other part of MTLDP. They are deliberately plain: they store attributes and references between elements and offer only light helpers. Building, loading, converting and analysing them is the job of mtldp.utils and mtldp.preproc.

There is no top-level mtldp.meta namespace to import from; the three sub-packages are imported directly:

from mtldp.meta.TrafficNetwork import Network, SignalizedNetwork, Node, Link, Segment, Movement
from mtldp.meta.Trajectory import Trajectory, TrajectoryDict, Trip
from mtldp.meta.SPaT import HistoryRegionSPaT, HistoryControllerSPaT

Traffic network

Element hierarchy and identifiers

Nodes, segments, links and movements of a network.

A Network is a set of dictionaries keyed by element ID:

Element	ID format	Meaning
OsmWay	OSM way ID	A raw OpenStreetMap way; the input from which segments are cut.
Node	OSM node ID	A point of the map. Typed as ordinary, connector (segments meet without a turn), signalized, unsignalized or end (boundary of the network).
Segment	<way id>0 / <way id>1	One direction of one OSM way with a constant lane count (0 is the forward direction of the way, 1 the backward direction). The finest topological element.
Link	<upstream node>_<downstream node>	A directed stretch of road between two intersections or end nodes, made of one or more segments.
Movement	<upstream node>_<junction>_<downstream node>	A turn at a junction from an upstream link to a downstream link. The artificial sink movement that terminates a link at the network boundary is <link id>_dest.
Arterial	Arterial name	A named corridor, holding one OnewayArterial per direction of travel.

Every element carries a geometry (Geometry, two parallel lists lon and lat) and a display_geometry shifted sideways so that the two directions of a road do not overlap when plotted.

Network and MetaNetwork

MetaNetwork is the container base class with the dictionaries nodes, links, segments, movements and arterials, plus networkx_graph, a NetworkX graph of the links used for shortest-path queries. Its helpers are add_node / add_link / add_segment / add_movement, get_movement_from_links(upstream_link_id, downstream_link_id) and get_link_sink_movement(link_id).

Network(region_name, city_id) adds the region metadata, the ways dictionary, the signalized_node_list, unsignalized_node_list and end_node_list of node IDs, and the signalized_network attribute that holds the derived SignalizedNetwork described next.

SignalizedNetwork

The signalized network is the view of the network that signal-related applications work with: only the signalized intersections are kept as nodes, and the links between them are rebuilt so that consecutive signals become adjacent. It is derived during build_traffic_network by build_signalized_network (mtldp.preproc) with two parameters read from the region configuration:

• search_link_length_threshold (default 2000 m): starting from each movement of a signalized node, the upstream and downstream links are extended through unsignalized and connector nodes until another signalized node is reached or this distance is exceeded. Links that belong to an arterial are followed without a distance limit so that arterials stay connected;

• min_link_entry_threshold (default 200 m): the minimum length kept when an extended link has to be cut because no signal was found within the search distance.

The result is a MetaNetwork with the same ID conventions (movement IDs are recomputed from the new links, and every link keeps its sink_movement), its own networkx_graph and the arterials rebuilt on the reduced link set. AppRegion loads this view as its network; process_trajs --process-mode signalized computes indices on it.

Node and its subclasses

Attribute	Meaning
node_id, name, type	Identifier, optional street-based name and the node type listed above.
latitude, longitude	Position, parsed from the OSM attributes.
osm_attrib, osm_tags	Raw OSM attributes and tags.
upstream_segments, downstream_segments	Segments entering and leaving the node.
upstream_links, downstream_links	Links entering and leaving the node.
movement_list	Movements through the node (add_movement de-duplicates by ID).

Helpers: is_intersection() (signalized or unsignalized), is_ordinary_node(), get_upstream_junctions(). The subclasses SignalizedNode (with controller_id, the key into the SPaT data), UnSignalizedNode, EndNode and SegmentConnectionNode pin the type and are created from a plain node with init_from_node.

Segment

Attribute	Meaning
segment_id, osm_way, osm_direction_flag	Identifier, the source OsmWay and forward / backward.
belonged_link	The Link the segment is part of.
upstream_node, downstream_node, node_list	End nodes and all nodes along the segment.
upstream_segments, downstream_segments	Connected segments.
downstream_directions_info, downstream_directions	Turn letter (l, s, r) to downstream segment ID, and the concatenated letters.
length, speed_limit, lane_number, lane_assignment	Length in meters, speed limit in m/s, lane count and lane-use string (e.g. left|through;right).
from_heading, to_heading, from_direction, to_direction	Headings in degrees and cardinal directions (N, E, S, W) at both ends.
geometry, display_geometry	Polylines.

Link

Attribute	Meaning
link_id, upstream_node, downstream_node	Identifier and end nodes.
segment_list, node_list	Ordered segments and nodes.
length, speed_limit, heading, from_direction	Length in meters, speed limit, heading and direction of origin.
sink_movement	The <link id>_dest movement.
geometry, display_geometry	Polylines.

to_dict() and to_df() export a link as a dictionary or a one-row DataFrame with the references replaced by IDs.

Movement

Attribute	Meaning
movement_id, node	Identifier and the junction.
upstream_link, downstream_link	Entering and exiting links (downstream_link is None for a sink movement).
direction	Turn code: l left, s straight, r right, combinations such as ls, and d for a sink movement.
movement_index	Standard intersection movement number (the numbering used by signal engineers and by the SPaT network_mapping_data.csv): 1 = eastbound left, 2 = westbound through, 3 = southbound left, 4 = northbound through, 5 = westbound left, 6 = eastbound through, 7 = northbound left, 8 = southbound through; 9 to 12 are the right turns and 13 to 16 are used for U-turns and special cases. -1 for sink movements.
geometry, display_geometry	Upstream link polyline followed by the downstream link polyline.

Movement indices at a four-leg intersection.

Arterial, OnewayArterial and NetworkLinkPath

NetworkLinkPath is an ordered chain of links in a network: link_list, origin_link, destination_link, the concatenated geometry, the total length, distance_by_link (cumulative distance at the end of each link) and movement_list (the movement between each pair of consecutive links, None when they are not connected, in which case connected is False). It is initialised with init_from_link_list(link_id_list) or init_from_od(origin_link_id, destination_link_id); fill_shortest_path() fills gaps between consecutive links with the shortest path of the networkx_graph.

Arterial(arterial_id) is a corridor with an optional ref_node and a dictionary oneways of direction letter to OnewayArterial. OnewayArterial is a NetworkLinkPath with a direction, a name ("<arterial id> <direction>B") and a back-reference sup_arterial. Arterials are defined in raw_map/arterial.json (see Stage 1: build_traffic_network).

BoundingBox and Geometry

BoundingBox(lon_1, lat_1, lon_2, lat_2) normalises its corners into min_lon, min_lat, max_lon, max_lat (rounded to 5 decimals). to_list() returns [min_lon, min_lat, max_lon, max_lat], get_expanded_bbox(meters) grows the box by a distance in meters and split_into_grids(lat_num, lon_num) returns a grid of boxes.

Geometry stores lon and lat lists. to_string() yields "lon lat;lon lat;…", to_coord_list(lat_ahead) a list of tuples, and + concatenates two polylines while merging the shared joint point.

Constants and types

mtldp.meta.TrafficNetwork.constants defines MPH_TO_METERS_PER_SEC (0.44704), MILE_TO_METER (1609.34), DISPLAY_LANE_INTERVAL (4) and SEGMENT_SHIFT (1.5) used for the display geometry. types defines the Literal aliases Direction (E W N S), OsmDirection (forward backward), Turn (l r s ls rs lr lsr) and NodeType.

Trajectory

Vocabulary

MTLDP distinguishes three levels of trajectory data:

• a point is one GPS record: timestamp, latitude, longitude, speed, error and, after processing, distance (distance to the stop bar of the junction);

• a trajectory is the sequence of points of one vehicle on one movement, i.e. from the moment the vehicle enters the link that leads to a junction until it leaves the junction. Its ID is traj_id;

• a trip (trip_id) is one journey of a vehicle: the chain of trajectories it produced on consecutive links.

Trajectory

Trajectory(traj_id, trip_id, points_df) holds the point table in points_df and any number of trajectory-level attributes. Attributes are dynamic: setting trajectory.control_delay = 12.3 registers control_delay in traj_attributes, and TrajectoryDict.get_trajs_df() uses that list to build a table. Helpers: get_points_df(attributes), get_points_attributes(*names) (one list per point column), set_points_attributes(**columns), get_traj_attributes(*names) and get_traj_attributes_dict(); len(trajectory) is the number of points.

The canonical attribute names are exported by mtldp.meta.Trajectory:

Constant	Attributes
basic_trajs_attributes	trip_id, traj_id, date, tod, points_num, junction_id, link_id, segment_list, movement_id, movement_index, timestamp, avg_error
simple_trajs_attributes	travel_time, avg_speed, travel_distance, dis_diff
performance_attributes	the simple attributes plus min_dis_to_junction, control_delay, service_level, stop_delay, stop_nums, free_arrival_time, arrival_time, free_v, queue_dis, stop_details, split_failure
complete_trajs_attributes	comments plus the performance attributes plus distance_list
basic_trajs_table_attributes	the basic attributes plus the point lists timestamp_list, latitude_list, longitude_list, speed_list, error_list
complete_points_attributes	the point lists plus distance_list

The definition of each performance attribute is given in Stage 4: process_trajs.

TrajectoryDict

A dict of traj_id to Trajectory with table helpers: get_trajs_df(attributes) (one row per trajectory), get_points_df(attributes) (all points with trip_id and traj_id columns), groupby(attribute) (a dictionary of TrajectoryDict per value, e.g. per movement_id), sort_by(key), add_trajectory and del_trajectory. It is what AppRegion.load_trajs_dict returns.

Trip

Trip(trip_id) groups the trajectories of one journey in trajs_dict and can rebuild the route as a NetworkLinkPath with get_trip_path(network) (stored in link_path). get_trajs_attrs(*names) and get_points_attrs(*names) concatenate attributes over the trajectories, optionally dropping the first and last trajectory (head / tail), and get_trajs_total(*names) sums them. trip_geometry, origin_coord and destination_coord describe the whole journey.

SPaT

Signal Phase and Timing data is stored as the complete history of what every controller was programmed to do, and resolved on demand into a period snapshot for a list of dates and a time of day window. None of the classes inherit from each other; they nest as shown below.

HistoryRegionSPaT                                  all controllers of a region, all versions
└── controller_spat_history_dict {controller_id: HistoryControllerSPaT}
      │  get_controller_spat_period(date_list, time_range)
      ▼
    PeriodControllerSPaT                           one controller, one resolved timing plan
    ├── movement_spat_period_dict {node_id: {movement_index: PeriodMovementSPaT}}
    └── phase_spat_period_dict    {node_id: {phase: PeriodPhaseSPaT}}

HistoryRegionSPaT.get_corridor_spat_period(...) -> PeriodCorridorSPaT
└── controller_spat_period_dict {controller_id: PeriodControllerSPaT}

HistoryRegionSPaT

Built by parse_spat from the seven CSV files of raw_spat (their content is described in Preparation of Signal Phase and Timing (SPaT) data) and stored as spat/spat.pickle. It keeps the seven tables as DataFrame attributes, the mapping node_controller_dict from network node ID to controller ID (a controller can run several intersections), and one HistoryControllerSPaT per controller in controller_spat_history_dict.

Query methods:

• get_controller_spat_history(node_id);

• get_controller_spat_period(node_id, date_list, time_range) returns a PeriodControllerSPaT;

• get_corridor_spat_period(corridor_id, node_id_ls, date_list, time_range) returns a PeriodCorridorSPaT;

• get_tod_event_time_splits(date_ls, node_id_ls=None) returns the [start, end] windows (hours) delimited by the time-of-day events of the given controllers, i.e. the natural analysis periods to pass as time_range.

HistoryControllerSPaT

One controller with its node_id_ls, the mappings from phases to movement indices (phase_mappings, inverse movement_mappings), permissive movements (perm_movements_mappings), lane counts (no_lanes_mappings) and flashing modes (flashing_mappings), and the versioned programs: version_history (start dates of the dial-split, ring-structure and TOD-event versions), tod_event_versions (version → program day → list of TODEvent) and dial_split_versions (version → "dial:split" → DialSplit).

get_controller_spat_period(date_ls, time_range) resolves the plan in effect:

1. pick, for every date, the latest version that started before the date. The ring-structure version must be the same for all dates; differing dial-split or TOD versions only warn;

2. map each date to its program week (program_week_data.csv) and program day (program_day_data.csv, exact date first, otherwise day of week plus ten times the week);

3. select the TOD events of each program day that overlap time_range and read their dial:split;

4. look up the DialSplit (phase splits, phase modes, cycle, offset, sequence). All dates and the whole time range must resolve to a single set of parameters, otherwise an exception is raised. The reserved dial split 5:5 stands for flashing operation (cycle 0).

PeriodControllerSPaT

The resolved plan of one controller: cycle, offset, phase_splits, phase_modes and the ring structure (ring_phases, next_phases, concurrent_phases, lock_starts, lock_ends, min_splits, ped_min, yellow_intervals, clearance_intervals, channels, ped_channels). From these it computes phase_starts (green start of each phase within the cycle), fixed_starts / fixed_ends (whether a phase boundary is fixed or actuated, from the phase mode) and early_phase_starts (earliest green start under actuation, with a variant assuming a minor-street call). get_movement_spat_period(node_id, movement_index) returns the leaf object below; phase_spat_period_dict[node_id][phase] gives the phase view. The provenance of the plan is kept in version_history, program_weeks, program_days, tod_events and dial_splits.

PeriodMovementSPaT and PeriodPhaseSPaT

PeriodMovementSPaT is the object most analyses consume: for one node_id and movement_index over date_ls and time_range it gives cycle, offset, green_start, green_duration (including yellow and all-red), yellow, clearance, early_green_start, early_green_start_minor_street_call, the controlling phase_ls and the flags fx_start / fx_end. Movements that are also served permissively carry permissive_start, permissive_duration, permissive_yellow and permissive_clearance.

PeriodPhaseSPaT is the same information keyed by phase, with movement_index_ls, permissive_movements and no_lanes.

PeriodCorridorSPaT

Coordination view of several controllers along a corridor: controller_spat_period_dict, controller_cycle_lengths, same_cycle_lengths (False when the non-flashing controllers disagree), the common cycle_length and node_offsets. get_movement_spat_period(node_id, movement_index) delegates to the right controller.

Typical use

from mtldp.utils.config import AppRegion

region = AppRegion('configs/oakland_co_subs/0_1.json')          # loads network and SPaT
spat = region.spat

windows = spat.get_tod_event_time_splits(['2025-03-03', '2025-03-04'], ['62590214'])
period = spat.get_controller_spat_period('62590214', ['2025-03-03', '2025-03-04'], windows[1])
nb_through = period.get_movement_spat_period('62590214', 4)
print(nb_through.cycle, nb_through.offset, nb_through.green_start, nb_through.green_duration)