Command line tools of mtldp-preproc#
Installing mtldp-preproc registers the console scripts below (the entry_points of the
package). They implement the raw data processing pipeline described in The raw data pipeline. Every
command accepts --help.
All commands that take --config accept the configuration of a master region and process
each of its sub-regions in turn (or in parallel for build_traffic_network); passing the
configuration of a single sub-region processes only that sub-region. The two exceptions are
merge_osm, which works on plain files, and parse_spat, which works on exactly the region it
is given.
merge_osm#
(mtldp) $ merge_osm -i raw_data/my_region/raw_map/map_raw.osm \
-o raw_data/my_region/raw_map/map.osm -d 60
Merges signalized intersection nodes of an OpenStreetMap file that lie within a distance
threshold of each other into a single node. Large intersections are often mapped as two or four
separate highway=traffic_signals nodes (one per carriageway); MTLDP expects one node per
intersection, otherwise the movements and the SPaT mapping cannot be attached to it. This command
performs the merge automatically, which replaces the manual editing step in JOSM described in
Preparation of OSM data.
Option |
Default |
Description |
|---|---|---|
|
Input OSM XML file. |
|
|
Output OSM XML file. |
|
|
|
Maximum distance in meters between two signalized nodes for them to be merged. |
build_traffic_network#
(mtldp) $ build_traffic_network -c configs/my_region.json -e 0_3
Builds the MTLDP traffic network of every sub-region from its raw_map inputs and writes all
derived network files into network/ (see Stage 1: build_traffic_network for the algorithm and the
outputs). Sub-regions are built in parallel, one process per sub-region up to the number of CPU
cores. Because the network changed, the cached ubodt.txt of the map matcher is deleted so that
it is regenerated on the next match_points_to_map run.
Option |
Default |
Description |
|---|---|---|
|
Configuration file of the master region. |
|
|
none |
Comma separated sub-region IDs to skip. |
The thresholds that control how far a signalized link is extended upstream and downstream are read
from the configuration file: search_link_length_threshold (default 2000 m) and
min_link_entry_threshold (default 200 m, at most half of the search threshold).
parse_spat#
(mtldp) $ parse_spat -c configs/my_region.json
Reads the seven SPaT CSV files of raw_spat/ (version_history.csv,
network_mapping_data.csv, tod_event_data.csv, ring_structure_data.csv,
dial_split_data.csv, program_week_data.csv and program_day_data.csv; their content is
described in Preparation of Signal Phase and Timing (SPaT) data), builds a HistoryRegionSPaT object and stores it as
spat/spat.pickle. With --estimate-spat the files are read from raw_spat/estimate/ and
the result is written next to the regular pickle with an _estimate suffix.
Option |
Default |
Description |
|---|---|---|
|
Configuration file of the region whose SPaT is parsed. |
|
|
|
Parse the estimated SPaT instead of the agency SPaT. |
split_points#
(mtldp) $ split_points -c configs/my_region.json -v SL_2025 \
-d /data/raw_trajs/year=2025
Reads the raw GPS trajectory files delivered by the data vendor and redistributes the points into
one CSV file per sub-region and per local date (trajectories/raw_pts/<date>_raw.csv). Each
raw file is read once; for each sub-region the points inside the sub-region bounding box are kept,
the local date, time and tod (time of day, in hours) columns are added using the region’s
time zone, and the rows are appended to the file of their date.
Option |
Default |
Description |
|---|---|---|
|
Configuration file of the master region. |
|
|
|
Layout of the raw files: |
|
|
Absolute path of the directory holding the raw files. For CSV formats only the files directly
inside the directory are read. For Parquet formats |
|
|
Expansion of the sub-region bounding box in meters. A warning is printed when it exceeds the built-in GPS buffer. |
match_points_to_map#
(mtldp) $ match_points_to_map -c configs/my_region.json -s 2025-03-03 -e 2025-03-09 \
--exclude-weekend
Runs the Fast Map Matching (FMM) algorithm on the daily point files of every sub-region and
writes trajectories/points/<date>_matched.csv with the matched road element and the map
attributes of every point. Despite the .csv extension these intermediate files are stored in
the binary Feather format for speed. See Stage 3: match_points_to_map for the details.
Option |
Default |
Description |
|---|---|---|
|
Configuration file of the master region. |
|
|
First date to process ( |
|
|
none |
Last date to process (inclusive). Without it only the start date is processed. |
|
include |
Skip Saturdays and Sundays of the date range. |
|
include |
Skip Mondays to Fridays of the date range. |
|
|
Reuse the upper-bounded origin destination table ( |
process_trajs#
(mtldp) $ process_trajs -c configs/my_region.json -s 2025-03-03 -e 2025-03-09 \
--process-mode signalized --filter-junctions
Cuts the matched points of every sub-region into movement-level trajectories and computes their
performance indices (travel time, control delay, number of stops, queue distance, …). The result
is one CSV per date: trajectories/trajectories/<date>_trajs.csv in all mode, or
<date>_signalized_trajs.csv in signalized mode. See Stage 4: process_trajs for the
columns and the definitions.
Option |
Default |
Description |
|---|---|---|
|
Configuration file of the master region. |
|
|
Date range, as for |
|
|
include both |
Date filters, as for |
|
|
|
|
|
Keep only the trajectories whose junction is listed in the |
|
|
Complete processing computes all performance indices; simple processing computes only travel time, average speed, travel distance and distance difference. |
|
|
Insert trajectories without points for the links a trip must have traversed between two matched links, so that every trip is a feasible path in the network. |
|
|
Maximum lateral distance of a point from the road centreline, per lane, in meters. Points further away are treated as noise and dropped. |
|
|
Process the trips of a day in a process pool. The points are split into chunks of complete trips, one per worker. |
|
|
Number of worker processes; |
eva_trajs_qc#
(mtldp) $ eva_trajs_qc -c configs/my_region.json \
-d 2025-03-03,2025-03-09@2025-06-02,2025-06-08 -o qc_march_vs_june
Quality control of the processed trajectories. For each date range the daily trajectory metrics (number of trajectories, points, matched share, …) are computed, cached under the region’s trajectory cache directory, and plotted side by side so that two or more periods can be compared day by day.
Option |
Default |
Description |
|---|---|---|
|
Configuration file of the region. |
|
|
|
|
|
|
Whether weekends are skipped when the missing daily metrics are generated. |
|
all junctions |
Comma separated junction IDs to include. |
|
auto-generated |
File name of the figure, without extension. |
|
|
Figure size in inches. |
|
|
Output format and resolution. |
Maintenance scripts in tools/#
Two stand-alone scripts live in the tools/ directory of the mtldp-preproc repository. They
are not installed as commands, need no mtldp import and are run with python tools/<script>.
convert_network_json_to_geojson.pyOne-shot migration of network geometry files (
traffic_network.json,signalized_*.json) written in the legacy{nodes, links, segments}layout with[lat, lon]coordinates into the current GeoJSON layout (see Network geometry as GeoJSON). Files are converted in place and the original is kept as<name>.pre-geojson.bak. Accepts files or directories (searched recursively) and--dry-run.reformat_network_json.pyRewrites GeoJSON network files in the compact style produced by
save_network_geometry_to_json(indented structure, everycoordinatesarray on one line). The content is verified to be unchanged before the file is written. Accepts--dry-runand--backup.