.. _map_module: Map data and network ################################ .. sectionauthor:: Xingmin Wang Overview ************************************** **Map data and network module (mtldp.mtlmap) is used to process OpenStreetMap data and convert it to a well-defined python class object. This module also includes a series of functions built on the network class including implementing network-related algorithms, plotting, etc.** Why map matters? ============================================= Map data and the network data structure is one of the most critical foundations for both the trajectory data processing and network analysis. As shown in the following figure, map data and network is usually the base for all the other data including the trajectory data, signal phase and timing (SPaT), and traffic demand and flow data, etc. Most of the data needs to be indexed by the map data before further processing; for example, the map data will be used for the trajectory map-matching. For the network flow data and the SPaT data, we need to know the mapping between the map and the data. .. image:: ../_static/overview/map_base.svg :width: 75% :align: center :alt: map data workflow | Therefore, :ref:`mtldp.mtlmap ` provides a bunch of map data processing function including parsing the ``OpenStreetMap`` data to a well-organized network data structure :class:`mtldp.mtlmap.Network`, saving the network data into different map formats including ``.osm(xml)/ESRI shapefile``, and some basic plotting functions. Parsing the OpenStreetMap ======================================== The core of the ``mtldp.mtlmap`` is a ``OpenStreetMap`` data parsing function :func:`mtldp.mtlmap.build_network_from_xml` that parses ``OpenStreetMap`` data to a well-organized network data structure :class:`mtldp.mtlmap.Network`. The following illustration shows the flowchart of this map data parsing function. The map data parsing function can construct different layers from the ``OSM`` data including ``nodes``, ``segments``, ``movements``, and ``laneSets``, etc. .. _map_mode: .. image:: ../_static/overview/map_process.svg :width: 85% :align: center :alt: map data process | The map data parsing and reconstruction is always about a trade-off between accuracy and precision. The generation of a more accurate layer tends to have more errors since the original ``OSM`` data usually does not have sufficient lane-level information including the assignment and usage, etc. Therefore, we divide the map data parsing and generation into different modes with different output levels: .. autoclass:: mtldp.mtlmap.MapMode :noindex: .. note:: The ``OpenStreetMap`` data could be very complicated with many different corner cases, errors including absent/wrong attributes/tags, unexpected topology, etc. Therefore, it is inevitable that there are many situations that could not be parsed or recognized correctly using the current map data parsing function. To deal with this, we might need to either manually fix some bugs or keep improving the current map parsing function. For now, the map parsing function will output a log file with all the unexpected cases for users' information. Besides, the map parsing function will call :class:`mtldp.mtlmap.Network.build_networkx_graph` automatically to get a `NetworkX `_ graph object according to different layers (``segment``, ``laneSet``, ``link``). This means that the implemented algorithms (e.g., shrotest path, network partition) in the NetworkX can be easily adopted. Map data edit and workflow ========================================== Since the map data parsing and reconstruction will inevitably involve some noise and errors, the manual edit would be helpful to deal with some complicated corner cases or some customized demand such as adding additional attributes or tags to the road segments. To accomplish this, the workflow of the map data should be considered carefully. The following figure shows different production flowcharts for the map data. .. image:: ../_static/overview/map_workflow.svg :width: 80% :align: center :alt: map data workflow `JSOM `_ is a light-weight and easy-to-use open-source software that can help to view and edit the OpenStreetMap data. We recommend users to use this software to edit the map data before loading it to the data platform if needed. Map data visualization ========================================== .. todo: add the plotter We also create an easy web-based map data visualization tool hosted at `mtldp map web `_. This web can take the ``.json`` file as the input and show the map data accordingly. The ``.json`` file can be generated using the :func:`mtldp.mtlmap.output_static_geometry_json`. Here is a screenshot of the website, you can choose different layers and click on the road segment to see the detailed information: .. image:: ../_static/map_related/web.png :width: 100% :align: center :alt: Web app to display the network Network data structure *********************************************** .. note:: The network data is designed as two versions: static network and network of certain time of day (TOD). The static network contains the static network topology and geometry. The network of TOD inherits all the static network information; besides, users can also dump the trajectory data as well as other data set into this network object to perform location-based study. .. _static_core: Static network ================================================ +-----------+--------------------------------+--------------------------------------------------------------------------------+ | Name | Class | Content | +===========+================================+================================================================================+ | Network | :class:`mtldp.mtlmap.Network` | Class object for the whole network | +-----------+--------------------------------+--------------------------------------------------------------------------------+ | Nodes | :class:`mtldp.mtlmap.Node` | Class for the nodes in the network | +-----------+--------------------------------+--------------------------------------------------------------------------------+ | Links | :class:`mtldp.mtlmap.Link` | Road segment connects intersections composed of a list of segments | +-----------+--------------------------------+--------------------------------------------------------------------------------+ | Movement | :class:`mtldp.mtlmap.Movement` | A movement is defined as a feasible in-link and a out-link at intersections | +-----------+--------------------------------+--------------------------------------------------------------------------------+ | Segment | :class:`mtldp.mtlmap.Segment` | Road segment with homogeneous parameters (``way`` in OSM data) | +-----------+--------------------------------+--------------------------------------------------------------------------------+ | LaneSet | :class:`mtldp.mtlmap.LaneSet` | Class for a group of lanes that have the same downstream direction | +-----------+--------------------------------+--------------------------------------------------------------------------------+ .. note:: All the road segments including ``Link``, ``Segment``, ``LaneSet`` are directed. Network of TOD ============================ The current classes extends the static network, all the other classes not included here remain the same with the static network. +----------------------------+------------------------------------------+--------------------------------------------------------------------------------+ | Name | Class | Content | +============================+==========================================+================================================================================+ | Network of TOD | :class:`mtldp.mtlmap.NetworkOfTOD` | Class object for the whole network with additional data of certain time of day | +----------------------------+------------------------------------------+--------------------------------------------------------------------------------+ | Movement of TOD | :class:`mtldp.mtlmap.MovementOfTOD` | Class of the movement with data (trajectory, SPaT) of certain TOD | +----------------------------+------------------------------------------+--------------------------------------------------------------------------------+ Map functions ************************************************ Core functions ================================================== +--------------------------------------------------------+--------------------------------------------------------------------------------+ | Function | Content | +========================================================+================================================================================+ | :func:`mtldp.mtlmap.build_network_from_xml` | Build the :class:`mtldp.mtlmap.Network` class from OSM data | +--------------------------------------------------------+--------------------------------------------------------------------------------+ .. todo: put all the commonly-used function here Utility functions ================================================ .. todo: put all the commonly-used function here Reference ************************************ Source code :ref:`Map data and network module ` Tutorial :ref:`Download OpenStreetMap data `