Real-Time Grain Movement by Rail

Skip to text

Text begins

Acknowledgements

The development of this grain railway mapping tool benefited from the collaboration and knowledge sharing with Dennis Huynh, of the Data Exploration and Integration Lab of Statistics Canada. Special thanks to Alessandro Alasia, Mahamat Hamit-Haggar, and the colleagues of the Dissemination and The Daily teams of Statistics Canada, who provided valuable assistance that contributed to Statistics Canada's first daily release.

The authors are also immensely grateful for the efforts of Greg Northey of the Ag Transport Coalition, as well as Milt Poirier and Vincent Roy of QGI Consulting in the development of this product. These individuals and their respective organizations are acknowledged for their valuable contributions. Any errors and omissions remain the sole responsibility of the authors.

Overview

The COVID‑19 pandemic and the following global shocks have created disruptions to supply chains all over the world in various economic sectors. In 2021, a survey of manufacturers and exporters reported that 90% of firms in those sectors were struggling with supply chain issues (Sadi-Nezhad, 2021). In 2022, results of four quarterly cycles of the Canadian Survey on Business Conditions indicated that supply chain challenges were a consistent and prominent issue for businesses (Statistics Canada, 2022). Of the roughly 30% of businesses that forecasted more supply chain problems in the first quarter of 2023, over half (56.0%) reported that they expected these problems to worsen (Statistics Canada, 2022).

This rapidly evolving context highlighted the limits of traditional ways to monitor and report on supply chain performance, notably with respect to the timeliness and frequency of available data. There is a growing interest in accessing real-time data from policy makers and industry stakeholders to make decisions based on as recent as possible data.

Statistics Canada has launched several initiatives to this effect, including flash estimates for several macroeconomic statistics in addition to a new quarterly survey on business conditions (Statistics Canada, 2022). The Grain Supply Chain Dashboard (GSCD) is implemented as a part of this effort. This daily updated data product allows users to study the current and historical day-to-day movement of grain by railways; hence, supporting the monitoring of grain supply chain performance and sustainable supply chain management. The reference scope of this data product spans from August 2016 onwards and will be updated daily to reflect current grain movements by rail.

This document accompanies the inaugural introductory release of the GSCD and presents the conceptual structure of the railway movement, the data and computation methods, and current limitations. The GSCD is released as an experimental data product. It is acknowledged that these metrics do not capture all dimensions of railway conditions. Moreover, the methodological developments of this dashboard are ongoing. New data sources may be incorporated into the tool over time and, consequently, the computation methods may too be revised over time.

Innovation through partnerships

In the aftermath of post-pandemic disruptions, the Government of Canada organized a National Supply Chain Summit emphasizing the importance of innovation and new approaches to supply chain management to boost economic recovery (National Supply Chain Summit, 2022).

The Summit called to increase the level of data and information sharing among key stakeholders, particularly in the agriculture and agri-food sector, so as not to disadvantage small and medium scale enterprises within the sector against their foreign counterparts (National Supply Chain Summit, 2022). Following reports emphasized the need to create end-to-end supply chain visibility for efficiency, accountability, planning, investment, and security; all requiring as timely data as possible (National Supply Chain Summit, 2022).

The GSCD addresses some of the recommendations that emerged from the 2022 Summit by enabling real-time monitoring of grain rail movements within Canada through a data partnership with key stakeholders. This mapping tool allows users to observe current conditions of the railways' networks, which could lead to improved planning and direct mitigation action in the eventuality of port congestion.

Real-time: Daily frequency

The concept of timely data has evolved with changing economic context and new information technology. Today, the term real-time generally implies data of a weekly frequency or greater. This project has pushed the frontier of real-time data production at Statistics Canada by enabling daily releases of statistics with the reference period of the previous day.

The implication of such a significant improvement is the minimal lag time between acquisition and release for easily accessible information to the public. The dashboard provides a means to track Canadian grain movement by rail on a daily frequency. Filter options provide users the ability to customize the visual by selected date and province.

Data partnership

The GSCD was made possible thanks to a data partnership between Statistics Canada and the Ag Transport Coalition (ATC). ATC is a coalition of agricultural associations that have come together to jointly fund a 5-year Growing Forward 2 initiative aimed at enhancing the competitiveness of the agriculture supply chain in Canada. ATC provides custom services to its members and publicly releases daily and weekly reports on the status of the Canadian railway network.

ATC members, at the time of writing, consist of the following organizations: Alberta Wheat Commission; The Canadian Canola Growers Association; The Canadian Oilseed Processors Association; The Inland Terminal Association of Canada; The Manitoba Pulse & Soybean Growers; Pulse Canada; and The Western Grain Elevator Association.

The data provided by ATC are further described in the following section.

Structure of the GSCD

The current version of the GSCD is built off two components. The first is a static component intended to capture the geographical location of grain elevators and railway network. The second component captures the real-time movement of railway cars transporting Canadian grain.

Grain elevators and railway network – static component

The static component geographically locates the Canadian grain elevators and railway network components, including main tracks, stations, and subdivisions. A railway subdivision refers to a delimited portion of the railway network, including all tracks, and stations along this delimited portion.

Railway movement – real-time components

The multi-faceted railway movement component is generated from data on the daily location of individual hopper cars transporting grain each day. This detailed dataset allows users to estimate metrics related to the movement of grain by rail across Canada. By coupling new data everyday along side historical data users can analyze trends and patterns within the data.

Three  metrics guided the development of the GSCD. These metrics make the specification of this mapping tool unique, and it is expected to remain at the base of further development and exploratory work of this nature:

1. Number of cars entering, en route and exiting at the corridors level
2. Cars daily last known location
3. 48-hour or more dwelling time of cars

Data sources

There are two main sources of data that are used in the current version of the GSCD. The first data source is the Open Government data, which is used for extracting data on the Canadian railway networks, as well as the geographical location of grain elevators. The second data source is provided by the Ag Transport Coalition, which is used to track daily grain movement by rail in Canada.

Open data

Open Government data features raw datasets from the Government of Canada. It is one of the three main resource channels of the Open Government Portal (Open Data, 2023. Open Information, 2023. Open Dialogue, 2023). It carries relevant datasets from various government agencies, thereby making it readily available to the public. The railway networks were sourced from the Canada Centre for Mapping and Earth Observation and Natural Resources Canada. The grain elevators dataset is from Agriculture and Agri-Food Canada and is based on a list provided by the Canadian Grain Commission. 

Ag Transport Coalition data

Real-time data on the location of rail cars containing grain are purchased by ATC from Railinc Corporation, a for-profit subsidiary of the Association of American Railroads, which provides rail data and messaging services to the North American freight railway industry.

Raw data are continuously collected by ATC and processed in a daily batch for cleaning and reporting purposes. Processed daily batches are then validated and shared with Statistics Canada to update the GSCD.

Due to Railinc reporting errors there can be a delay of events reported in the daily data. This will result in the GSCD event reporting when data is received, that is all data received daily will be published that day regardless of the time lag of specific events received. To compensate for this the GSCD will undergo quarterly maintenance updates to ensure the most possible accuracy of the historical data.

This data contains a unique ID for each rail car that is used to track its trip. Throughout each car's trip, every rail car event is documented with the exact time and location. The range of recorded events spans from the initial loading of grain, arrivals and departures at each train station, unloading of grain at final destination, and more.

In keeping with its mandate, the data provided by ATC to support this initiative is being done with the consent of the member companies of ATC. All data provided by ATC removes data elements, including shipper identity and origin station location in order to preserve the confidentiality of commercially sensitive information for all participating companies. 

It is from these events that the metrics in the GSCD are produced.

Computation methods

This mapping tool combines the static components of grain elevators and railways, obtained from Open Government, with the real-time data from Ag Transport Coalition. The methodology entails the use of advanced algorithms to extract the number of cars at different locations; as well as the times to depict movement patterns.

Each railway car is given a unique ID code which is supplied alongside the latitude and longitude locations of each event that occurs in the rail car's trip from grain elevator to final destination. In addition, information is given about the specific car's final destination which determines the corridor it belongs to. With this information the computation algorithm aggregates this data into the final outputs presented on the dashboard. The specific methods used for the four main metrics are as follows.

1. Number of cars entering, en route, and exiting at the corridor level
• Filters the raw daily data and summarizes all the unique car IDs that are entering the system, enroute to destination, and unloading at destination for each corridor.
2. Cars daily last known station
• Filters the raw data into groups for each unique rail car ID and then takes the last recorded event station for each day. These groups are then joined and summed for total number of cars at each station.
3. 48-hour or more dwelling time of cars
   • Counts the number of days each car has been dwelling at the same station. When this number reaches 2 or more it is counted in the dwelling category.

The use of daily data imposes similarly short processing timelines. In the development of the railway movement, specific attention was paid to the automation of data extraction processing and computation. A seamless processing protocol was implemented and is written in R to reduce the extraction time. The process to generate the visual is written using the Shiny package in R.

Confidentiality provisions

The current version of the GSCD blurs the origin of every trip to ensure that no confidential information is released. This confidentiality provision results in a loss of approximately 2% of railway station specific data, these events are still counted in the number of cars entering, en route, and exiting at the corridor level data.

Current coverage and update schedule

Overall, the GSCD covers 422 grain elevators, 3,094 train stations and 358 unique subdivisions.

The GSCD covers grain elevators and railway network components from the following eight Canadian provinces:

• Alberta
• British Columbia
• Manitoba
• New Brunswick
• Nova Scotia
• Ontario
• Quebec
• Saskatchewan

The GSCD covers six unique destination corridors, among which three are geographical regions and three are ports:

Geographical regions

1. Eastern Canada
• Refers to rail traffic moving towards the provinces of Ontario, Quebec, New Brunswick and Nova Scotia regardless of the cars' final destination (e.g., ports, primary/processing elevators). Cars moving towards the port of Thunder Bay are excluded from this destination corridor.
2. Western Canada
   • Refers to rail traffic moving towards the provinces of Manitoba, Saskatchewan, Alberta, and British Columbia regardless of the cars' final destination (e.g., ports, primary/processing elevators). Cars moving towards the ports of Prince Rupert and Vancouver are excluded from this destination corridor.
3. USA/MEX
   • Refers to rail traffic moving towards the United States or Mexico.

Ports

1. Prince Rupert
• Refers to rail traffic moving towards terminal elevators located at the port of Prince Rupert, BC.
2. Thunder Bay
   • Refers to rail traffic moving towards terminal elevators located at the port of Thunder Bay, ON.
3. Vancouver
   • Refers to rail traffic moving towards terminal elevators located at the port of Vancouver, BC.

The dashboard is updated daily at 10:00 a.m. (EST) for the preceding reference day. The update schedule may be subject to change, notably on statutory holidays when the product will not be updated.

Further developments

This is an introductory product which is expected to be further developed, revised, and expanded.  Future developments of the dashboard may include indicators of velocity (the time it takes each train to travel between stations; as well as predictive metrics (estimates of future railway traffic and potential dwelling times).

Additional filtering options are also expected to be developed. For example, the existing and future metrics could be presented by week, month, season and year; by time of day (morning / afternoon / evening), or by owner of railway tracks.

Geographic coverage is also expected to include additional granular regional information. This is likely to result in changes to the methodology and computational processes. Finally, further integration with Statistics Canada data holdings, in particular monthly surveys will be further explored.

Glossary

Carload

Shipment that uses one rail car for its transportation.

Corridor

In the context of the GSCD, used to identify the destination to which a rail car is heading towards, which is either a port or a geographical region. The GSCD covers six unique corridors: Eastern Canada, Western Canada, USA/MEX, Prince Rupert, Vancouver, and Thunder Bay. Also referred to as "destination corridor".

Dwell Time

Time spent by a car at a location stop without moving.

Eastern Canada

Refers to the provinces of Ontario, Quebec, Nova Scotia, and Prince Edward Island.

Primary Elevators

Grain elevators licensed by the Canadian Grain Commission as a "primary elevator", the principal use of which is the receiving of grain directly from producers for storage or shipping.

Rail Car

Mobile equipment running exclusively on rails moving within a train powered by one or more locomotive.

Released Loads

Number of loaded rail cars entering the system after being loaded by shippers at origin.

Subdivision

Refers to a delimited portion of the railway network, including all tracks and stations along this delimited portion.

Terminal Elevators

Grain elevators licensed by the Canadian Grain Commission as a "terminal elevator", the principal uses of which are the receiving of grain on or after official inspection and official weighing of the grain and the cleaning, storing and treating of the grain before it is shipped.

Western Canada

Refers to the provinces of Manitoba, Saskatchewan, Alberta, and British Columbia.

References

Grain Elevators in Canada. (2023).

National Railway Network - NRWN - GeoBase Series. (2023).

Open Data. (2023).

Open Dialogue. (2023).

Open Information. (2023).

Sadi-Nezhad, S. (2021). A survey on the effect of supply chain disruption on Canadian economy. Journal of Future Sustainability, 1(1), 17-20.

Statistics Canada. (2022). Canadian Survey on Business Conditions, fourth quarter 2022.

Transport Canada. (January 31, 2022). National Supply Chain Summit: What we heard report.