The Crittenden Automotive Library

Revised Carrier Safety Measurement System

Publication: Federal Register
Agency: Federal Motor Carrier Safety Administration
Byline: Robin Hutcheson
Date: 15 February 2023
Subjects: American Government

, Safety

[Federal Register Volume 88, Number 31 (Wednesday, February 15, 2023)]
[Notices]
[Pages 9954-9960]
From the Federal Register Online via the Government Publishing Office [www.gpo.gov]
[FR Doc No: 2023-02947]

-----------------------------------------------------------------------

DEPARTMENT OF TRANSPORTATION

Federal Motor Carrier Safety Administration

[Docket No. FMCSA-2022-0066]

Revised Carrier Safety Measurement System

AGENCY: Federal Motor Carrier Safety Administration (FMCSA), Department
of Transportation (DOT).

ACTION: Notice; request for comments.

-----------------------------------------------------------------------

SUMMARY: Since 2010, FMCSA has used its Safety Measurement System (SMS)
to identify motor carriers for safety interventions. The National
Research Council of the National Academy of Sciences (NAS) recommended
on June 27, 2017, that FMCSA develop and test a new statistical model.
This notice explains FMCSA's analysis and the Agency's proposed changes
to SMS, announces FMCSA's preview of the proposed changes, and requests
comments and input on the Agency's system to identify motor carriers
for safety interventions.

DATES: Comments must be received on or before May 16, 2023.

ADDRESSES: You may submit comments identified by Docket Number FMCSA-
2022-0066 using any of the following methods:
Federal eRulemaking Portal: Go to https://www.regulations.gov/docket/FMCSA-2022-0066/document. Follow the online instructions for
submitting comments.
Mail: Dockets Operations, U.S. Department of Transportation, 1200
New Jersey Avenue SE, West Building, Ground Floor, Room W12-140,
Washington, DC 20590-0001.
Hand Delivery or Courier: Dockets Operations, West Building, Ground
Floor, Room W12-140, 1200 New Jersey Avenue SE, Washington, DC 20590-
0001 between 9 a.m. and 5 p.m., Monday through Friday, except Federal
holidays. To be sure someone is there to help you, please call (202)
366-9317 or (202) 366-9826 before visiting Dockets Operations.
Fax: (202) 493-2251.
To avoid duplication, please use only one of these four methods.
See the ``Public Participation and Request for Comments'' portion of
the SUPPLEMENTARY INFORMATION section for instructions on submitting
comments.

FOR FURTHER INFORMATION CONTACT: Mr. Catterson Oh, Compliance Division,
FMCSA, 1200 New Jersey Avenue SE, Washington, DC 20590-0001, (202) 366-
6160, Catterson.Oh@dot.gov. If you have questions regarding viewing or
submitting material to the docket, contact Dockets Operations, (202)
366-9826.

SUPPLEMENTARY INFORMATION:

Public Participation and Request for Comments

If you submit a comment, please include the docket number for this
notice FMCSA-2022-0066, indicate the specific section of this document
to which each comment applies, and provide a reason for each suggestion
or recommendation. You may submit your comments and material online or
by fax, mail, or hand delivery, but please use only one of these means.
FMCSA recommends that you include your name and a mailing address, an
email address, or a phone number in the body of your document so the
Agency can contact you if it has questions regarding your submission.
To submit your comment online, go to https://www.regulations.gov/docket/FMCSA-2022-0066/document, click on this notice, click
``Comment,'' and type your comment into the text box on the following
screen.
If you submit your comments by mail or hand delivery, submit them
in an unbound format, no larger than 8\1/2\ by 11 inches, suitable for
copying and electronic filing. If you submit comments by mail and would
like to know that they reached the facility, please enclose a stamped,
self-addressed postcard or envelope.
FMCSA will consider all comments and material received during the
comment period and may change this notice based on your comments.

Confidential Business Information (CBI)

CBI is commercial or financial information that is both customarily
and actually treated as private by its owner. Under the Freedom of
Information Act (5 U.S.C. 552), CBI is exempt from public disclosure.
If your comments responsive to the notice contain commercial or
financial information that is customarily treated as private, that you
actually treat as private, and that is relevant or responsive to the
notice, it is important that you clearly designate the submitted
comments as CBI. Please mark each page of your submission that
constitutes CBI as ``PROPIN'' to indicate it contains proprietary
information. FMCSA will treat such marked submissions as confidential
under the Freedom of Information Act, and they will not be placed in
the public docket of the notice. Submissions containing CBI should be
sent to Mr. Brian Dahlin, Chief, Regulatory Analysis Division, Office
of Policy, FMCSA, 1200 New Jersey Avenue SE, Washington, DC 20590-0001.
Any comments FMCSA receives not specifically designated as CBI will be
placed in the public docket for this notice.

Viewing Comments and Documents

To view any documents mentioned as being available in the docket,
go to https://www.regulations.gov/docket/FMCSA-2022-0066/document and
choose the document to review. To view comments, click this notice,
then click ``Browse Comments.'' If you do not have access to the
internet, you may view the docket in person by visiting Dockets
Operations in Room W12-140 on the ground floor of the DOT West
Building, 1200 New Jersey Avenue SE, Washington, DC 20590-0001, between
9 a.m. and 5 p.m., Monday through Friday, except Federal holidays. To
be sure someone is there to help you, please call (202) 366-9317 or
(202) 366-9826 before visiting Dockets Operations.

Privacy Act

DOT solicits comments from the public to better inform its
processes, in accordance with 5 U.S.C. 553(c). DOT posts these
comments, without edit, including any personal information the
commenter provides, to

[[Page 9955]]

www.regulations.gov, as described in the system of records notice (DOT/
ALL 14--Federal Docket Management System), which can be reviewed at
www.transportation.gov/privacy.

Background

SMS Overview

In December 2010, FMCSA implemented SMS to identify high risk motor
carriers for investigations (75 FR 18256, April 9, 2010). Section
5305(a) of the Fixing America's Surface Transportation (FAST) Act,
Public Law 114-94 (Dec. 4, 2015; 129 Stat. 1312) requires FMCSA to
ensure, at a minimum, that a review is conducted on motor carriers that
demonstrate, through performance data, that they are among the highest
risk carriers for four consecutive months. FMCSA and its State
enforcement partners also use SMS to identify and prioritize motor
carriers for inspections and less resource-intensive interventions,
such as automated warning letters.
SMS determines a carrier's prioritization status (i.e., prioritized
or not prioritized) in each Behavior Analysis and Safety Improvement
Category (BASIC) based on the carrier's on-road performance and/or
investigation results. A carrier's relative on-road performance is
indicated by its BASIC percentile. Investigation results reflect if any
Acute and/or Critical (A/C) violations are found in a given BASIC
during investigations. A carrier can be prioritized for interventions
because its percentile is at or above the Intervention Threshold and/or
it has one or more A/C violations related to a particular BASIC.
SMS also provides motor carriers and other stakeholders with safety
performance data, which is updated monthly, through the public website
at http://ai.fmcsa.dot.gov/SMS. Under section 5223 of the FAST Act,
FMCSA removed SMS percentiles and alerts from the public SMS website
for motor carriers transporting property. Passenger carrier percentiles
and alerts remain publicly available, as well as inspection,
investigation, crash, and registration data for all carriers.
SMS quantifies the safety performance of motor carriers using data
available in FMCSA's motor carrier database, the Motor Carrier
Management Information System (MCMIS). This database includes
violations found during inspections, traffic enforcement, and
investigations, as well as crash and motor carrier census data. For
detailed information on the current structure of SMS, see the SMS
Methodology at http://csa.fmcsa.dot.gov. A copy of the SMS Methodology
is available in the docket for this notice.
FMCSA's analysis has shown that SMS is effective in helping the
Agency identify high crash risk carriers for interventions. FMCSA's SMS
Effectiveness Test (ET) found that the group of carriers that SMS
identified for intervention in one or more BASICs had a crash rate that
was 61 percent higher than the group of carriers not identified for
intervention. In addition, the group of carriers that met FMCSA's high
risk criteria had a crash rate that was 178 percent higher than the
national average crash rate. A copy of FMCSA's ET, which was first
published in 2014 and updated in 2018, is available in the docket for
this notice.
Section 5221 of FAST Act required that NAS conduct a study of
FMCSA's Compliance, Safety, Accountability (CSA) program and SMS.
Specifically, the FAST Act required that NAS:
(1) shall analyze--
(A) the accuracy with which the Behavior Analysis and Safety
Improvement Categories referred to in this part as ``BASIC'')--
(i) identify high risk carriers; and
(ii) predict or are correlated with future crash risk, crash
severity, or other safety indicators for motor carriers, including the
highest risk carriers;
(B) the methodology used to calculate BASIC percentiles and
identify carriers for enforcement, including the weights assigned to
particular violations and the tie between crash risk and specific
regulatory violations, with respect to accurately identifying and
predicting future crash risk for motor carriers;
(C) the relative value of inspection information and roadside
enforcement data;
(D) any data collection gaps or data sufficiency problems that may
exist and the impact of those gaps and problems on the efficacy of the
CSA program;
(E) the accuracy of safety data, including the use of crash data
from crashes in which a motor carrier was free from fault;
(F) whether BASIC percentiles for motor carriers of passengers
should be calculated separately from motor carriers of freight;
(G) the differences in the rates at which safety violations are
reported to the Federal Motor Carrier Safety Administration for
inclusion in the SMS by various enforcement authorities, including
States, territories, and Federal inspectors; and
(H) how members of the public use the SMS and what effect making
the SMS information public has had on reducing crashes and eliminating
unsafe motor carriers from the industry; and
(2) shall consider--
(A) whether the SMS provides comparable precision and confidence,
through SMS alerts and percentiles, for the relative crash risk of
individual large and small motor carriers;
(B) whether alternatives to the SMS would identify high risk
carriers more accurately; and
(C) the recommendations and findings of the Comptroller General of
the United States and the Inspector General of the Department, and
independent review team reports, issued before the date of enactment of
this Act.

NAS Study

On June 27, 2017, NAS published a report titled ``Improving Motor
Carrier Safety Measurement.'' FMCSA commissioned this report under
Section 5221 of the FAST Act. The report is available at https://www.nap.edu/catalog/24818/improving-motor-carrier-safety-measurement.
However, NAS did not complete all the reviews requested by the FAST
Act. The NAS report notes, ``This study is not concerned with non-SMS
aspects of CSA, and it is concerned only with CSMS [Carrier SMS], not
with DSMS [Driver SMS], but we will refer to our topic as SMS in the
remainder of this report.'' The NAS report concluded that SMS, in its
current form, is structured in a reasonable way and its method of
identifying motor carriers for alert status is defensible. NAS agreed
that FMCSA's overall approach, based on crash prevention rather than
prediction, is sound. NAS provided FMCSA with six recommendations to
improve the system. This notice focuses on FMCSA's actions in response
to the first NAS recommendation to develop an Item Response Theory
(IRT) model. FMCSA will update its full corrective action plan
addressing all six NAS recommendations after reviewing comments to this
proposal. The corrective action plan is available on FMCSA's website at
https://www.fmcsa.dot.gov/mission/policy/nas-correlation-study-corrective-action-plan-report-congress.
Pursuant to the FAST Act, FMCSA submitted the results of this study
to both Congress and the DOT Office of Inspector General (OIG) on
August 7, 2017. FMCSA also submitted the corrective action plan
required by the FAST Act to Congress on June 25, 2018. Copies of the
NAS report and FMCSA's action plan are available in the docket for this
notice.
OIG reviewed FMCSA's action plan as required by the FAST Act and on
September 25, 2019, provided its report titled ``FMCSA's Plan Addresses
Recommendations on Prioritizing Safety

[[Page 9956]]

Interventions but Lacks Implementation Details,'' available on the
OIG's website at https://www.oig.dot.gov/library-item/37465. The OIG
made two recommendations for FMCSA to provide additional details to
improve the corrective action plan in relation to three of the NAS
recommendations. FMCSA partially concurred with both recommendations,
stating that the Agency would first decide how to move forward with its
prioritization methodology before providing the cost estimates and
benchmarks recommended by OIG. The OIG considers the recommendations
resolved but open pending completion of planned actions. As one of
those planned actions, FMCSA conducted a full review of the IRT model
and made a decision on how to move forward with the prioritization
methodology, which is described in this FRN. A copy of the OIG report
is available in the docket for this notice.

IRT Modeling

The NAS report recommended that FMCSA develop an IRT model and
``[i]f it is then demonstrated to perform well in identifying motor
carriers for alerts, FMCSA should use it to replace SMS in a manner
akin to the way SMS replaced SafeStat.'' FMCSA contracted with NAS for
the establishment and operation of a standing committee of experts, as
well as with subject matter experts with experience in large-scale IRT
modeling, to provide advice and guidance to the Agency during the
development and testing of the IRT model. The IRT model was designed
and tested using inspection data from FMCSA's MCMIS database. The full
modeling report titled, ``Development and Evaluation of an Item
Response Theory (IRT) Model for Motor Carrier Prioritization,'' which
details the statistical methodologies that were applied in developing
and testing the IRT model, is available in the docket for this notice.
The Agency's IRT modeling work revealed many limitations and
practical challenges with using an IRT model. As a result, FMCSA has
concluded that IRT modeling does not perform well for the Agency's use
in identifying motor carriers for safety interventions, and therefore,
does not improve overall safety. First, IRT is heavily biased towards
identifying smaller carriers that have few inspections with violations
and limited on-road exposure to crash risk. When the safety event
groups and data sufficiency standards used in SMS were applied to the
IRT model, IRT produced similar results to SMS.
Second, IRT does not use Vehicle Miles Traveled (VMT) or Power
Units (PUs) to adjust for differences in on-road exposure in the Unsafe
Driving BASIC. As a result, IRT identified carriers with much lower
crash rates in that BASIC compared to SMS.
Third, IRT modeling is not readily understandable by most
stakeholders or the public. IRT's inherent complexity makes it
challenging for the industry and public to replicate and interpret
results. While SMS results can be reproduced and explained using simple
math, IRT requires an advanced understanding of statistical modeling
and analysis.
Fourth, a motor carrier cannot independently compute its IRT
results. IRT results can be computed only for the entire carrier
population. A carrier would not be able to identify how specific
violations or areas of regulatory noncompliance impacted its
prioritization status or how it could improve its status.
Finally, IRT's runtime is incompatible with FMCSA's operational
needs. The IRT model takes four weeks to run as compared to two days
for SMS. The long runtime would make it difficult to make even minor
changes to the system.
Because IRT is overly complex and adopting the IRT model would
reduce transparency without improving safety, FMCSA will not replace
SMS with an IRT model. Instead, FMCSA continues its commitment to
continuously improving SMS to identify motor carriers that present the
highest crash risk through a transparent and effective system.

Changes to SMS

The Agency conducted analyses during the IRT modeling study that
revealed areas in which SMS could be improved to better identify high
risk carriers for intervention, without the complications inherent in
adopting an IRT model. Those improvements include reorganizing the
BASICs, now called ``safety categories,'' to better identify specific
safety problems and combining the 959 violations used in SMS, plus 14
additional violations not currently used in SMS, into 116 violation
groups. In addition, the changes include simplifying violation severity
weights, removing percentile jumps that occur when carriers move into a
new safety event group, and adjusting the Intervention Thresholds.
FMCSA also previously published proposed changes as part of its efforts
to improve SMS (81 FR 69185, Oct. 5, 2016). The previously proposed
improvements included moving certain Out-of-Service (OOS) violations to
the Unsafe Driving BASIC, segmenting the Hazardous Materials (HM)
Compliance BASIC, focusing on recent violations, and updating the
Utilization Factor.
Taken together, FMCSA proposes the following combined improvements
to SMS: (1) reorganized and updated safety categories, including new
segmentation; (2) consolidated violations; (3) simplified violation
severity weights; (4) proportionate percentiles instead of safety event
groups; (5) improved Intervention Thresholds; (6) greater focus on
recent violations; and (7) an updated Utilization Factor.
FMCSA conducted the ET to measure the impact of the proposed
changes on potential future crash reduction. In addition, the Agency
analyzed other measures such as the A/C violation rate, which measures
egregious and systemic safety issues found during in-depth
investigations. Thus, a high A/C violation rate among prioritized
carriers affirms the ability of the prioritization system to identify
carriers that are more likely to exhibit these egregious safety issues.
In addition to the safety impacts measured with the ET and A/C
violation rate, the proposed changes were guided by FMCSA's continuing
commitment to enhance the accuracy, fairness, and clarity of its
prioritization system.
A document which describes the newly proposed changes and provides
additional analysis to support the proposed changes, titled
``Foundational Document'' and dated March 2022, is available in the
docket for this notice.

Reorganized and Updated Safety Categories

During the development and testing of the IRT model, FMCSA gained
valuable insight and concluded that reorganizing the BASICs, now called
``safety categories,'' could make it easier for FMCSA and motor
carriers to pinpoint and address safety issues. FMCSA proposes
reorganizing the Controlled Substances/Alcohol, Unsafe Driving, and
Vehicle Maintenance safety categories as described below. FMCSA also
proposes to segment the Driver Fitness and HM Compliance safety
categories to account for differences in carrier operations.
The new safety categories would be: (1) Unsafe Driving; (2) Crash
Indicator; (3) Hours of Service (HOS) Compliance; (4) Vehicle
Maintenance; (5) Vehicle Maintenance: Driver Observed; (6) HM
Compliance; and (7) Driver Fitness. A copy of the complete list of
violations in each safety category is available in the docket for this
notice and can also be found in Appendix A of the Foundational
Document.

[[Page 9957]]

Controlled Substances/Alcohol and Unsafe Driving

FMCSA conducted an Exploratory Factor Analysis (EFA) to identify
potential new groupings of violations by highlighting statistical
relationships between the violations in each BASIC. Controlled
Substances/Alcohol has the fewest violations of any BASIC, and those
violations are also cited relatively infrequently. The EFA demonstrated
that controlled substances and alcohol violations were strongly
associated with the Unsafe Driving BASIC and supported removing the
Controlled Substances/Alcohol category as a standalone BASIC. The new
Unsafe Driving safety category now includes the drug and alcohol
violations that were previously captured in the Controlled Substances/
Alcohol BASIC.
In addition, FMCSA's analysis found that violations for operating
while under an OOS Order issued under the Commercial Vehicle Safety
Alliance North American Standard OOS Criteria belong in the new Unsafe
Driving safety category. Currently, SMS places these types of
violations across multiple BASICs based on the underlying OOS
violation. For example, a carrier that had a violation cited against
its driver who operated after being placed OOS for an HOS violation and
another driver who operated after being placed OOS for a vehicle
violation would now have both violations placed in the new Unsafe
Driving safety category, rather than one in the HOS Compliance safety
category and the other in the Vehicle Maintenance safety category.
Moving and consolidating these violations to the new Unsafe Driving
safety category would allow motor carriers and enforcement officials to
more effectively identify and correct driver-based safety problems
related to disregarding OOS Orders.
FMCSA's evaluation of the new Unsafe Driving safety category,
illustrated in the table below, showed that this new combined safety
category identified more carriers for intervention that were involved
in more crashes and had a higher crash rate and A/C violation rate than
the groups of carriers identified in the current Unsafe Driving and
Controlled Substances/Alcohol BASICs.

Vehicle Maintenance

Vehicle Maintenance is the largest BASIC in terms of both the
number of violation identifiers (i.e., CFR provisions or unique
enforcement codes) included in the BASIC and the number of violations
cited during inspections. The EFA results showed that breaking this
category into two separate categories would provide greater specificity
to help carriers improve and enforcement officials to conduct targeted
investigations.
Therefore, Vehicle Maintenance violations would be divided into two
separate categories: Vehicle Maintenance: Driver Observed, which
includes violations that may be identified by a driver during a pre- or
post-trip inspection and/or while operating the vehicle; and Vehicle
Maintenance, which includes all other vehicle maintenance violations.
FMCSA's evaluation showed that although splitting Vehicle
Maintenance into two separate categories identifies groups of carriers
with a lower crash rate in each category, more carriers with more
crashes are identified for intervention and those carriers have a very
similar A/C violation rate, as illustrated in the table below.

----------------------------------------------------------------------------------------------------------------
A/C violation
Crash rate rate
Category (crashes per Number of (violations per Number of
100 PUs) crashes 100 carriers
investigations)
----------------------------------------------------------------------------------------------------------------
Current Vehicle Maintenance BASIC.............. 8.06 23,675 108.4 18,764
Proposed Vehicle Maintenance Safety Category... 7.55 19,039 103.8 11,019
Proposed Vehicle Maintenance: Driver Observed 7.44 23,618 109.7 17,167
Safety Category...............................
Combined Proposed Vehicle Maintenance and/or 7.47 31,666 107.1 22,092
Proposed Vehicle Maintenance: Driver Observed
Safety Category*..............................
----------------------------------------------------------------------------------------------------------------
* Carriers in this row have percentiles above the 80th percentile threshold in one or both proposed new Vehicle
Maintenance safety categories. This row is not the sum of the prior two rows since some carriers are
prioritized under both new safety categories.

Segmentation in Driver Fitness and HM Compliance

SMS accounts for differences in carrier operations in the Unsafe
Driving and Crash Indicator BASICs by segmenting carriers according to
whether they primarily operate Combination vehicles (i.e., more than 70
percent of their total PUs) or Straight vehicles. Carriers that are not
considered Combination carriers are considered Straight carriers. This
segmentation ensures that carriers are compared to other carriers with
fundamentally similar exposure to crash risk when operating their
vehicles. FMCSA tested whether applying segmentation to other safety
categories would improve the identification of the highest risk
carriers in those categories. Based on its analysis, FMCSA proposes to
segment the Driver Fitness and HM Compliance safety categories to more
effectively pinpoint safety issues relating to each operation type.
FMCSA determined that segmenting HOS Compliance, Vehicle Maintenance:
Driver Observed, and Vehicle Maintenance would not improve those safety
categories.
In the Driver Fitness BASIC, carriers that operate Straight trucks
and similar vehicles have much higher violation rates than motor
carriers that operate

[[Page 9958]]

Combination vehicles. Segmenting the Driver Fitness BASIC into Straight
and Combination segments more effectively identifies carriers with
higher crash rates in both segments. Although fewer carriers were
prioritized for intervention in the Driver Fitness BASIC, the carriers
that were removed from prioritization had a lower crash rate, which
allows the Agency to better focus on those carriers that pose a higher
risk to public safety.
The current HM Compliance BASIC compares Cargo Tank carriers to
non-Cargo Tank carriers, but these carriers have fundamentally
different operations. A carrier is categorized as a Cargo Tank carrier
for purposes of segmentation if more than 50 percent of its inspections
indicated the vehicles were Cargo Tanks. FMCSA's analysis found that
segmenting carriers as Cargo Tank carriers and Non-Cargo Tank carriers
in the HM Compliance safety category in conjunction with adjusting the
HM Compliance threshold from the 80th to 90th percentile identifies a
group of carriers that has (1) an HM inspection violation rate that is
22 percent higher and (2) an HM A/C violation rate that is 46 percent
higher than carriers identified for intervention under the current HM
Compliance BASIC.

Consolidated Violations

Over the past decade, the number of CFR provisions or distinct
enforcement codes used as violations in SMS has grown from about 650
violations to 959 violations. Most of the new violation codes provide
more specific descriptions for existing violations and do not reflect
new Federal safety regulations. For example, an inspector could cite an
inoperative vehicle brake by citing Sec. Sec. 393.48(a) (Inoperative/
defective brakes), 393.45UV (Brake tubing and hose adequacy under
vehicle), or 393.45PC (Brake tubing and hose adequacy--connections to
power unit).
FMCSA's analysis during IRT modeling confirmed that similar
violation provisions could be consolidated to mitigate differences that
result from inspectors citing different violation codes. Grouping
similar violations together would also allow motor carriers and
enforcement officials to identify and address specific safety issues
more easily. The following table shows a summary of the consolidated
violations by safety category.

------------------------------------------------------------------------
Number of Number of
violation consolidated
Violations in category provisions/codes groups in new
in SMS system
------------------------------------------------------------------------
Unsafe Driving.................. * 59 32
HOS Compliance.................. 73 9
Vehicle Maintenance............. 406 15
Vehicle Maintenance: Driver N/A 35
Observed.......................
Controlled Substance/Alcohol.... 11 N/A
HM Compliance................... 369 14
Driver Fitness.................. 55 11
---------------------------------------
Total....................... 973 116
------------------------------------------------------------------------
* Number includes 14 additional violations for operating while under an
OOS Order that are not used in the current SMS methodology.

A report titled, ``New Prioritization System: Proposed Violation
Groups,'' which maps the consolidation of the violations, is available
in the docket for this notice.

Severity Weights

SMS assigns each violation a specific severity weight that is
intended to correlate with the crash risk associated with that
violation. The assignment of severity weights to violations in SMS on a
scale of 1 through 10 has been criticized as overly subjective. FMCSA
tested many different models to improve the severity weights attached
to violations in SMS, including models that applied regression analysis
and IRT. Based on that analysis, FMCSA proposes to simplify violation
severity weights by assigning each consolidated violation group a
weight of either one or two. OOS violations and violations in the
Unsafe Driving safety category that are disqualifying offenses under 49
CFR 383.51 would be assigned a weight of two and all other violations
would be assigned a weight of one. If an OOS violation is combined with
a non-OOS violation in the consolidated violation grouping, the
consolidated group would be assigned the higher weight of two.
FMCSA's evaluation found that simplifying the severity weights
identifies carriers with higher crash rates. This change would maintain
the safety focus on those violations severe enough to result in an OOS
Order while removing the subjectivity and complications of
distinguishing each violation by severity on a scale of 1 through 10.

Proportionate Percentiles

FMCSA places motor carriers into safety event groups in SMS based
on their number of inspections and crashes. For example, carriers in
the HOS Compliance BASIC with 3 to 10 driver inspections are compared
to each other, while carriers with 11 to 20 driver inspections are
compared to each other, and so forth. SMS uses violations and crashes
to calculate a quantifiable ``measure'' of a motor carrier's safety
performance. SMS then ranks carriers within safety event groups by
assigning each carrier in the safety event group a percentile rank that
compares their measure to the measure of other carriers in the same
safety event group. A higher percentile rank in a BASIC indicates that
a carrier has a worse measure than other carriers in that safety event
group. Safety event groups allow FMCSA to provide safety oversight of
carriers of all sizes. Some carriers, however, have experienced large
percentile jumps based solely on a no-violation inspection that places
them in a new safety event group.
FMCSA proposes to use a new method of ``proportionate percentiles''
that will remove sudden jumps in percentiles, which can occur when a
carrier moves into a different safety event group. By removing those
percentile jumps, FMCSA would be able to more accurately evaluate
whether a carrier's safety performance is improving or declining from
month to month. The proportionate percentile approach would use safety
event groups only to calculate the benchmark median value of each
grouping, which would be calculated periodically. A carrier's
proportionate percentile would be calculated from a weighted average of
percentiles based on those benchmark medians. After the benchmark run
has been established, any changes to a carrier's percentile would be
based

[[Page 9959]]

solely on the carrier's own safety performance and would not be
impacted by the safety performance of other carriers.
The table below provides an illustration of how proportionate
percentiles more accurately reflect a carrier's change in safety
performance. For a detailed description of the method used to calculate
the proportionate percentiles, see the Foundational Document in the
docket for this notice.

------------------------------------------------------------------------
Current Proposed
Example carrier methodology methodology
------------------------------------------------------------------------
Carrier with 10 inspections..... Measure: 1.51..... Measure: 1.51.
Percentile: 53.0%. Percentile: 67.4%.
Same carrier after receiving 1 Measure: 1.37..... Measure: 1.37.
additional inspection with no ([darr] .14)...... ([darr] .14).
violations (and moving to next
largest safety event group,
with 11 total inspections).
Percentile: 75.0%. Percentile: 67.0%.
([uarr] 22%)...... ([darr] .4%).
------------------------------------------------------------------------

FMCSA's analysis showed that this approach would reduce the number
of unexpected jumps in a carrier's percentiles. In addition, the
proportionate percentile method would more closely align a carrier's
percentile ranking to changes in its safety performance, ensure stable
monthly results for carriers, and provide customized results that are
specific to the carrier's exact number of inspections or crashes.

Improved Intervention Thresholds

FMCSA prioritizes carriers for safety interventions when their SMS
percentiles reach or exceed pre-established levels called Intervention
Thresholds. Because higher percentiles represent worse safety
performance, a lower Intervention Threshold in a BASIC represents a
more stringent safety criterion. FMCSA's ET found that the Unsafe
Driving, Crash Indicator, and HOS Compliance BASICs have the strongest
correlation to crash risk. Therefore, those BASICs have lower
Intervention Thresholds than the other BASICs, at 65 percent for
property carriers, 60 percent for HM carriers, and 50 percent for
passenger carriers. The Intervention Thresholds for the Vehicle
Maintenance, Controlled Substances/Alcohol, and Driver Fitness BASICs
currently are set at 80 percent for property carriers, 75 percent for
HM carriers, and 65 percent for passenger carriers, and the HM
Compliance Intervention Thresholds are set at 80 percent for all
carriers.
FMCSA examined whether adjusting the Intervention Thresholds for
the Driver Fitness, HM Compliance, Vehicle Maintenance, and Vehicle
Maintenance: Driver Observed safety categories could improve the
Agency's focus on carriers with the highest crash risk. FMCSA's updated
ET continues to show that the Driver Fitness and HM Compliance safety
categories have the lowest correlation to crash risk. FMCSA believes
raising the Intervention Thresholds in those safety categories, as
shown in the table below, would allow the Agency to focus on
populations with a greater safety risk.
FMCSA also considered lowering the Intervention Thresholds in the
Vehicle Maintenance and Vehicle Maintenance: Driver Observed safety
categories. However, because the Agency is now proposing to split
Vehicle Maintenance into two safety categories, FMCSA determined that
more carriers would be prioritized for vehicle maintenance issues by
applying the current Intervention Thresholds to the new Vehicle
Maintenance and Vehicle Maintenance: Driver Observed safety categories
than are prioritized in the current Vehicle Maintenance BASIC. FMCSA,
therefore, does not propose to change the Intervention Thresholds for
the Vehicle Maintenance safety categories, as shown in the table below.

--------------------------------------------------------------------------------------------------------------------------------------------------------
Current intervention thresholds Proposed intervention thresholds
-----------------------------------------------------------------------------------------------
Category Passenger Passenger
carrier HM General carrier HM General
--------------------------------------------------------------------------------------------------------------------------------------------------------
Vehicle Maintenance..................................... 65 75 80 65 75 80
Vehicle Maintenance: Driver Observed.................... N/A N/A N/A 65 75 80
HM Compliance........................................... 80 80 80 90 90 90
Driver Fitness.......................................... 65 75 80 75 85 90
--------------------------------------------------------------------------------------------------------------------------------------------------------

Focusing on Recent Violations

SMS currently assigns percentiles in the HOS Compliance, Vehicle
Maintenance, and Driver Fitness BASICs if the last inspection in the
past two years resulted in a violation. Under this standard, a carrier
may be prioritized for intervention even if the carrier had no recent
violation. FMCSA proposes to sharpen the focus on carriers with more
recent violations by assigning percentiles only to carriers that had at
least one violation in the safety category in the past 12 months. This
change means that if all a carrier's violations in a particular safety
category are 12 months or older, the carrier will not be assigned a
percentile in that category.
FMCSA's evaluation showed that this change would result in 1,081
carriers no longer having a safety category at or above the
Intervention Threshold and that those carriers had a crash rate that
was 13 percent lower than the national average. Removing carriers with
no recent violation in those safety categories would allow the Agency
to focus its resources on carriers that pose a greater safety risk.

Updated Utilization Factor

The Utilization Factor in SMS helps to account for a carrier's
exposure in the Unsafe Driving and Crash Indicator BASICs. Carriers
with higher-than-average exposure to safety events, as measured by VMT
per PUs, receive an adjustment in those BASICs. The Utilization Factor
currently covers carriers that drive up to 200,000 VMT per PU per year.
FMCSA's analysis found that more carriers are reporting higher VMT now
than when the Utilization Factor was developed in 2009, and the 314
carriers with 200,000 to 250,000 VMT per PU were involved in about
three times as many inspections per PU than the national average. This
result indicates that these carriers exhibit much higher exposure to
inspections than most carriers. FMCSA

[[Page 9960]]

proposes to extend the Utilization Factor to carriers that drive up to
250,000 VMT per PU in the Unsafe Driving and Crash Indicator safety
categories to more accurately account for carriers with increased
exposure.

Other Changes Considered and Not Proposed

FMCSA analyzed other potential changes to SMS and determined that
they would not improve safety, as described below.

Geographic Variation

A consistent criticism of SMS has been that differences among State
enforcement agencies in commercial motor vehicle (CMV) inspection and
violation rates may lead to unfair SMS results for carriers that
operate primarily in States with higher-than-average enforcement rates.
During the IRT model design, FMCSA explored a statistical model to
better account for enforcement variation among States. That model is
detailed in the report titled ``Development and Evaluation of an Item
Response Theory (IRT) Model for Motor Carrier Prioritization,'' which
is available in the docket for this notice.
FMCSA determined that incorporating a model to account for
geographic variation would not improve the Agency's ability to identify
high risk carriers and would run contrary to the goals of the Motor
Carrier Safety Assistance Program (MCSAP), the Agency's grant program
to support State and local efforts to reduce crashes involving CMVs.
States face varying challenges to reducing crashes due to different
road types, congestion, topography, and weather conditions, among other
factors. Through MCSAP, FMCSA encourages States to tailor their crash
reduction strategies by addressing local conditions and challenges.
Applying a model that de-emphasizes enforcement in certain States would
disincentivize FMCSA's MCSAP partners from undertaking enforcement
initiatives that are intended to address particular safety issues in
their States. FMCSA believes that it should encourage all States to
continually raise the bar for safety rather than discounting the safety
efforts of certain States.

Crash Indicator

The Crash Indicator BASIC applies severity weights to reportable
crashes and places more weight on crashes involving an injury or
fatality and crashes involving the release of HM than on tow-away
crashes. FMCSA analyzed whether removing severity weights to simplify
the calculation would improve this BASIC. Because removing the severity
weights from the Crash Indicator BASIC has a minimal impact on the
group of carriers identified for intervention, FMCSA does not propose
to make this change.
FMCSA also studied the impact of raising the minimum number of
crashes required to assign a percentile in the Crash Indicator BASIC
from two to three. FMCSA's ET results, however, showed that carriers
with exactly two crashes have a future crash rate that is more than
twice the national average future crash rate. Approximately two-thirds
of those carriers were not prioritized in another BASIC, meaning they
would not receive any safety interventions from FMCSA if the data
sufficiency standard in the Crash Indicator BASIC were increased from
two to three crashes. FMCSA has concluded that raising the minimum
number of crashes from two to three in the Crash Indicator BASIC would
not improve safety. Crashes that are reviewed through FMCSA's Crash
Preventability Determination Program and found to be Not Preventable
will continue to be excluded from the prioritization methodology.

Preview

With the February 2023 SMS update, the Agency provided a preview
opportunity of the system before implementation, as it has historically
done with SMS implementation and enhancements, to allow motor carriers,
law enforcement, and other interested stakeholders to see the impacts
of these proposed changes on measures, percentiles, and alerts. Motor
carriers can log in to the preview at https://csa.fmcsa.dot.gov/prioritizationpreview/ or through the CSA website or the FMCSA Portal
to see how the proposed methodology may impact their prioritization
results. The public can view the new methodology using an example
carrier. To support the preview, FMCSA will hold a series of question
and answer (Q&A) sessions for the industry and the public, where
participants will be able to ask questions about the proposed changes
and receive real-time responses. All sessions will have closed
captioning. The dates and times for these sessions will be announced on
the Agency's website. Before the Q&A sessions, participants have the
opportunity to view the preview website and additional resources at
https://csa.fmcsa.dot.gov/prioritizationpreview/_ where they can learn
more about the proposed changes and review their results under the
proposed methodology. FMCSA encourages all stakeholders to participate
in these Q&A sessions.
FMCSA requests comments on the above proposed enhancements, as well
as the changes that were considered but are not proposed. In addition,
input is requested on other changes that should be considered.
Submitters should provide data to support their recommendations.

Robin Hutcheson,
Administrator.
[FR Doc. 2023-02947 Filed 2-14-23; 8:45 am]
BILLING CODE 4910-EX-P