Reading, MA 01867

According to a presentation (December 15, 2016) by the Automated Vehicles Working  Group (AVWG) of the Massachusetts Department of Transportation (MASS DOT), there is a requirement for technical experts to support this group:

• Provide input guidance to MASSDOT for safe testing of AV technologies
• Ensure that AVs, which have completed testing, can be operated safely to advance the welfare of Massachusetts residents
• Follow developments in AV technology, Federal policy and laws/policies in other states
• Facilitate the widespread deployment of Automated Vehicles (AV) in Massachusetts.

Several other states have similar interests in Autonomous Vehicles.

I can support these requirements based on my technical background and program management experience. The latter stems from programs awarded to me by the US Department of Transportation (US DOT).

Crash Warning Systems: I have extensive contractual experience as Principal Investigator (PI) with vehicle crash warning systems for Run-Off-Road (ROR),and Intersection collisions for automobiles, as well as crash warning systems for inner city transit buses. This work was a precursor development leading to driverless vehicles. Contracts for these programs were awarded by the National Highway Traffic Safety Administration (NHTSA) and the Federal Transit Administration (FTA). The first two programs were performed at Battelle Memorial Institute as a subcontractor to Carnegie Mellon University, while the third was undertaken at Foster-Miller, Inc. as a subcontractor to Calspan.

On Board Vehicle Sensors: Sensors for these vehicles were selected from visual/infrared, acoustic, radar, and Light Detection and Ranging (LIDAR) technologies. Data from these sensors served as input to onboard computers equipped with algorithms designed to issue warnings for crash avoidance. Warning modalities included visual, audio and haptic (i.e., vibrating driver seat or steering wheel).

Computer Simulations: The ROR project involved a computer simulation to test various driver warning algorithms. The simulation included human factor inputs for driver steering, throttling and braking. The simulated automobile was a Ford Taurus that was operated on a roadway designed with various curves and straight segments.

Test Vehicle Design: During the ROR program, I was involved with the design of a test vehicle. This included:

• Review state-of-the-art sensing, processing and driver interface technologies for their applicability to Run-Off-Road collision prevention

• Design an advanced test bed vehicle for evaluating alternative countermeasures

Driver Training Simulator for Warning Algorithm Evaluation: My work on inner city transit buses involved a test of various warning algorithms integrated within a driver training simulator utilized by the New York City Transit Authority. Transit operators were recruited as study participants. My work also included a statistical analysis of transit operator responses to alerts as a function of warning modality and timing with transit operator experience, age and gender as control parameters.

Sensor Performance During Inclement Weather: A serious issue for driverless vehicles is operation during inclement weather. I worked on this problem in relation to sensor performance and weather effects modeling.

My Experience with Autonomous Vehicles: According to the Society of Automotive Engineers (SAE), there are 6 levels of vehicle automation, starting with 0 (i.e., no automation) and ending with 5 (i.e., complete system level automation with no driver involvement).** My work described above spans levels 0-2, and overlaps levels 3-5.

My Future Work: I continue to follow driverless vehicle developments regarding Testing, Technology, Machine Learning, Policy, Weather and Cyber Security.

*PhD, Physics, Boston College

** www.massdot.gov, op. cit. p 15.

According to the Guardian, “The FBI believes the “game changing” vehicle (i.e., driverless vehicles) could revolutionize high-speed car chases within a matter of years. The report also warned that autonomous cars may be used as ‘lethal weapons'”. This blog post presents findings that support the FBI’s apprehensions. Driverless vehicles could become the weapon of choice for terrorists by eliminating the need for suicide drivers.

Many late model vehicles are equipped with a variety of technologies that support driver assistance, lane position monitoring, emergency braking and infotainment. Some of these subsystems will undoubtedly lead to driverless vehicles that are under development.

According to a report written for Senator Markey (D-MA) , “The proliferation of these technologies raises concerns about the ability of hackers to gain access and control to the essential functions and features of those cars and for others to utilize information on drivers’ habits for commercial purposes without the drivers’ knowledge or consent.”

“Senator Markey sent letters to the major automobile manufacturers to learn how prevalent these technologies are, what is being done to secure them against hacking attacks, and how personal driving information is managed…These letters were sent to16 major automobile manufacturers: BMW, Chrysler, Ford, General Motors, Honda, Hyundai, Jaguar Land Rover, Mazda, Mercedes-Benz, Mitsubishi, Nissan, Porsche, Subaru, Toyota, Volkswagen (with Audi), and Volvo. Letters were also sent to Aston Martin, Lamborghini, and Tesla, but those manufacturers did not respond.”

Here is a summary of the manufacturers’ responses taken from the Markey Report:

1. Nearly 100% of cars on the market include wireless technologies that could pose vulnerabilities to hacking or privacy intrusions.

2. Most automobile manufacturers were unaware of or unable to report on past hacking incidents.

3. Security measures to prevent remote access to vehicle electronics are inconsistent and haphazard across all automobile manufacturers, and many manufacturers did not seem to understand the questions posed by Senator Markey.

4. Only two automobile manufacturers were able to describe any capabilities to diagnose or meaningfully respond to an infiltration in real-time, and most say they rely on technologies that cannot be used for this purpose at all.

5. Automobile manufacturers collect large amounts of data on driving history and vehicle performance.

6. A majority of automakers offer technologies that collect and wirelessly transmit driving history data to data centers, including third-party data centers, and most do not describe effective means to secure the data.

7. Manufacturers use personal vehicle data in various ways, often vaguely to “improve the customer experience” and usually involving third parties, and retention policies – how long they store information about drivers – vary considerably among manufacturers.

8. Customers are often not explicitly made aware of data collection and, when they are, they often cannot opt out without disabling valuable features, such as navigation.

SUMMARY: The Markey report noted, “These findings reveal that there is a clear lack of appropriate security measures to protect drivers against hackers who may be able to take control of a vehicle or against those who may wish to collect and use personal driver information…In response to the privacy concerns raised by Senator Markey and others, the two major coalitions of automobile manufacturers recently issued a voluntary set of privacy principles by which their members have agreed to abide.” It remains to be seen how effective these “voluntary principles” will be.

Based on an article written in March 29, 2015, “Last week, Tesla Motors unveiled another first for the auto industry: starting immediately, the company will be delivering upgrades directly to vehicles via the Internet

Tesla plans to distribute these upgrades over a period of months. “Unlike most of the auto industry’s upgrades, which are delivered to customers through an independent dealer network, Tesla is building on a sales and marketing philosophy that cuts out the middleman by sending the new software directly to its cars over their embedded wireless connections.”

However, Congress is concerned about the vulnerability of wirelessly transmitted up- grades. In 2015, “Senator Edward Markey (D-MA) sent a letter to 20 car manufacturers asking them about their vehicles’ reliance on wireless computing technology and, in turn, the vulnerability of their systems. In February, he published the companies’ replies, and they weren’t completely reassuring (the full report is here).

“Manufacturers that responded to the Senator’s inquiry gave mostly ambiguous answers about the cyber security of their products. Some said they encrypt information such as driving history and physical location, while others admitted that they don’t use encryption. The same is true for third-party testing of vehicle cyber security—some do it, but many do not.”

Note: “Tesla was one of three companies that chose not to respond to Sen. Markey’s questions.”

In the meantime, “The Alliance of Automobile Manufacturers, composed of 12 automakers, and the Association of Global Automakers, comprising 12 manufacturers and five suppliers, have developed a framework for automotive cyber security best practices.”

• What has this Alliance achieved to date on driverless vehicle cyber security?
• Who is responsible for checking these accomplishments?
• Has anyone from this Alliance responded clearly to Senator Markey’s questions?

Driverless vehicles are equipped with several sensors to monitor the roadway and surrounding environment. These sensors include radar on the front end, a forward-looking video camera, a roof mounted LiDAR sensor, rear-mounted ultrasonic sensors, also another radar sensor affixed to the rear of the vehicle and GPS for vehicle location data. An onboard computer processes these sensor data to provide navigational/control signals for vehicle guidance. This sensor configuration is illustrated in the following link.

Rather than relying on an extensive set of if-then rules applied to these sensor data, the onboard algorithm is trained on a vast set of traffic situations. This procedure is explained as follows. “… the programmers feed the software with many traffic situations and specify the correct action for each situation. The program then searches by itself for the best configuration of internal parameters and internal decision logic, which allow it to act correctly in all of these situations. Like with us humans, it then becomes difficult to answer the question why the car exhibits a specific behavior in a new situation: no “explicit rules” have been specified; the decision results from the many traffic situations to which the algorithm had been exposed beforehand.”

Before my family or I use a driverless vehicle, I would appreciate convincing answers to the following questions:

1. How many sensors adequately characterize the vehicle trajectory and its highway environment? What does adequate mean? How can one answer this question?

2. How many highway traffic scenarios should be presented to this driverless vehicle-learning algorithm? 1000? 10,000? How does one know?

3. How well should this algorithm learn? Provide correct vehicle control signals 95 percent of the time? 99 percent? What is the test for judging learning performance?

4. Google is gathering vast amounts of highway data in support of Question #2. Although Google is an impressive organization, who evaluates their learning algorithm performance?

5. What does the National Highway Traffic Safety Administration(NHTSA) have to say on this matter?

6. What about insurance companies and lawyers? What are their positions on driverless vehicle safety performance?

According to CBS News, “Companies are aggressively recruiting retirees. Drivers more than 65 years old make up about 10 percent of commercial vehicle operators in the U.S. A five-month investigation by CBS News looks at how the increase in older drivers translates to potential danger on the nation’s highways…Individuals are working well past the retirement age of 65. But as the industry has changed, the rules of the roads have not kept up with the times — raising the question: Is more screening needed for commercial drivers?”

The Hill Law Firm noted, “A recent NHTSA data analysis by CBS News looked at truck crashes for the past three years in 12 states and found a 19 percent increase in accidents involving commercial truck and bus drivers who were more than 70 years of age. In all, between 2013-2015, more than 6,600 trucks involved drivers in that age range. (Note: The National Highway Traffic Safety Administration [NHTSA])

Possible Solution: The accident problem caused by older truck drivers could be mitigated with the advent of self-driving trucks. This statement is quite plausible given the advantages cited for driverless cars. For example, “There’s potential in efficiency, in terms of better traffic flow, but also less fuel consumption…Because cars will be automated, there will be less chance of accidents caused by human error, leading to less traffic congestion.”

The headlines below show a distinct trend toward the deployment of this driverless truck technology innovation:

• Mercedes-Benz Shows Off Self-Driving 18-Wheeler – Sep 30, 2014
• The First Self-Driving 18-Wheeler Hits the Highways – May 6, 2015
• A Fleet of Self-Driving Trucks Rumbles Across Europe – Apr 7, 2016

Opposition to Possible Solution:

There are at least 2 reasons why the commercialization of driverless trucks will be a protracted process:

1. “There are more than 37,000 members of the American Trucking Associations (ATA) covering every type of motor carrier in the United States.” This group will undoubtedly seek to preserve truck driver jobs by influencing Congress.

2. In the United States, there are approximately 2 million tractor-trailer units. They will be replaced slowly over many years as they age. Thus, replacement by driverless trucks will be slow on a national scale.

Note: “The mandatory retirement age of airline pilots is 65. The Fair Treatment for Experienced Pilots Act (Public Law 110-135) went into effect on December 13, 2007, raising the age to 65 from the previous 60.” A similar provision could be imposed upon truck drivers in the interest of public safety. 

During his post-election speech on November 9^th, President-elect Donald Trump stated, “We are going to fix our inner cities and rebuild our highways, bridges, tunnels, airports, schools, hospitals. We’re going to rebuild our infrastructure, which will become, by the way, second to none. And we will put millions of our people to work as we rebuild it.”

In a previous post, I wrote, “Failure to invest in roadway infrastructure in the United States may delay the ultimate commercialization of driverless vehicles. “An estimated 65 percent of U.S. roads are in poor condition, according to the U.S. Department of Transportation, with the transportation infrastructure system rated 12th in the World Economic Forum’s 2014-2015 global competitiveness report.”

“The Huffington Post noted, ‘Shoddy infrastructure has become a roadblock to the development of self-driving cars, vexing engineers and adding time and cost. Poor markings and uneven signage on the 3 million miles of paved roads in the United States are forcing automakers to develop more sophisticated sensors and maps to compensate, industry executives say.’ More advanced sensors will add more cost to driverless vehicles.”

As for federal support of driverless vehicles, “The government’s new support includes a request in President Obama’s proposed budget for the next fiscal year for $4 billion, to be spent over 10 years, to finance research projects and infrastructure improvements tied to driverless cars.” It remains to be seen whether Donald Trump puts this level of spending on a ‘fast track’ to accelerate driverless vehicle development.

There was a tragic crash of a Model S Tesla on May 7, 2016 in Williston, Florida. The driver, Joshua D. Brown, died from this accident.

According to Tesla, “ What we know is that the vehicle was on a divided highway with Autopilot engaged when a tractor trailer drove across the highway perpendicular to the Model S”… “Neither Autopilot nor the driver noticed the white side of the tractor trailer against a brightly lit sky, so the brake was not applied.” 

“The current vintage of Model S autopilot cars (Q4 2014 to Q1 2015) have one front radar, one front monocular Mobileye optical camera, and a 360-degree set of ultrasonic sensors.”

To what extent was this optical camera optimized to deal with a multitude of driving scenarios with varying contrasts due to the roadway, sky, cars, trucks, pedestrians and miscellaneous objects?

I addressed this contrast problem in a paper that was published in 1994 (Proc. Of SPIE, Photonics for Electronic Products, Vol. 2544, Boston, MA, 1994). The subject of this paper was contrast optimization of optical systems for Run-Off-Road crash avoidance.

According to my paper, “Contrast is determined by material properties affecting reflected and radiated intensities, as well as weather and visibility conditions. This paper discusses the modeling of these parameters and characterizes the contrast performance effects due to reduced visibility. The analysis process first involves generation of inherent road and off-road contrasts, followed by weather effects as a contrast modification…The results of the sensor/weather modeling will be used to predict the performance off an in-vehicle warning system under various levels of adverse weather.”

“Varying contrast results were generated for…original image (a divided four lane highway) and three sets of corresponding images related to fog, medium rain and fog plus light rain…The degraded images…included atmospheric effects due to transmission loss and the addition of path radiance… These contrast plots were created to underscore the need for an optimum sensor waveband for run-off-road collision avoidance.”

Contrasts can be considered as signal to noise ratios where higher ratios can be correlated with greater detection probabilities for cars, trucks, buses, pedestrians, etc. Such ratios could be included in detection warning algorithms for driverless vehicles. Low ratios could be used to trigger driver-warning algorithms.

• Optical Camera: To what extent, if any, was the Tesla Model S algorithm developed with detection probabilities for various driving scenarios?

• Who is responsible for testing optical camera detection algorithms?

• Who is responsible for assessing other sensor modality algorithms for radar, LIDAR and acoustic?

During 2016

According to Business Insider, “…19 companies are striving to put driverless cars on the road by 2020. These companies include Tesla, Google, Uber, Toyota, BMW, Volvo, Ford, General Motors and Nissan, among others.

During 1995

Much of the current driverless vehicle technology was pioneered by Carnegie Mellon University and culminated with a near driverless trip called, No Hands Across America. “During this tour of America, which was sponsored by Delco Electronics, AssistWare Technology, and Carnegie Mellon University, two researcher from CMU’s Robotics Institute “drove” from Pittsburgh, PA to San Diego, CA using the RALPH computer program.”

“RALPH (Rapidly Adapting Lateral Position Handler) uses video images to determine the location of the road ahead and the appropriate steering direction to keep the vehicle on the road. (The researchers handled the throttle and brake.)”

“RALPH decomposes the problem of steering a vehicle into three steps, 1) sampling of the image, 2) determining the road curvature, and 3) determining the lateral offset of the vehicle relative to the lane center. The output of the later two steps are combined into a steering command, which can be compared with the human driver’s current steering direction as part of a road departure warning system, or sent directly to the steering motor on our Navlab 5 testbed vehicle for autonomous steering control.”

J. H. Everson SBIR Consultant

The Hill recently reported, “NHTSA uses the Federal Aviation Administration (FAA) pre-market regulatory powers as an example of the type of process they would like to emulate…The FAA certification often lasts three to five years…That fact alone should foreclose further discussion about the wisdom of NHTSA employing an FAA-like pre-market approval regime for driverless cars.” As a worst case scenario, nearly 150,000 motorists would have died during a five-year approval review.  Note that there are currently more than 30,000 annual vehicle fatalities in the United States

What’s more important, expanding Government regulations or saving lives?

J. H. Everson SBIR Consultant

The Huffington Post noted, “Shoddy infrastructure has become a roadblock to the development of self-driving cars, vexing engineers and adding time and cost. Poor markings and uneven signage on the 3 million miles of paved roads in the United States are forcing automakers to develop more sophisticated sensors and maps to compensate, industry executives say.” More advanced sensors will add more cost to driverless vehicles.

New street materials could mitigate the need for expensive sensors. “…An easier fix might be customizing road materials to make streets more visible in all kinds of conditions. Roadways can also vary widely in terms of materials and signage. As driverless cars increase in popularity, a new set of road standards will emerge to ensure that street materials and markings are optimized for the new vehicles.”

Repaving 3 million miles of roadways for the benefit of driverless vehicles may slow the commercialization of this form of transportation to a crawl. This effect may delight the auto insurance industry, which could otherwise witness a huge loss of revenue because driverless vehicles are inherently safer than vehicles under human control. Thus, the demand for auto insurance may drastically plummet with the advent of driverless vehicles. This insurance issue was addressed in my previous post.

J. H. Everson SBIR Consultant