NASA Programmer Recalls Debugging Lisp in Deep Space
A tale of ingenuity,perseverance, and the enduring power of Lisp in the face of unexpected challenges.
In the vast expanse of space, where communication is measured in light-years and every line of code carries immense weight, a story of ingenuity and perseverance unfolds.Ron Garret, a former NASA programmer, recently shared his experience debugging a Lisp software malfunction during a deep space mission on the Corecursive podcast. His taleoffers a glimpse into the unique challenges of coding for spacecraft and the enduring power of Lisp, a language often overlooked but still relevant in modern software development.
Garret, a research scientist at NASA’s Jet Propulsion Laboratory from 1988 to 2000, and again from 2001 to 2004, specialized in autonomous mobile robots. He played a crucial role in developing the control architecture for the Sojourner rover, apioneering effort in the field of robotic exploration.
During his time at NASA, Garret witnessed the evolution of programming languages, transitioning from the limited options of the 1980s to the more sophisticated tools of today. He recalls the era before Java, Python, JavaScript, and even C++, wherePascal, C, Basic, and assembly language were the dominant forces. It was very, very hard to get anything done with any of those languages, he explained.
Enter Lisp, a language that offered a different approach, abstracting problems into lists and functions. While C programmers grappled with issues likedangling pointers, Lisp provided automatic memory management. When you have a language that gives you some higher-level abstractions, it’s much faster and easier to get things done, Garret said. In a world where Lisp was the only language that had that capability, it was really like a superpower.
Garret’s team embraced Lisp for its ability to create custom languages tailored to specific problems, particularly valuable for the memory-constrained hardware of their robots. As Garret put it, Every problem became a compiler problem. They meticulously developed and tested their code on a Macintosh computer, then transferred it to theactual rover for a grueling test drive on the Arroyo.
Despite their efforts, the Sojourner rover ultimately relied on C code when it landed on Mars. However, in 1998, a new NASA administrator launched the New Millennium Program, a series of deep space missions designed to test different, more cost-effective technologies. This presented a second chance for Garret’s Lisp code. The team’s autonomous technology, originally developed for the rover, was repurposed for a new mission – the flight controller.
Garret’s team created a novel decision-making software, meticulously designed ina custom Lisp language to avoid the dreaded race conditions – situations where multiple threads compete for the same memory space. They rigorously tested their code on the exact hardware that would be used in space, ensuring confidence in its success.
However, during the three-day flight control, disaster struck. It ran for a while, did what it was supposed to do, and then it just died, alarms went off…, Garret recounted. Now, this code that was proven to be deadlock-free seemed to be frozen 1.5 million miles from home.
Tension mounted as the team grappledwith the unknown. They waited an hour, carefully crafting their commands, which underwent multiple levels of review and approval before being sent through a dedicated line to a 70-meter antenna in the Deep Space Network. The commands then traveled at the speed of light, across the vast expanse of space.
First, they requested a traceback, a common procedure that generates a list of all active processes, revealing their current state. Almost immediately, you could see the problem, because there was one process waiting for something that was supposed to happen…, Garret explained.
The culprit was a race condition that, theoretically,shouldn’t have existed. One of Garret’s programmers had inadvertently created a security hole in their carefully crafted language by calling a lower-level Lisp function. The team decided to manually trigger the event, forcing the software to restart.
We didn’t lose the spacecraft, we accomplishedall the missions – technically it was a success, Garret said on the podcast. But the process was so excruciating, so difficult – and then, there’s the politics. Even though we did get it working, the autonomous project was canceled, it never flew again.
In a 2002 article on his personal website, Garret lamented the demise of Lisp at JPL, arguing that the language was uniquely suited for developing unique, highly dynamic applications under tight budget and schedule constraints. He believed that Lisp’s neglect in favor of C++, Java, and other languages stemmed from an overemphasison best practices, which he felt were often confused with standard practices.
Garret emphasized that best is not a static standard, but rather a context-dependent concept that should be tailored to the specific needs of each project. In a discussion on Hacker News, a commenter claiming to be aNASA engineer who worked on the Lunar South Pole mission in 2009 stated that they used Lisp to create their own custom language for instrument command sequences and simulations. The simple, flexible syntax and macros of Lisp made it easier to express commands and timing patterns, they wrote. This suggests that Lisp mightstill be used in various corners of NASA, defying Garret’s concerns.
Garret’s story serves as a reminder that even in the face of seemingly insurmountable challenges, ingenuity and perseverance can prevail. Lisp, a language often dismissed as outdated or esoteric, demonstrated its enduring power in a high-stakesenvironment, proving its value even in the face of unexpected obstacles. As technology continues to evolve, the lessons learned from this deep space adventure offer valuable insights into the importance of embracing diverse tools and approaches, while always striving for innovation and adaptability.
Views: 0