In aerospace engineering, safety is sacred. Every circuit, every system, every calculation must pass through the processes of safety assessment. Among the core activities in this rigorous ecosystem is the Reliability Prediction Report (RPR)—a quantitative, exhaustive analysis of the electronic components’ expected behavior and failure rates over time. At Anzen, we are always looking for ways to make safety smarter, faster, and more resilient. In collaboration with Tecnalia, we’ve undertaken a forward-looking innovation activity that explores the use of artificial intelligence (AI) to support engineers in generating RPRs based on the MIL-HDBK-217F standard. This initiative isn’t about replacing engineers with machines. It’s about finding the sweet spot where AI-Assisted Reliability Predictions can amplify human efficiency—all while keeping integrity, explainability, and certification compliance at the forefront.
Why Reliability Prediction Reports Are Critical
In aerospace systems engineering, RPRs are far more than documentation—they are the numerical backbone of safety assessments. Here’s why:
RPRs Serve as Input for:
- Failure Mode and Effects Analyses (FMEAs)
- Fault Tree Analyses (FTAs)
- System Safety Assessments (SSAs)
RPRs Quantify:
- Failure rates (λ) for each electronic component
- Environmental and stress factors based on mission profiles
- Component quality grades and stress derating
MIL-HDBK-217F Standard:
- The most widely used methodology for estimating electronic component reliability
- Defines component categories and stress models
- Used globally in both civil and military aerospace programs
But despite their importance, generating these reports is extremely time-consuming. Engineers must:
- Scrutinize every component in a Bill of Materials (BOM)
- Manually classify components based on text descriptions or datasheets
- Match those to the right failure models from MIL-HDBK-217F
- Input and cross-reference manufacturer specifications, part numbers, and datasheet details
The result? A slow, expert-driven process that’s hard to scale.
Innovation Through AI
This activity—developed alongside Tecnalia—was initiated as a research effort to explore how AI could streamline a traditionally labor-intensive process. While it’s not yet deployed in live projects, the findings are shaping the path forward.
Component classification is a simple but repetitive and time-consuming task, this factor makes it very appropriate for AI techniques. At its core, the question was simple:
Can artificial intelligence help engineers classify components according to MIL-HDBK-217F?
The answer? Yes, but only with the right guardrails. That’s where AI-Assisted Reliability Predictions begin to prove their worth.
Tackling the Technical Challenge: From Free Text to Classified Components
One of the main challenges we encountered was the unstructured nature of component descriptions. They’re written by manufacturers and distributors in varying formats, often full of abbreviations, non-standard wording, and overlapping terminology.
To address this, we developed two complementary approaches to classification:
🔹 Keyword-Based Classification
- Technique: Build a “bag of words” model for each MIL-HDBK-217F category
- Process:
- Extract most frequent keywords from sample components
- Apply ranking algorithms to determine best category match
- Performance:
- ~73% accuracy
✅ Pros: Simple, explainable
❌ Cons: Easily confused by synonymy, poor performance
🔹 Prompt-Based Large Language Models (LLMs)
- Models Tested: Llama 3.1 (8B and 70B parameters)
- Method:
- Zero-shot classification using carefully engineered prompts
- Supplied component description + standardized list of MIL-HDBK-217F categories
- Performance:
- Up to 95% accuracy using Llama 3.1 70B
✅ Pros: Accurate
❌ Cons: Computational costs are high
🎯 Key Findings:
- The prompt format significantly impacts accuracy
- The best results came from concise, controlled prompts with forced-format answers
- Some categories are easier to classify than others
Going Deeper: Second-Level Classification Isn’t So Easy
Classifying components into top-level groups (e.g., capacitors, resistors, diodes) is only half the battle. Sometimes MIL-HDBK-217F also demands a second level of classification—known as component “style”.
For example:
- A capacitor could be labeled as CDR, CWR, etc.
- A resistor might be RM, RZ, etc.
We tested a range of methods to assign these styles automatically using a combination of:
- BOM metadata
- Part numbers
- Descriptions
- Manufacturer datasheets
- Web-scraped product pages
Methods and Findings:
This time, we repeated the two approaches described above. Three approaches were tested—depending on the quantity of information provided to the AI.
Based on Component Description or Name Only:
- Poor precision (less than 10%) due to ambiguous or generic wording
- LLMs often confused by similar styles or acronyms
Based on Description + Other Selected Fields:
- Best balance between signal and noise
- Accuracy improved, but still low (less than 40%)
- Same results in LLM and Keyword-Based classification
Conclusion: While AI can help, second-level classification still requires engineer supervision, especially when descriptions are vague or conflicting.
Summary of What Worked (And What Didn’t)
Here’s a quick wrap-up of the key takeaways from this AI-Assisted Reliability Predictions activity:
✅ What Worked
- First-level classification using LLMs (Llama 3.1 70B) yielded >95% accuracy
- Prompting strategy was crucial to achieving high precision
- Combining LLMs with keyword methods created a flexible hybrid pipeline, reducing cost and maximizing accuracy
- Clear value in reducing cognitive load on engineers by narrowing down candidate categories
❌ What Didn’t Work (Yet)
- Second-level style classification suffered from:
- Ambiguous terminology
- Lack of consistent metadata
- LLM misinterpretation of subtle differences (e.g., CWR vs CDR)
AI as a Tool, Not a Replacement
Let’s be clear: AI doesn’t replace the reliability engineer.
What it can do is act as a smart assistant—processing large volumes of component data, suggesting likely classifications, and helping engineers make faster, more informed decisions.
But AI lacks:
- Contextual awareness of mission safety
- Understanding of nuanced design intent
- Accountability and traceability
That’s why a human-in-the-loop model is the only acceptable path forward—especially in regulated, safety-critical sectors like aerospace and defense.
Final Thoughts: Toward a New Era of Engineering Efficiency
This innovation activity—developed collaboratively by Anzen and Tecnalia—is a proof of concept that points toward a promising future:
- AI can make reliability prediction faster and more scalable
- Engineers will remain the final arbiters of safety
- Traceability and supervision are essential for certification and trust
As we continue refining these approaches, our focus remains clear: to empower engineers with the tools they need to make safer, more efficient systems—without compromising on integrity.
🔗 Stay Connected
Want to learn more about Anzen’s approach to safety and reliability engineering? Reach out to our team, read more about ATICA or follow our updates. We’re building a future where safety isn’t a bottleneck—it’s a strategic advantage.
| Aerospace Engineer with expertise in avionics, systems engineering and model-based design and analysis. At Anzen, Daniel’s work is focused on ATICA, our Model Based product. Daniel is in charge of building systems models and applying systems engineering processes while using ATICA to improve results on safety and reliability analyses for aerospace avionics projects. |
