This is a linkpost for a research initiative at MIT I just discovered while following up on some earlier reading. I have linked to the Publications page to make it easiest for people to get in and start perusing.
The goal of this initiative is to improve state-of-the-art systems engineering, and in particular to be able to account for uncertainty and changing contexts during and after the design phase.
What drove me to bring it to the attention of this community is the MATE program, which stands for Multi-Attribute Tradespace Exploration. This is interesting because it consists of defining desirable qualities and then building a utility function out of them, upon which design decisions will be based.
Among the publications are a series of applied attempts at building actual utility functions for real things, then using decision theory with them.
They increase the amount of things you can trade off for by expanding what they call the “Ilities” (reliability, versatility, etc). Among these new ilities is a more advanced notion of safety.
This means there is a body of applied work which has a concept of safety, calculated into utility functions, and operated on according to decision theory.
Further updates will be made in the comments as I finish readings.