Collaborative Trustworthiness for Good Decision Making in Autonomous Systems

Abstract

As autonomous systems become increasingly prevalent in fields like transportation, ensuring their safe and reliable operation in dynamic, complex environments remains a critical challenge. These systems must make accurate decisions despite conflicting or uncertain information, such as differing sensor readings in self-driving vehicles. This article examines a pioneering approach to enhance the trustworthiness of autonomous systems through collaborative decision-making. By ranking systems based on their quality attributes and using structured methods to aggregate and propagate beliefs, the proposed framework improves decision accuracy and safety. The approach employs formal models like lattices and Binary Decision Diagrams (BDDs) to streamline belief processing, with experimental results demonstrating significant improvements in reliability, particularly when prioritizing high-quality systems.

Summary

The paper presents a sophisticated framework aimed at improving the decision-making reliability of autonomous systems, such as self-driving cars or robotic swarms, operating in unpredictable environments. A central challenge for these systems is navigating conflicting information, such as when sensors provide contradictory data about obstacles. To address this, the paper proposes a collaborative approach that leverages the varying capabilities of autonomous systems to enhance collective decision-making. The framework evaluates systems based on quality attributes, such as sensor accuracy or perception reliability, and uses these evaluations to guide belief aggregation and propagation.

A key element of the framework is the lattice structure, illustrated in Figure 1, which ranks autonomous systems according to their quality attributes. The lattice organizes systems hierarchically, with those possessing superior attributes, such as high-precision sensors, placed at the top. This ranking allows the identification of a subgroup of the most trustworthy systems, whose beliefs are prioritized in decision-making. By focusing on this subgroup rather than the entire group, the framework reduces the influence of less reliable systems, leading to more accurate collective decisions. For example, in a fleet of autonomous vehicles, the lattice might rank vehicles with advanced LiDAR systems higher than those with basic cameras, ensuring their observations carry more weight.

Figure 1. Lattice as a formal representation of partial-order relationships between autonomous systems based on their attributes. Green (resp. red) area represent autonomous systems with better (resp. worse) quality of attributes than 𝑠4.

The process of belief aggregation and propagation is modeled using Binary Decision Diagrams (BDDs), as depicted in Figure 2. This figure illustrates how a belief aggregation function processes multiple input beliefs about a predicate, such as whether an object is detected. Each node in the BDD represents a system’s belief, with edges indicating possible states (true or false). If disagreements occur among input beliefs, the aggregation function may adjust a system’s belief to align with the collective decision. To improve computational efficiency, the paper uses Ordered Binary Decision Diagrams (OBDDs), which impose a strict order on variables based on the lattice ranking. This ensures that beliefs from more trustworthy systems are processed first, reducing the complexity of the diagrams through reduction rules. The figure likely shows a scenario where a system’s belief changes due to disagreements, highlighting the dynamic nature of belief updating in collaborative settings.

Figure 2. A belief aggregation function considers 𝑘 input beliefs 𝑦𝑗(𝑝) on a predicate 𝑝. Belief of a node 𝑥𝑖 (𝑝) can change (i.e., 𝑦𝑗 (𝑝)= ¬𝑥𝑖(𝑝)) if there is a disagreement with input beliefs.

Figure 3 provides a high-level overview of the collaborative decision-making process. It likely depicts the flow of information among autonomous systems, starting with each system forming an initial belief based on its observations. These beliefs are shared, aggregated using rules that prioritize higher-ranked systems, and then propagated back to update individual decisions. This iterative process aims to converge on a reliable collective belief, particularly in scenarios with conflicting data. The figure underscores the importance of structured collaboration in achieving consensus, ensuring that decisions reflect the most accurate information available.

Figure 3. Examples of autonomous systems operating in the same environment (a) vehicles with different quality attributes (distance and perception angle) possibly detecting a pedestrian, (b) ranking between vehicles based on the quality of their attributes (assuming the smaller is the better). Vehicle 𝑠1 (resp., 𝑠4) has better (resp. worse) distance to the pedestrian and a better (resp. worse) perception angle compared to other vehicles. Both 𝑠2 and 𝑠3 are not comparable since 𝑠2 has a better perception angle but a larger distance to the pedestrian compared to 𝑠3, while for 𝑠3 it is the opposite.

The paper further elaborates on belief aggregation through Figure 4, which illustrates a two-expert aggregation function. This function considers the beliefs of two highly ranked systems to compute a collective belief. If both experts agree, their belief is adopted; otherwise, the system may retain its original belief or adjust based on predefined rules. The figure shows how the belief of a node changes in response to disagreements, emphasizing the role of expert systems in guiding decisions. Similarly, Figure 5 compares non-reduced and reduced OBDDs, demonstrating how aggregation rules simplify decision diagrams. For instance, in a scenario with three systems ranked from most to least expert, the reduced BDD adopts the belief of the most expert system or the two most expert systems if they agree, significantly reducing computational overhead.

Figure 4. Example of a 2-expert aggregate function for belief aggregation. The function considers for 𝑥𝑖 (𝑝) two input beliefs 𝑦𝑖−1 and 𝑦𝑖−2 on a predicate 𝑝 to compute 𝑥𝑖 (𝑝). Belief of a node 𝑥𝑖 (𝑝) is changed if there is a disagreement with all input beliefs.

Figure 5. (a) non reduced OBDDs with three autonomous systems 𝑠1 > 𝑠2 > 𝑠3 with order from the most expert to the least expert one, (b) reduced BDD after aggregation and propagation and their beliefs 𝑦𝑖 (𝑝) considering the most expert rule where all remaining autonomous systems adopt the belief of the most expert one, (c) reduced BDD after aggregation and propagation considering the 2-experts rule where paths are reduced by adopting the belief of the two most expert ones when these latter have the same belief.

The experimental validation, detailed in later sections, uses synthetic benchmarks with 20 autonomous systems and varying quality attributes. Table 1 presents a sample benchmark, showing systems with quality attributes and their corresponding beliefs for a predicate. The table highlights how systems with higher quality attributes tend to have beliefs closer to the ground truth, reinforcing the importance of the lattice ranking. Figures 6 and 7 provide insights into the effectiveness of different aggregation rules. Figure 6 compares the number of errors introduced and corrected by rules like the two-expert rule and the majority rule, showing that the two-expert rule often corrects more errors in high-quality groups. Figure 7 details per-configuration results, revealing that configurations with lower initial error rates benefit more from expert-based rules, achieving collaborative reliability ratios close to the optimal value.

Table 1. Example of a synthetic benchmark where quality of attributes and beliefs for a predicate 𝑝 are generated randomly for two different configurations. The resulting ranking of autonomous systems is modeled after the lattice in Fig. 1

Figure 6. The effect of rules on the number of introduced errors and corrected errors, (a) Individual errors before belief aggregation and propagation, and introduced errors caused by each rule, (b) Corrected errors after beliefs aggregation and propagation for each rule.

Figure 7. Detailed results per configuration. (a) Each configuration has a different percentage of errors (blue bars), all percentages are more than 30%, the percentage of errors from experts are also displayed (red bars). (b) Results of collaborative reliability for each configuration for the two-expert and majority rule.

Figures 8 and 9 further explore the impact of group quality on reliability. Figure 8 illustrates how groups with different quality distributions exhibit varying error rates, with high-quality groups having lower error distributions. Figure 9 compares the collaborative reliability of high- and low-quality groups, showing that high-quality groups achieve ratios near 0.5, indicating effective error correction with minimal new errors. Low-quality groups, however, often have ratios above 1, reflecting more introduced errors. These figures emphasize that the quality of participating systems significantly influences the success of collaborative decision-making.

Figure 8. Different Groups and their Error Distribution. Each group is generated independently by changing the quality of attributes of the autonomous systems in the group, different error distributions are obtained, as the lower the quality of attributes, the higher the error distribution is for each autonomous system.

Figure 9. (a) Collaborative Reliability by applying the Two-Expert Rule for the high-quality and low-quality group. The ratio for the low-quality group is always above 1 resulting in a high Collaborative Reliability and the high-quality group has an average of 0.5 Collaborative Reliability Ratio reflecting the low error distribution of the group. (b) Average Collaborative Reliability for Different Groups and Different Rules. For each rule and each group, depending on the group’s error distribution, the average collaborative reliability is calculated, and the lower the average collaborative reliability ratio, the better.

The paper contextualizes its contributions within related work, drawing on concepts from collective intelligence and social choice theory. It critiques traditional majority-based aggregation, which assumes equal reliability across systems and may lead to suboptimal outcomes, as noted in references to Arrow’s impossibility theorem. By contrast, the proposed framework exploits the heterogeneity of autonomous systems, prioritizing those with superior attributes. The paper also discusses practical considerations, such as the role of advanced communication technologies like 5G/6G in enabling real-time belief sharing. However, they acknowledge challenges, including the computational cost of processing complex BDDs and the need to carefully select attributes for ranking systems.

In conclusion, the paper offers a robust and innovative approach to enhancing the trustworthiness of autonomous systems through collaborative decision-making. By using formal models like lattices and OBDDs, and by prioritizing high-quality systems, the framework achieves significant improvements in decision accuracy and safety. The experimental results validate the effectiveness of expert-based aggregation rules, particularly in high-quality groups, and highlight the importance of group composition in achieving reliable outcomes. This work provides a strong foundation for future research into scalable, real-world applications, potentially transforming how autonomous systems collaborate in critical domains like transportation.