Towards a richer model of deliberation dialogue: Closure problem and change of circumstances

3.1.The need for an open knowledge base

Consider the following dialogue between Alice and Bob extracted from an extended example in Walton [13].

Example 1. In this dialogue, the governing question and the inform stage are motivated by the fact that Alice and Bob would like to find a suitable house in Windsor. At the propose stage they narrowed their choices down to two: a condo and a two-story house. Both Alice and Bob place a high value on health, costs, and on the environment. A segment of the consider stage of dialogue follows:

Fig. 2.

Formalization of the speech acts in Example 1.

The MHP protocol formally represents the stages of the deliberation between Alice and Bob. However, if the circumstances change, and more information is needed before making a decision, the protocol may not be able to capture such characteristics. This is the case for example of move u11 where new information is now relevant. Such move cannot be represented in the current formalization of the MHP model. In fact, there is an assumption that all the participants have already gathered full knowledge of the circumstances either prior to the dialogue or during the information-seeking stage. New knowledge can only be added to the commitment store using speech acts assert(fact), exclusively during the inform stage. During the revise stage, revisions to facts may be possible via a propose(fact) speech act, which does not change the commitment store, however. This ensures that no other information during the propose and consider phases will change the view of the circumstances.

The deliberation dialogue always arises out of the question of what to do in any given set of circumstances and the information relevant to the circumstances is vital to arriving at a rational decision. In any real case, however, new information will need to be introduced during all sequences of argumentation prior to the closing stage and the dialogue model must support the new changes in the knowledge base. The following example will clarify this problem further.

Example 2. Brian had a problem with his printer. Whenever he scanned a document using the automatic document feeder, a black line appeared down the middle of the page. To solve the problem he searches for a troubleshooting guide for its printer. This guide gave a series of instructions. If this problem happens when you print and copy/scan, the problem is in the print engine itself, hence refer to a certain website for help troubleshooting. However in Brian’s case, the defect only occurred when scanning from the automatic document feeder. Following the instruction in the troubleshooting guide, first Brian opened the scanner cover checked for debris. There was none. Next he located the small strip of glass at the left of the main glass area. He carefully cleaned the glass with a soft cloth, and then scanned a document to see if this fixed the problem. It did not. Then he looked at the glass and he could see that it was covered by a thin plastic piece. He then managed to pull the plastic piece out. He found a small black mark in the middle on the bottom of the plastic piece. He tried cleaning the plastic, but it did not work, as he found by scanning a test document. He then showed the plastic piece to his wife Anna, and asked her if there was some way it might be possible to clean it to remove the small black mark. She proposed to use a soft cleaning pad and they managed to remove the black mark. Brian went through the scanning procedure as a test. Success! There was no longer a black line down the middle of the pages that were scanned. Brian’s problem was solved. An interpretation of the formal dialogue is shown in Fig. 3.

Fig. 3.

Interpretation of the printer example dialogue.

One of the most important aspects of this case is that the deliberation was based on a prior process of information-seeking dialogue. Moreover, the information-seeking dialogue was interwoven with the steps taken during the sequence of practical reasoning in the deliberation dialogue. The MHP dialogue permits the representation of Brian’s monologue in order to make a decision on what to do to fix the printer, considering the search for information as the troubleshooting instructions. However, if an agent were to act on Brian’s behalf, it would have to add a new piece of information to its knowledge base; i.e., u6: “The scanner is not working even if the instructions were followed”. The MHP dialogue does not allow this to be included. In order to identify the next solution, this information must be taken into consideration since it changes the circumstances, otherwise the agent would continue to propose the same decision; i.e., “Follow the n steps in the guide”.

The above discussion suggests that the changes in the knowledge-base during the dialogue and their effect on choosing new proposals must be investigated further. The MHP model of deliberation dialogue needs some revisions to bring it in line with an open knowledge-base that takes into account the change of circumstances during the argumentation stage.

3.2.An extended dialogue model for open knowledge

One of the most important factors in intelligent deliberation is that in real scenarios the circumstances of the world can change, and it is vitally important for the agent who is to deliberate intelligently to be aware of these changes and to take them into account during the course of the deliberation itself. Intelligent deliberation needs to be both informed and flexible. Some of the agents may know things that the others do not know, or information may only become relevant during the course of the dialogue. It is important that the knowledge base is left open during the argumentation stage so that new, relevant information that might affect proposals and commitments can come in. The MHP model has led to the development of a number of deliberation protocols employed in multiagent systems where agents have complete shared knowledge of the circumstances [3,7,9]. Optional actions in these frameworks are assigned to the agents for discussion at the initial stage of dialogue and information shared during the dialogue is not taken into account when selecting new proposals to be discussed. An error in a rational deliberation, however, is for the agent to become inflexible by failing to take new relevant developments into account when deciding what to do.

For this reason, deliberation as a framework for rational argumentation needs to be extended from the structure described above. Instead of the knowledge base being fixed at the opening stage, propositions need to be added to it and deleted from it as the deliberation proceeds through the argumentation stage.

Fig. 4.

Outline of a revised model of deliberation dialogue.

In Fig. 4, we propose a revised model, where we highlight the role of an open knowledge base. A knowledge base is set in place at the opening stage and agreed to by all parties in the deliberation as representing the circumstances of the situation in which the governing question is framed. This knowledge base is fixed in place at the opening stage and is part of what defines the choice that is to be made as stated by the governing question of the deliberation. The knowledge base can be updated during an information-seeking dialogue. More importantly, once we reach the argumentation stage, this knowledge base needs to be open so that if the situation changes, and information about these changes becomes available to the agents, this knowledge is taken into account during the stage where proposals are put forward, considered and evaluated.

The proposals may need to be modified or evaluated differently once this new information comes to be known. During this stage, some agents will have knowledge that others lack, and thus one important type of speech act is that of asserting a proposition that represents factual knowledge to inform the other agents involved in the deliberation process. There is the need to introduce a new speech act here that allows the assertion of the new knowledge and the revision of the commitment store. Here we refer to this speech act that is meant to intentionally exchange new information with disclose(fact). This fact may be a new piece of information that was not known at previous stages of the dialogue. Thus, during the argumentation stage, new information that comes in may alter the structure of the choice being made, by dynamically revising each participant’s knowledge of the world, which may lead to suggest a new option.

In this section, we discussed the need for the MHP model to be revised, especially in relation to the cases where the agents taking part in the deliberation have partial views of the circumstances of the world. The revised version offered here provides for the sharing of information between agents while deliberating so that better proposal can be identified. In the next section, we discuss a further problem of the MHP dialogue that may arise when agents have open knowledge: when a dialogue is to be considered successful.

4.1.The closure problem

Here we attempt to define conditions for closure of practical reasoning. A deliberation may have to be closed off and a decision taken based on the pro and con arguments put forward for practical reasons, typically time and money. A decision by majority vote may have to be taken to meet the practical demand for closing of the discussion. However, the depth, comprehensiveness and thoroughness of the pro and con arguments brought for and against the proposals is the most valuable feature of a deliberation leading to a supported conclusion offering a well-reasoned decision for action. There are also some issues remaining about how to represent the closing stage. What if it cannot be determined, on the evidence, which is the best proposal?

In a case of deliberation, like that between Alice and Bob, the dialogue can be closed when they have collected enough information about what is available on the real estate market, and they have discussed the matter thoroughly enough to critically examine all the pro and con arguments on both sides of every available option. At that point, even though the deliberation could be reopened before they make an offer and are committed to it, the dialogue has reached the closing point. If one party has put forward a particular proposal, and nobody else has any objections, then, at least temporarily, the deliberation is closed. They have found a proposed course of action that they all agree on, or at least that nobody disagrees with, or is willing to contend further. Hence the deliberation is over. A reasoned decision can now be arrived at on what to do. There are different options. The original proposal can be accepted or the original proposal can be rejected, in which cases the deliberation stops. The proposal can simply be modified, to consider the objections made during the dialogue, or a counterproposal can be brought forward. In the latter two cases, the procedure of formulating a proposal goes around the cycle again, except that this time a new proposal has been formulated in place of the one that was not accepted.

The decision of when the deliberation has reached its closing stage cannot be made by any current model of deliberation. In an emergency, the closing stage may have to be reached quickly. In theory, the closing stage should only be reached when the arguments and proposals considered on all sides have been sufficiently discussed so that all the relevant factors have been considered. In practice, a decision in the circumstances may have to be made within time constraints, and so a determination of which proposal is best may still be subject to pro and contra argumentation. In such cases, the argumentation stage should be closed off and some means taken to arrive at a decision, such as taking a vote. It is also possible, however, that in real instances of deliberation, there may be no agreement on the best proposal for action to solve the problem posed by the governing question. In this case, in the MHP model, presumably the deliberation dialogue must be considered a failure, because the governing question has not been answered. Nevertheless, the deliberation could have educational value, in that the consideration of the pro and con arguments might have shown deficiencies in some of the proposals. This revelation might have deepened the understanding of the participants.

Note that the only stages which must occur in every dialogue that terminates normally are the opening and the closing stage. It appears that we must have the closing stage in every deliberation dialogue, or at least in every successful deliberation dialogue. Hence, in the MHP model, a dialogue cannot really be a deliberation dialogue unless it terminates in a closing stage where a decision is arrived at on the best course of action to take. This remark is a revealing comment on the general problem of formulating goals in formal systems of dialogue. It also affects the persuasion type of dialogue. The goal of a persuasion dialogue is to resolve the conflict of opinions agreed upon as the issue at the opening stage. However, in many instances, a persuasion dialogue can be very successful educationally in revealing the arguments and positions on both sides. Determining the criteria of success of a dialogue in meeting its goal is a general problem for formal dialogue systems.

4.2.Criteria for closure of deliberative dialogue

In order to address the closure problem of a dialogue, here we define ten criteria that determine the extent to which a deliberation has been successful:

The relevance of an argument is determined by how it fits into a sequence of argumentation that connects to the problem or choice of action set as the issue of the deliberation at the opening stage. To declare that the closing stage has been reached, the participants in the deliberation must reach a consensus on whether the deliberation has been successful, based on the ten criteria above, so that one proposal has been shown to be superior to the others. This is called an internal evaluation of the success of the dialogue. The other way is an external evaluation carried out at a metalevel. Once a deliberation has been carried out in a given case by a group of agents, another group of agents can then keep a record of the argumentation in the deliberation, for example by keeping a transcript of the discussions. Then they can analyze and evaluate the argumentation in the deliberation dialogue, and arrive at an evaluation of how successful the dialogue was according to the ten criteria set out above.

This solution to the closure problem, while not yet implemented in a computational system, is useful for the project of improving current models of deliberation. The closure problem in other dialogues is different than using the closure conditions for deliberation dialogue proposed here. To explain this difference consider the inquiry dialogue. The inquiry is closed only when it can be shown that the proposition at issue can be proved, or where it can be definitively shown that it cannot be proved even though a large body of relevant evidence has been collected. The goal of the inquiry dialogue is to prove a proposition is true by a sequence of argumentation that is strong enough to meet the standard of proof appropriate for the type of inquiry. Argumentation in the inquiry is characterized by the goal of establishing a cumulative process of argument that builds only on established facts in order to prove a conclusion to a high standard of proof. Burden of proof in an inquiry is characteristically set at high standard ([15], p. 73). In contrast, there is no burden of proof in a deliberation dialogue, only a burden of reasonable explanation [14]. This situation is then different from that of the deliberation dialogue where the purpose of the dialogue is to make a decision on what to do or solve a practical problem. In this case, it may be necessary to take action, or decide not to take action based on a lower standard to show that an action can be justified or not such as the standard of the preponderance of the evidence. One reason for this difference is the propensity of the circumstances to change, hence the need for a different definition of the closure problem as the one we proposed in this section.

6.1.Experiment settings

The empirical evaluation that we performed aimed at showing that using Pcir agents are able to identify better solutions to interdependent conflicts. This would provide evidence for the fact that it is necessary for agents to adopt a model of open knowledge base. Experiments were run by generating plans within the disaster response domain, where one agent represents a humanitarian organization and the other represents a local authority. We generated 275 pairs of individual plans, where each plan had an average of 5 individual different objectives and 50 actions. The pair of plans varied in number of initial conflicts related to scheduling or norm violations forming a total of 11 conflict configurations in a range from 25 to 75 conflicts (i.e., 25, 30, …, 70, 75). Collaboration between agents was enforced for 5%–10% of the total number of actions in each plan, and these represent the dependencies that required discussion between agents.

In an experimental run with two agents, each agent was assigned an individual plan and a number of dependent actions Adl that would need to be discussed with the other agent following either Pcir or Pncir. An experimental run would only terminate when all the dependencies of both agents were discussed resulting in many dialogues during one run. At the end of each dialogue agents replanned using the information shared during dialogue if it was required. In Fig. 6 we propose a simplified formalization of a dialogue between two agents x and y within the disaster response domain as presented in the previous section.

The plans for each experimental run were analyzed to measure the conflicts before the dialogue, referred to as TOT and after the dialogue. We recorded the number of conflicts that agents were able to solve SOLV, the number of conflicts removed by dropping goals when an agreement could not be found DROP, and the total number of arguments used ARGS.

6.2.Better plans with information disclosure: Experiment results

A summary of the experiment results is proposed in Table 2. Here we report only those relevant to our discussion, more details can be found in [12]. The results showed that the percentage of conflicts resolved (SOLV/TOT) on average is significantly higher by about 18% in protocol Pcir than in protocol Pncir. This shows that by volunteering further information during the dialogue agents are able to solve more conflicts among existing plan, norm and goal commitments. Furthermore, we showed that the percentage of conflicts removed (DROP/TOT) due to goals being dropped is significantly higher by about 19% in Pncir than in protocol Pcir. Agents remove a very similar number of conflicts with the two protocols, but with Pcir those are resolved while with Pncir agents discard more goals because they are not able to find a suitable solution. Finally, regarding the number of arguments exchanged ARGS, we showed that the use of Pcir is more effective in conveying information about conflicts between plans due to the use of more supporting arguments. There is a significant difference between medians of about 28%, showing that more information about plan requirements was exchanged.11

In conclusion, the dialogue is more likely to be successful with a protocol where agents are able to intentionally exchange information about circumstances. This is because at any point in dialogue, an agent may disclose some information about the reason for adopting a particular action, due to causal constraints or norm requirements. These explanations, in the form of new information, permit agents to find more favorable alternatives to fulfill the dependency, in case of conflicts. The results prove that by permitting agents to disclose information about individual circumstances we obtain a significant improvement on the number of agreements established.

These results show that the introduction of a flexible knowledge-base where agents can add pieces of information about circumstances while deliberating is fundamental for establishing more successful agreements. In the next section, we report in more detail some results obtained in the investigation of our second objectives: whether a dialogue has been successful.

7.1.Criteria and hypotheses

The criteria proposed are necessary in human deliberation to determine whether enough proposals have been discussed and they are particularly useful to determine the educational value of deliberation in situations when an open knowledge base is introduced. However, existing systems for evaluation (e.g., [4]) use dialogue protocols based on the MHP model, thus their methods for evaluation are limited to assessing whether the agreement was found or not. We argue that if agents were to follow a more natural deliberation dialogue our criteria are necessary to assess the educational benefits of protocols for agent deliberation. While the criteria function as guidelines for evaluation, we must define specific measures to compute these benefits. We propose a measure for evaluating the following criteria within the system presented in Section 4:

We intend to show the benefits of a more flexible protocol, Pcir, able to share information about new circumstances, in comparison with a more restrictive protocol Pncir within a deliberation dialogue. As previously mentioned, the 275 pairs of plans vary in number of conflicts between them due to scheduling or norm violation. This permits us to study different aspects of robustness of the argumentation system as the complexity of the problem increases. The complexity is given by the total number of conflicts that exist between two initial plans (TOT).

The following hypotheses may be used to evaluate the criteria:

7.2.Results

In Fig. 7 we report a subset of selected results obtained with similar experiments of [12]. Side A represents the data and linear regressions of the number of conflicts solved (SOLV), while side B represents the number of arguments exchanged (ARGS) in both cases, when the total number of conflicts increases (TOT).22

Fig. 7.

Conflicts solved and arguments as total number of conflicts increases.

Figure 7(A) shows the results on the number of conflicts solved when the complexity of the problems increases providing evidence for Hypothesis 1. The graph shows that the number of conflicts solved is significantly higher when agents employ Pcir than using Pncir. This is a measure for testing the accuracy and completeness of the information regarding the circumstances of the case. We showed that many conflicts may only be solved through the exchange of information about the circumstances especially when the complexity of the problem increases. These results also give some information about Criterion 1, and, in fact, we notice that conflicts that are solved using Pcir are not solved using Pncir. This indicates that some proposals found in Pcir were probably not considered in Pncir, although further proposal tracking would be needed to disambiguate this case from the case in which those proposals were considered but not chosen as suitable options.

Our second hypothesis is verified in Fig. 7(B). The top line corresponds to the number of arguments moved in Pcir and the bottom line is Pncir. In a protocol where more information about circumstances is exchanged, there is more possibility for agents to discuss their proposal and claims more thoroughly. The results prove the claim that the arguments exchange ARGS in Pcir are significantly higher than with Pncir. We attribute the increase in the number of conflicts solved showed for Hypothesis 1 to the information shared within the two protocols, since the experiments were performed on the same pairs of plans.

We have presented here an initial simple method for measuring some of the criteria presented for resolving the closure problem: we considered the variation of number of conflicts solved as indicator of the accuracy and completeness of information exchanged, and the variation of number of arguments as indicator of the thoroughness of the discussion about a proposal. There is the need, however, to develop a more standard method for measuring these criteria in order to determine the educational value of deliberation in agent-based systems. More investigation within computational system is required to establish general conditions under which it is possible to measure these criteria, or to determine whether certain criteria are more useful than others in specific domains. In order to inform this framework, the work of Amgoud and Dupin De Saint Cyr [1] on measures of argumentation-based persuasion dialogue may be useful, especially in relation to those measures related to agents borrowing each other’s information, with adequate revisions to consider the differences that deliberation dialogue presents.