Confronting value-based argumentation frameworks with people’s assessment of argument strength

3.1.Two variations on VAF

As we carried out the experiments reported below, some hypotheses arose about variations on VAFs that could account for the obtained results. For the sake of readability, we introduce those variations in this section and will refer to them in due course.

4.1.Experiment #1

Participants were 64 voluntary undergraduates from different fields of study at the Universidad Nacional del Sur, Bahía Blanca, Argentina. Since we had no hypotheses regarding either gender or age, we did not collect data on those variables in these experiments. However, we knew it was a balanced representation by gender, while the vast majority of participants were between eighteen and twenty years old.

The experiment was conducted by means of an online single-variant Google form. We used a scenario similar to the one examined in [14],33 with the following formulation:44

First, we asked to identify the values promoted by each argument by choosing an option among “Right to life,” “Right to property,” “Right to life and right to property,” and “Neither.” Second, we requested to express preferences between the values regarding the situation in question, by comparing them in terms of “more important than,” “equally important,” and “I don’t know.”

Then, participants were asked to identify compatibilities and incompatibilities between the arguments, instead of detecting attacks. This is because we assumed that, for an audience not specialized in argument systems, the term ‘attack’55 could have intentional or pragmatical connotations that are difficult to perceive, beyond the fact that one argument can be used to oppose another only if it is somehow incompatible with it. In that sense, we considered that incompatibility is a necessary condition for attack, since the attacked and the attacker cannot stand together on a reasonable basis. Hence, we asked the question below:

Finally, participants were requested to evaluate arguments and conclusions. For arguments, we asked the following question:

Results. The scenario was presented by Bench-Capon in [14] as the VAF depicted in Fig. 1. He identifies A and D with the value of life, and B and C with the value of property. Our results were analyzed in comparison with that presentation. All percentages are rounded.

Fig. 1.

VAF of the insulin scenario according to Bench-Capon (taken from [14]).

Fig. 2.

Percentage of individuals who perceived each defeat. In parenthesis, percentage of participants who also observed both incompatibility between the arguments and non-preference for the value promoted by the defeated argument. For simplicity, we recorded only those defeats (arrows) observed by at least 20% of individuals.

We start by checking the perception of defeat. Figure 2 shows the percentages of participants that perceived each defeat and, among them, those who considered that both conditions were satisfied. Present participants’ intuitions about defeat were fairly consistent with Bench-Capon’s notion in some cases, but not in others. By definition, defeat implies attack (at least, incompatibility, following our hypothesis) and non-preference of the value promoted by the defeated argument, but this implication was only verified to a good extent for the pairs of arguments (A, B), (B, A), (C, B), and (C, D). Overall, 238 defeats were reported, and participants identified an incompatibility between the arguments in tension in 159 of those 238 cases (67%). Therefore, most participants’ opinions are consistent with the notion that people should perceive incompatibility between a defeated argument and its attacker. Having said that, it is noteworthy that a significant minority of judgments (33%) did not conform to that premise. In addition, it is clear that the representations of the scenario as a VAF that arise from the opinions of participants are mostly at odds with the Bench-Capon’s representation given in Fig. 1.

As reported in Table 1, the experiment allowed us to observe that people quite clearly identify the values in agreement with Bench-Capon’s opinion when they are explicit in the premises (arguments A and B), but not when they are not (arguments C and D). Of the total number of participants, 72% linked argument A exclusively to the right to life, and 94 % related argument B exclusively to the right to property. The case of argument C (“Martina can do that if she replaces Carla’s insulin after the emergency”) is surprising. In our opinion (and in line with Bench-Capon), it promotes the value of private property, but only one participant associated the argument exclusively with that value, while 31% identified it with both life and property rights, 30% did not link it to any of those values and, strangely, 38% related it only to the value of life. Maybe the answers of the last two groups can be explained by the fact that C defends or reinstates argument A, which is clearly associated by most participants with the value of life. For its part, argument D was identified with the value of life only by 11% of the participants, and with both values by 12%, while just one subject related it to the value of property only. This shows some tendency to associate the argument with the value of life, as expected, but the similar connection with both values at the same time is striking. Moreover, most of the participants (73%) did not respond, which can be interpreted as a difficulty in identifying the value promoted by D.

Opinions on winning arguments are summarized in Table 2. The “extension” {A, C} was the most accepted choice (n=20, 31%). Within this group, the majority considered –curiously– that A promotes the values of both life and property. Argument C alone was the second most accepted option (n=15, 23%), and the majority within this group determined, as expected, that it promotes the value of life. The third most accepted choice was A alone (n=7, 11%), and here again the majority believed that it promotes the expected value, life. From here down the table, as we find smaller minorities, there are also more seemingly irrational answers. B was selected by 9% of the participants, the majority of whom judged that it promotes the value of life, contrary to expectations. However, nobody within that group stated that life is more important than property. The remaining choices were made by too few participants and with even rarer answers to draw any conclusions (for this reason, we did not show the data in the table). All in all, we can see some correlations between the arguments judged to be winners with the value they promote and value preferences, suggesting that the latter is a good predictor of argument choice, irrespective of other variables.

Regarding the conclusions, the responses were consistent with the selection of arguments. The acceptance (resp., non-acceptance) of an argument given the answers “agree” or “totally agree” (resp., “totally disagree,” “disagree,” or “neither agree nor disagree”) with respect to its conclusion was observed in 86% of the cases (221 out of 256). Conversely, the agreement (resp., disagreement) with a conclusion given the acceptance (resp., non-acceptance) of its supporting argument was observed in 98% of the cases (251 out of 256). The difference seems to make sense given that, in some situations, people are inclined to accept a conclusion even if they do not accept some supporting arguments.

Next, we assessed the extent to which VAF can be used to make correct predictions about argument acceptance and compared that with other variables. In each case, we took into account perceived defeats, perception of what values the arguments promote, and value preferences. We checked the following variants:

Results are displayed in Table 3, which shows that differences are in a small range of around 5.5 percentage points. As can be seen, VAF worked relatively well, but the best performance was obtained by taking into account only the promoted values and value preferences. Beyond mean differences, the overlap of the confidence intervals evidences the closeness of the predictive efficacy among all models.

Discussion. We observed that the association of arguments and values was very heterogeneous with the exception of argument B, which almost all participants perceived as promoting the value of property. The case of argument C, which 31% of the participants linked to both values, evidences the need to consider that arguments can promote more than one value in order to, at least, get more accurate representations. In sum, the association of arguments with values seems to be influenced by subjective factors, which poses a representational issue, and different values can converge in one single argument, unlike VAFs’ underlying intuition. Bench-Capon himself was aware of these limitations and identified certain important factors to take into account in some specific domains, such as the authorities the audience respects, their attitude to risk and their degree of loss aversion [16]. In fact, it is also interesting to investigate a variety of biases as factors that modulate value preferences.66

A different source of doubt regarding the association of arguments with values lies in a possible inadequate design of our experiment. In view of the results, the option “right to life and right to property” to identify the promoted values could have induced some guessing responses in case of doubt or confusion. This was taken into account for the next experiment.

We compared VAF-based predictions with several variants, obtaining similar results. However, the best performance was that of the model that considers value preferences as the only relevant variable. This suggested that improved measurement of appreciations relative to values could give us better differentiated results. Likewise, we suppose that intuitions about defeat might be better tested in simpler scenarios where attacks are more clearly one-way directed and value promotion is more easily identifiable.

We can also analyze other sources of representational problems. The directionality of the attack relation is certainly a very important issue. Bench-Capon himself was aware of the problem and he did not have a definite solution. In fact, the insuline scenario, which he represented in [14] by a linear relation where D attacks C attacks B attacks A (the representation we used in our study), was instead represented in [13] by a 3-cycle where A attacks C attacks B attacks A. Moreover, in [12], the author said the following:

Another sign of a possible representation issue is the low acceptance of argument D (and of its conclusion). This might suggest the intervention of an unspoken argument in the minds of the participants, one that defeats D. Given that D considers that a person with limited economic resources would have the right to take the insulin without the obligation to compensate the owner, the argument could be interpreted as referring to exceptional cases, and that the information in the context does not indicate that Martina’s case fits such an exception. This suggests, at least, three representational rectifications of the models. One is to take into account the incidence of some argument E attacking D, based on considerations of irrelevance for the case in question. This fifth argument would promote, in turn, the value of information, or the lack of it, prevailing over the others. The argument D can raise some critical questions that can be expressed in the form of arguments, and that can be represented through structured argumentation versions of VAF (see, for example, [2,3,18], and [8] for representing exceptions). A second rectification consists of simply deleting D from the representation, while a third one consists of considering an attack from C to D. All these rectifications would explain the high esteem for argument C, whose only attacker, D, would be ignored or rejected.77

These possibilities are in line with [26], where the authors claimed that, in order to create argumentation systems, designers must take into account implicit domain-specific knowledge or beliefs. Maybe these representational problems are more important than the extent to which the empirical data match some semantics,88 in the sense that they call for a foregoing resolution. In the meantime, these problems could be tackled with experiments and questionnaires specifically aimed at elucidating what is being represented, and/or using simpler scenarios whose representations raise fewer doubts.

In view of the above results, we planned to test the following hypotheses:

4.2.Experiment #2

Sixty-two undergraduate students from different fields of study were voluntarily recruited at the Universidad Nacional del Sur, Bahía Blanca, Argentina. In order to test hypotheses H1 and H2, we maintained the questionnaire from the first experiment, except for the questions that involved the variables relevant to the hypotheses, namely, association of arguments with values, varying degrees of importance assigned to values, and possible influences of external arguments. Regarding hypothesis H1, participants were asked to express their beliefs about the value(s) each argument promotes. To answer, they had to mark cells in a table with four rows headed by the names of the arguments (A, B, C, D) and two columns titled “right to life” and “right to property”. Then, participants were requested to indicate the importance of the values with respect to the referred situation (the insulin case), each one on a 10-point scale where 1=“Not important at all,” and 10=“Absolutely important.” Regarding hypothesis H2, participants were asked to answer the following question:

Results. Table 4 summarizes the results about the perception of the values promoted by the arguments. Our change of strategy in presenting of answer options seemed to make a significant difference in the assessment of argument D, which in this experiment was associated with only the value of life by 69% of the participants. We also found that A and C were more clearly linked to the value of life only (in the case of A, no participant identified it with both values).

To test H1, we formulated the hypothesis in a more precise way by taking into account the following operationalization of functions, where X ∈{A, B, C, D} and i is a participant:

∙ H1: The set of arguments accepted as winners by the individual i is

Then we analyzed a quantitative version QV of model V, that predicts the choice set Wi for each participant i. Like V, this model does not take into account attacks or defeats.

Additionally, we aimed to analyze the quantitative version QMVAF (Definitions 7 and 8), with the variants of considering defeat perceptions, on the one hand, and defeat reinforced with incompatibility perceptions (QMVAF (I)), on the other.

The comparison of correct prediction rates is displayed in Table 5. As in Experiment #1, we tested VAF-based predictions only in those cases in which the arguments were perceived as promoting at most one value (n=45).

Regarding H2, i.e., the hypothesis that participants considered the incidence of external arguments, 19 participants (31%) introduced new arguments or, at least, some comments, either for explaining their decisions or for criticizing some point of view. Among them, we identified 11 out of 19 arguments either presenting objections to argument D or supporting the contrary conclusion, i.e., Martina must replace the insulin. Here are some examples:

Discussion. We can see that VAF predicted comparatively quite well in this experiment. Now, two results are striking:

(1) When defeat and incompatibility were considered together to predict accepted arguments (I), predictions were less accurate than when defeat alone was taken into account (in contrast to results in Experiment #1). We could think that this issue is due to the fact that, when we considered defeat and incompatibility together to predict acceptance, we ended up establishing a stronger criterion for making a prediction (to wit, a perceived attack was dismissed if it was not accompanied by a perceived incompatibility between the attacker and the defeated argument). This criterion seemingly leads the models to avoid taking into account weak signals that have predictive power.

(2) Model V achieved a lower performance than in Experiment #1 (though it improved somewhat in the quantitative version QV). Unfortunately, our possible explanation for (1) does not explain (2). Alternatively, we could relate both issues to how we inferred the participants’ preference among the values, which is the only methodological variation between the experiments. Whereas in Experiment #1, we directly asked them to choose whether they prefer one value over the other or were indifferent (i.e., a comparative framework), in Experiment #2 we requested the participants to numerically express the level of importance of each value separately (non-comparative framework). This might have weakened the predictive power of the values in Experiment #2, since that information affects validations of both defeat and value preference. Although we do not have an explanation of how the information difference could generate the effects of (1), it provides a good reason to account for (2). Indeed, some classic findings in experimental psychology (e.g. [38]) show that when individuals evaluate alternatives numerically, they do not necessarily make comparative judgments, and preferences inferred from those judgments may be reversed compared to more direct choices.

Arguments were more clearly associated with values than in the first experiment. Though, in some cases, a single value assignment tendency was appreciated, in concordance with Bench-Capon’s model, some arguments were still associated to both values. The perception of the values promoted by arguments and the measure of their importance were still correlated with argument acceptance regardless other variables intrinsic to the model (such as attacks, defeats, conflict-freeness, or any usual extension semantics), which makes V and QV simple estimation methods for argument acceptance. Predictions include those cases that cannot be represented as VAFs because arguments are perceived as promoting more than one value. In cases that can be represented as VAFs and values can be ordered asymmetrically, Bench-Capon’s model performs something better (67% vs. 58%). Still, V and QV are more parsimonious, in the sense that they suggest less complex computations with fewer variables.

4.3.Experiment #3

In the previous experiments, we were not able to determine on a good basis how participants perceived attack directions, independently of incompatibilities. That was in part due to the fact that the arguments’ conclusions in that scenario were contradicting, which could suggest an attack in either direction. As Bench-Capon [15] noted, knowing the type of attack can be crucial.1010 In this experiment, we aimed to test the hypothesis that the preference of the value promoted by an attacked argument may have the effect of avoiding defeat, but the attacking argument is not acceptable together with the attacked argument since the conflict persists (H3).

Fifty-six volunteer students from different disciplines at the Universidad Nacional del Sur, Bahía Blanca, Argentina, participated in the experiment. Paper questionnaires were delivered by hand. Unlike the previous experiments, in the scenario of the present study we used different arguments A and B in such a way that B denies that the evidence can be used to condemn X. Therefore, B is an undercutting defeater of A. Moreover, in the wording we used, B begins with ‘However’, which suggests that B is a counterargument of A, hence, favoring a one-way interpretation of the attack:

A: X should be convicted, because the evidence shows that he was responsible for the crime.

B: However, the evidence was obtained illegally, so it cannot be used to convict X.

Note that we could not have gotten the desired interpretation if we had asked about an incompatibility instead of an attack. If participants were able to understand that B attacks A and not the other way around, then we would have a good basis for analyzing defeat as dependent on a correct perception of the values promoted and preference over those values, in accordance with VAF’s underlying intuition. In addition, we asked about the comparative preference between the promoted values (as in Experiment #1, to avoid the possible effect that occurred in Experiment #2) and about any other possible values that the participants would consider to bias their choice. The questionnaire was as follows:

Results. Results in Table 6 show clear tendencies to agree with the intuitions that the only attack is from B to A, that A promotes the value of evidence, and that B promotes the value of legality. This is in accordance with VAF’s canonical representations. The other variables rely on more subjective appreciations.

Now, according to VAF, we have the following predictions. If participants only perceived the attack from B to A, then:

(i) if the value promoted by A (evidence) is not more important than the value promoted by B (legality), then B defeats A; hence, participants should accept B and reject A;

(ii) if the value promoted by A (evidence) is more important than the value promoted by B (legality), then B does not defeat A; hence, participants should accept both B and A.

In either case, B should be accepted, i.e., B should be objectively accepted. Moreover, if evidence and legality are perceived with the same importance, then VAF reduces to a Dung’s AF, where the attacking argument is the winner. In turn, H3 predicts the same as VAF in (i) and the acceptance only of A in (ii).

In Table 7, we compared the VAF-based predictions and H3 with respect to (i) under two different conditions: (a) assuming the asymmetric attack from B to A as a fact, no matter what participants said about that, and (b) taking into account only cases in which participants recognized that asymmetric attack. In our data, the group that is crucially relevant to compare the hypotheses was formed by 27 participants who accepted that (1) A promotes evidence, (2) B promotes legality, and (3) evidence is preferred or indifferent to legality. According to VAF, the participants accepting these three conditions would still accept B, while H3 predicts the opposite (note that, if condition (3) is not fulfilled, then the acceptance of B would not contradict any hypothesis).

For (a), V predicts that individuals will not accept both A and B, which could be interpreted to mean that the arguments are anyway considered in conflict, in concordance with H3. Consequently, they should choose at most one argument. Results show that 23 (85%) participants accepted either only one argument or none: among the participants who preferred evidence over legality, 89% chose only A (8 out of 9), and among those who declared indifference between the values, 64% opted for only A, and 36% for only B (9 and 5 out 14, respectively). In contrast, only 6 individuals (22%) answered in accordance with the VAF-based prediction: 5 chose B when they were indifferent between the values, and 1 chose both A and B preferring evidence to legality.

For (b), H3 had a prediction efficacy of 81%: 13 out of 16 participants chose only one argument. Moreover, 5 out of 6 (83%) chose A when preferred evidence, and 57% (4 out of 7) and 43% (3 out of 7) chose A and B, respectively, when declared indifference between the values. In contrast, we registered a VAF-based prediction efficacy in 25% (4 out of 16): 1 out of 7 (14%) participants opted for both A and B when they preferred evidence over legality, and 3 out of 9 (33%) participants chose B when declared indifference between the values. Finally, when legality was perceived as more important than evidence, and provided that participants identified A with evidence and B with legality, both VAF and H3 had identical (low) efficacy: both succeed in the same 4 samples out of 8 (50%) under condition (a), and in the same 2 samples out of 6 (33%) under condition (b). However, if we ignored how participants identified the arguments and the values and just observed the value-argument correlation assuming that A promotes evidence and B promotes legality, both VAF and H3 were 73% successful in predicting the choice of B when legality is preferred.

Next, we considered the acceptance predictions in the frameworks elicited by the participants’ opinions, comparing VAF, AF, and V (Table 8). As can be seen, V is the model with the highest predictive accuracy and is the only one of the three that performs above chance level.

Regarding questions (7)–(9), we only obtained three answers, which is insufficient to shed any light on the behaviors.

Discussion. The results showed low agreement with the concept of objectivity proposed by Bench-Capon and replicated the participants’ tendency to choose the argument that promotes the preferred value. Indeed, argument B, which VAF deems as objectively acceptable, only obtained 27% acceptance (regardless of other variables), much less than A, which was accepted by 45% of the participants (Table 6). In contrast, H3 predicted with much more success. This suggests that VAF could be adjusted by giving more importance to value promotion in order to better fit as a theory of human argument acceptance.

It can be reasonably argued1111 that the wording “is used to,” when we asked about attacks, is not related to the conceptual assumptions of the model. In argumentation theory, an attack from X to Y generally means that X can be used as a counterargument against Y, not that X is actually used as a counterargument against Y in a given debate. Indeed, “is used to” may give rise to pragmatic considerations that are not at stake. Now, on the one hand, we believe that in the context of our experiment, either wording leads to a similar interpretation. Even if we used the wording “is used to attack”, we think that the most straightforward interpretation of that phrase is that “it could be used to attack,” because there were no subjects actually implementing the arguments to attack each other. On the other hand, suppose that the wording suggests a biased interpretation of directionality that does not correspond to an adequate theoretical concept. However, ‘attack’ is usually defined in such a way that the conditions for it to occur are not subject to individual interpretations but to objectively verifiable logical/linguistic relationships. In accordance with the definition given in ASPIC+, for instance, it is the case that B attacks A, but A does not attack B, because B is an undercutting defeater of A. The notion of ‘defeat’ in VAF, in turn, depends on both an (objective or given) relationship of attack and a (subjective, dependent on the audience) preference over the promoted values. So, it was expected that B was accepted by most of the participants. In terms of VAF, B is objectively acceptable, because, in any case, B attacks A and either a) the value promoted by A is not preferred, or b) it is preferred, implying that B does not pose a threat to A. However, the experiment showed that –contrary to the VAF-based prediction– B was not objectively accepted by the participants. All in all, the small difference found between conditions (a) and (b) in Table 7 indicates that the perception of the direction of attacks is not determinant for argument acceptability.

Regarding the low prediction efficacy of both VAF and H3 when participants preferred legality to evidence, we can only provide speculative explanations. A possible psychological reason is that participants could declare preference for the “correct” value of legality, according to the rule of law, while their sincere feelings are either on the side of evidence or balanced between both values. More generally, the content of the arguments could induce the occurrence of biases and framing effects that modulate the expected incidence of value preferences. In particular, we used a scenario where conviction was under discussion, which could generate a leniency bias that affected the evaluation of the arguments. Experiment #4 is proposed to test this bias in different framings, as a classic challenge to normative effects on judgment and decision making [51].

Finally, the same outcome as that of model V could be attained by a VAF in which there is a bidirectional attack between A and B. So, a possible explanation for this outcome could be that the participants really feel the conflict to be a bidirectional attack, but do not indicate this in the questions about attacks, because they do not correctly understand the meaning of the specialist term ‘attack’ (or are misled by the words “is used to,” as discussed before). As it can be seen, the problem of the directionality is recurrent.

4.4.Experiment #4

We recruited 74 undergraduate students from different disciplines at the Universidad Nacional del Sur and the Universidad Salesiana, Bahía Blanca, Argentina. The questionnaires were completed online in Google Forms. To test the hypothesis that the acceptance of value-based arguments can be modulated by biases and framing effects (H4), we designed two frames (scenarios) with identical representation as value-based argumentation frameworks and equal values, but inverting conclusions with opposites (acquit/convict):

The same questionnaire followed each frame:

According to our intuition, A, B, A′, and B′ all promote the value of evidence, while both C and C′, the value of procedural legality. On the other hand, B and B′ pose asymmetrical (specificity-based) attacks against A and A′, respectively, while C and C′ generate asymmetrical (undermining) attacks against B and B′, respectively (Fig. 3).

Fig. 3.

VAFs with identical structure representing the frames used in Experiment #4.

We tested two conditions, C1 and C2, only varying the order of presentation of the frames: in C1 (n=35), F1 was presented first and F2 second, while, in C2 (n=39), the order was inverted.

One point to observe was to what extent responses about the second frame showed different intrasubject perceptions of arguments and/or values with respect to the first frame. One variable of interest was the “positional concordance” among the arguments chosen in one frame and the other, and we classified it into three classes: a) strict concordance: argument X was chosen in one frame iff X’ was chosen in the other frame (i.e., the arguments chosen occupy the same position in their respective graphs); b) weak concordance: the selection of arguments changed regarding their positions but there is no conflict (attack) between their positions (i.e., from A to C′, or from C to A′); and c) non-concordance: the choice of arguments changed regarding their positions and there is conflict (attack) between their positions (i.e., from A to B′, from B to A′, from B to C′, or from C to B′).

Results. In both conditions C1 and C2, we observed similar percentages of concordance in general (differences are not statistically significant). We found 57% of strict concordance in C1 and 59% in C2, 17% of weak concordance in C1 and 15% in C2, 17% of non-concordance in C1 and 26% in C2 (in C1, there were 9% of participants who did not choose any argument in some of the frames, so we do not register that as a fact of concordance/no concordance). The greater tendency towards strict concordance would favor the robustness of the structural factors as predictors of choice under similar value preferences.

VAF predicts the following behaviors regarding acceptability. If the values are assessed as having the same importance, then the prediction matched that of Dung’s model, that is, to chose A and C, and A′ and C′, for F1 and F2, respectively. If evidence is evaluated as being more important than procedural legality, then the prediction is choosing B and C, and B′ and C′, in F1 and F2, respectively. And if procedural legality is assessed as more important than evidence, then the prediction is to opt for A and C, and A′ and C′, in F1 and F2, respectively. The results show that the effectiveness of these predictions in the intrasubject analysis is zero, if we regard the exact extension as the only successful prediction. However, it is around 35% (37% in C1 and 33% in C2) if we consider choosing some argument belonging to the predicted extension. Now, if we deviate from VAF and assume value as the only relevant independent variable, then we get more effective predictions, hovering at about 77% in C1 and 64% in C2 (average between both frames). This implies that, taking into account only the promoted values, the predictions improve by about 35 percentage points with respect to VAF. When both values are perceived with the same importance, the VAF-based prediction is just the same as that of Dung’s model, which show success in predicting the selection of some argument in the extension is around 90% in C1 and 79% in C2 (both frames considered in each condition).

With respect to biases, some of the participants changed the degree of importance given to at least one value from one frame to the other, which hovers at about 14% in C1 (n=5) and 20% in C2 (n=8). In addition, part of the participants varied the order or the relative importance between the values from one frame to the other, though the changes mainly consisted in going from different degrees to equivalent or vice versa. In this respect, we observed very similar proportions of change in both conditions, around 20%. In C1, 71% (5 out of 7) of the variations were from equivalence in F1 to difference in F2, of which 60% (n=3) changed in favor of evidence and 40% (n=2) in favor of legality. In C2, 88% (7 out of 8) varied from difference in F2 to equivalence in F1, of which 29% (n=2) changed in favor of evidence and 71% (n=5) in favor of legality. This shows that individuals have a slight tendency to move towards legality in the context of F1 and towards evidence in the context of F2. In both cases, variations in the perception of the relative importance of the values tend to concede more strength to lenient arguments (X cannot be convicted, X was not responsible of the crime).

Regarding the question about any other value involved that influenced the choice of arguments, and which arguments supported that value, B and B′ were the most mentioned arguments (both conditions considered, 5 mentions in F1 and 7 in F2, respectively) and were associated with values such as responsibility, truth, justice and ethics. Next we have A and A′ (3 mentions in F1 and 4 in F2, resp.), identified with justice, and C and C′ (2 mentions in each frame), linked to law. In the vast majority of cases, influences were recognized as favoring the choice of those arguments, but not the rejection of others.

The results on intersubject argument acceptance are summarized in Table 9. The first and second columns show the percentages of acceptance in each frame and condition. Although the percentage variations could suggest some order effect that causes the acceptance of arguments in the second frame with different strength than in the first one, they are not statistically significant. This could be due to a small size of the sample. In any case, combining the results obtained in both conditions (third column) we obtained similar values between F1 and F2. Moreover, we still observed non-significant differences when combining the results in the first frame of each condition, that is, when participants answered without any previous exercise (last subcolumn), non-significant differences were still observed.1212

Discussion. One important aspect of VAF is to model objective argument acceptance (Definition 4), as a way of capturing persuasion for any audience. In the frames involved in this experiment, according to the model, C and C′ are objectively acceptable in their respective frames: no argument defeats them, and if evidence is preferred to legality, then they are acceptable since they do not pose any threat to B and B′. On the contrary, results showed that individuals that prefer a value preferred to that perceived to be promoted by a given argument tend to make that argument not eligible. This result coincides with that of Experiment #3.

Now, a certain interpretation of Table 9 gives us a different picture regarding VAF-based predictions. In the last column, we can see that the arguments B (B′) and C (C′) have similar percentages of acceptance (around 40% on average), above those of A (A´) (around 26%). Taking into account that, on average, evidence is preferred over legality in 49% cases, against a 13% preference for legality over evidence (Table 10), we have a coincidence between the collective acceptance of those arguments and the outcome predicted by VAF. That is, if we ask which arguments are more likely to be collectively accepted in F1 and F2, since that evidence is at least as preferred as legality, then the preferred extensions {B,C} and {B′,C′}, respectively, give us the most probable acceptable arguments. Nevertheless, we do not have a good explanation, in terms of the model, about why this happens, because the data do not show the expected correlations according to VAF’s semantical considerations. In fact, 66% of the participants that preferred evidence to legality did not accept C or C′ (on average).

Another possible explanation1313 is that participants do not actually see a conflict between arguments B and C (resp., B′ and C′). An explicit conflict could be created by adding “and therefore should be convicted” (“and therefore should be acquitted”) at the end of B (B′). But then the conflict between B and C is a bidirectional attack. It only becomes unidirectional in favor of C (C′) when legality is preferred over evidence.

In terms of framing, on the one hand, we observed in both conditions that, when moving from one frame to another with an isomorphic structure but contrary conclusions, individuals tend to modulate the strength of argument acceptance and rejection towards a more balanced assessment. However, the statistical differences are not significant, maybe due to the small sample size; hence, we plan to conduct more experiments in the future to address these issues. On the other hand, a tendency to change perceptions about the relative importance of the values involved in the argumentation frames were correlated to some extent with the perception of greater strength in lenient arguments. The data indicate that the value promoted by lenient arguments could be perceived as more important than the value promoted by harsh arguments. This is in line with findings about the influence of leniency bias on mock jury deliberations [39] and the outcome favorability as a strong determinant of individuals’ willingness to accept authoritative decisions [32].