An essay on complex problems and simple solutions: Techno-fallacies of the information age¹

1.1.1The fallacy of autonomous technology, emanative development, inevitable adoption and happy endings

The technology is seen to drive itself and follow an internal logic that must unfold. It’s as if the technology resulted from immaculate conception, apart from human will or interests. This fallacy ignores the role of values, design, interests, contexts and contingencies in technical developments. Technology grows out of specific historical and social settings rather than being external to them.

Maximal gains from technology are expected and likely costs (if acknowledged) are minimized. Here we see assumptions of inevitability and irreversibility and optimism regarding outcomes. The assumption of an unstoppable, progressive (in both a technical and social sense) logic of technological determinism obscures responsibility and alternative approaches, as well as history. The development of surveillance technologies is hardly self-evident. The statement, “you can’t stop progress” cries out for social and cultural analysis of the meanings of progress. There are no natural laws that require particular technical developments and applications, and rarely can the social meanings and impacts of new technology be fully anticipated. We need to ask, “what are the likely consequences of using this technology, and how does its use compare to that of other technologies and to the consequences of doing nothing?”

1.1.2The fallacy that greater expenditures and more powerful, sophisticated and faster technology will continually yield benefits in a linear fashion

This is the American-inspired ideal that bigger is better; it might be termed a techno-phallicsy as well. With respect to opening up the coffers and ratcheting up the technology, we face issues of appropriateness of the technology, proportionality, threshold and time frame. There’s nothing inherently good or bad about the increased power of a technology. Our judgments must flow from analysis, not from the ability to increase the dosage. As Simmel (Coser, 1964) noted in situations of conflict, more may simply result in escalation, as opponents turn to equivalent means and discover ways of neutralization.

Greater speed can ironically offer advantages of currency and prevention but, in tandem with enhanced speed, there is less time for careful deliberation. Surveillance agents may tilt toward the kind of data that can be immediately gathered and processed, which may not be the kind of data that are most important or supportive of fairness or justice. With respect to linearity, as with medicine, one usually reaches a point where increases in dosage do not have equivalent therapeutic effects (e.g., some aspirin will help, but if you take the entire bottle, you may die).88 This aspect is central to the next fallacy.

1.1.3The fallacy that if some information is good, more is better

Issues of converting data to knowledge, data overkill, information glut, and drowning in data need consideration. For example, the East Germans were apparently able to tap most phone calls of West German and NATO officials, but do not appear to have been able to make very effective use of this. The quality not the quantity of information is what matters. The U.S. Census has found that samples of the U.S. population reveal more reliable data than is the case with data collection on the entire population.

Of course, the enlightenment heritage of asking questions and valuing knowledge is fundamental, but that doesn’t mean that all forms of personal information must be widely available or maximally processed (“e.g., “we never know when we might need it”). Openness can have negative consequences in some contexts (e.g., diplomacy, strategic endeavors, manners). At times, it is morally, strategically and practically better not to know, and at other times, “it’s none of your business”.

1.1.4The fallacy that technical solutions are always to be preferred

This is an error in both logic and definition. An even stronger version holds that when there is a technical problem (or better, a problem tied to the technology) there must be a technical solution. For example, a common response to problems created by Caller-ID (e.g., revealing unlisted numbers) or eavesdropping devices is to come up with a counter- technology that blocks or distorts. But such problems could also be addressed by regulating, prohibiting or limiting the technology and by education and manners. In a different context President Reagan’s “Star Wars” program was a continuation of the early response to atomic weapons which involved building bomb shelters and bigger bombs. An alternative was to define the problem as calling for understanding of mutual grievances, negotiation and disarmament. Renee Shelby (2020) offers a nice example of how responsible design and more explicit attention to values embedded in technological solutions to sexual violence (rape kit, reporting app) could create more meaningful, victim-centered interventions.

Rarely will complex problems existing within contexts of human liberty, conflict, and creativity yield best (or only) to simple technical solutions based on stand-alone, causal explanations. Even when all goes according to plan, at some point, the technology may malfunction, and given enough time or cases, it is likely to. A society that can only maintain civil behaviour by technical means is a society in deep yogurt.

1.1.5The fallacy that correlation must equal causality

This is a re-occurring failing of those in the persuasion business. Given dynamic conflict settings and a large number of interacting variables, it is often difficult to say with certainty that tactics work as well as advocates claim, nor that they fail as badly as critics claim, absent empirical specification and qualification.

A given cause or level of analysis is presented as sufficient for explaining and/or offering a simple (often unitary) solution. As the H.L. Mencken quote at the start of the paper implies, many problems have a multiplicity of causes at different levels, and the causes may show varied interactions (although if the Mencken quote is applied too universally, it illustrates its central idea). Also lurking here is the sub-fallacy of focusing on a minor cause. Misplaced responsibility is another sub-fallacy. Michael Welch (2003) shows how the moral panic of the 1990s led to the criminalization of immigration.

Even when very strong correlations are present, inferences of causality can be difficult to disprove. Consider a story about a young man who each evening played the flugelhorn in the town square. He refused any tips. When asked why he came each night to play, he responded, “to keep the elephants away”. He was told, “there are no elephants here”. To which he replied, “you see”.

1.1.6The fallacy that the facts speak for themselves

Data are not knowledge. Seeing should not automatically be equated with believing. The facts do not speak for themselves. They are inert artifacts. Any human knowledge, no matter how powerful, valid and useful is always abstracted out and partial. It represents a fraction of what might be attended to, and it might be intended to deceive.

Buried deep within this fallacy are sub-fallacies of literalism, universalism, sample representativeness and acontextuality. Here a normative principle or empirical finding is asserted with no qualifications or allowance for shadings, contingencies, mitigating circumstances, atypicality, or context.

The discretionary elements regarding measurement, standard-setting and interpretation must be seen. Alternative measures and seeing a fuller picture could suggest different meanings. To adequately interpret we need to know what is specific to the setting and how data determination and interpretation are constructed. “Objectivity” buried in tech narratives (along with strategic and heightened assumptions of risk) allow such practices to go unnoticed, much less critiqued.99

1.1.7The fallacy of delegating decision-making authority to the machine

When human life and life chances are involved, there are risks in automatically delegating decision-making authority to a machine, absent human review, as commonly happens. As William James (1950) argued, “the art of being wise is the art of knowing what to overlook”. Discretion – a capacity that is central to wise actions – can be severely limited by the automatic quality of the machine. As proponents claim, machines can be programmed to ignore variables deemed not to count by programmers such as religion or gender. Yet, because computer programs rely on selected broad categories of information, they are not equipped to deal with much of reality’s richness, atypical cases and sudden, unexpected developments the way a human can. The nightmare version of this is a war automatically generated in response to faulty data from sensors. Elephants, as well as soldiers (one’s own and adversaries), step on land mines.

1.1.8The fallacy of explicit agendas

Beyond goals that are not unitary, well thought out or in conflict, surveillance advocates may deceive subjects and the broader public in the reasons they give for using a tactic. It is wrong to automatically assume that the technology is applied only for the publicly stated, rather than unstated, reasons, including obfuscation. Such proclamations may receive legitimation when offered by elites. But even when sincerely offered as ideals, applications on the ground may serve other goals, whether for practitioners or the organization. For example Benjamin Fleury-Steiner (2019) nicely illustrates how the rhetoric surrounding a web platform Amazon designed for ICE (the U.S.’s Immigration and Customs Enforcement agency) obscures and distorts ICE’s objectives and abuses. Video surveillance ostensibly undertaken to counter theft has been used as a cover for gathering information during unionization drives.

In addition to asking what the agenda is, we must ask, “whose agenda? whose goals?”. In settings of crisis and risk, with the blurring of lines between the public and the private, ostensibly public goals may be undercut by commercial goals and ostensibly caring goals may be overwhelmed by coercive goals. In delegating enforcement of red-light violations to private contractors, cities such as San Diego and Washington, D.C. ran the risk of confounding the ostensible goal of public safety and justice with the business goal of profit maximization. Beyond the private sector, the city, too, while talking about traffic rules (whether for speeding or parking) may have as a more basic goal maximizing revenue. The mixing or obfuscation of goals with such delegation can also insulate and distance government from accountability.

1.1.9The fallacy of the sure-shot

Here we see a loose canon related to the fact that loose cannons may over-or under-shoot the target. This fallacy assumes that surveillance obtains its goal with laser-like precision and has no impact on adjacent or unintended targets and broader surroundings. But in a complex world much can go wrong and there are often missed shots and second order and spillover effects, as well as unrecognized tradeoffs.1010 The biases and failures of AI-fueled facial recognition technology (NIST, 2019) are illustrative, as are the early injustices of computer matching and profiling reported in Marx and Reichman (1984). Beyond errors, there may be subject displacement. As research on video surveillance and crime suggests, a problem may simply be moved. With respect to displacement a prosecutor notes, “the bad pennies never get lost, they just move from one pocket to another”.

Fallacies can be inextricably interlocked and function in tandem to spread obfuscation dust (note how this overlaps with several including that of the fail-safe system below).

1.1.10The fallacy of a passive, non-reactive environment

The previous fallacy too often assumes that environments, especially those where there are conflicts of interest, are passive rather than reactive. But subjects often find ways of thwarting surveillance and machine predictions via reverse engineering. Innovations in surveillance must be seen as variables in dynamic situations. When combined, humans and machines are co-constitutive in creating new contexts. The failure to note the changes (like generals always fighting the last war) is a factor in technological failures.

There is a Social Heisenberg principle in which the act affects what is acted upon in a changed environment. This may be particularly noticeable over time, as the effectiveness of a solution lessens. New controls create new challenges and opportunities. Every lock has a key. Any solution that one group of smart people creates can usually be circumvented by another group, whether through technical or social means. There are always tacks in the shoe.

1.1.11The fallacy of the short run

A focus on success in the present may mean a failure to consider longer-range negative consequences, including undesirable precedents and creeping encroachments upon liberty. Consider the story about the person falling from the top of a 200-floor building. As the 150th floor is passed, a friend on that floor asks, “how are you doing?”. The reply, “so far, so good”.

1.1.12The fallacy of the free lunch or painless dentistry

There are no free meals, and your teeth may hurt when the Novocaine wears off. Those under the sway of this fallacy may ignore or fail to see collateral costs, especially when they involve powerless groups and future costs. But any format or structure both channels and excludes, generating trade-offs.

In gaining one advantage we may lose another (e.g., breadth vs. depth, validity vs. low cost, short vs. long term gains, solve one problem while creating or worsening another). If nothing else (and in a highly interdependent world with imperfect knowledge, there almost always is something else), a given use of resources involves forgone opportunity costs. In the case of police for example, where might some of the vast expenditures for equipment go instead (training, better working conditions) or to social services involved in order maintenance and community support?

1.1.13The fallacy of the 100% fail-safe system

In complex environments rich in uncertainty, machines and those who run them can be imprecise and fallible. As the work of Perrow (1984) on Three Mile Island, Vaughan (1996) on the Challenger disaster, and Tenner (1996) on a broad range of technologies suggests, mistakes and unintended consequences adhere and inhere.

Claims such as, “but the computer says” or “it’s in the computer” are offered as equivalent to laws of nature. But being “in” the computer guarantees neither accuracy, nor appropriateness. Human agents have set the rules for collecting, entering, and analyzing the data; even machines may do it through processes incomprehensible to humans. Yet given the messiness of the real world the best made plans are too often awrysome. Consider some of the problems initially found with the use of electronic location-monitoring devices for those under house arrest in which mylar in the walls gave false readings or that eating a poppy seed bagel can cause one to fail a drug test. Even where the measure is in principle valid and reliable, it may be incompetently applied or thwarted by subjects.

1.1.14The fallacy that the means will never come to determine the end

Albert Einstein observed, “perfection of means and confusion over ends seems to characterize our age”. To a person with a can opener, the whole world looks like a can. Where here is a way, there is often a will. The means too often impact and can even determine the end; a related form involves the ritualistic danger of the means becoming the end. (Merton, 1956).

It is vital for civilization (if not always for self- or organizational interests) that public policy start with goals – asking what is to be accomplished – instead of starting with a tool and asking how can it be applied. Problems should drive solutions rather than the reverse.

1.1.15The fallacy that technology will always remain the solution rather than become the problem

All of the above contribute to this capstone fallacy. Today’s solutions often become (or contribute to) tomorrow’s problems. Contrary to Dr. Frankenstein’s experience, this fallacy involves the belief that we can fully control the technology, rather than the reverse. An aspect of this fallacy is failing to ask what the technology might lead to and what precedents it might create. A questionable means applied on behalf of an urgent goal is sometimes justified by the argument that “we can control it and will apply it only in this one narrow area”. But given power differentials and temptation, there is a tendency toward surveillance creep, as a once-restricted tactic spreads to new uses, users a and targets and becomes normalized as “just business as usual”.

The fallacies considered thus far tie directly to technology. We next turn to those more directly involving values, social consequences and beliefs about persons.

1.2.1The fallacy of neutrality

When asked, “isn’t the technology neutral?” George Orwell reportedly replied, “yes and so is the jungle”. The neutrality argument can conceal the hidden hands and unequal social terrain often lurking in the background. Rob Pallitto (2013) offers a way of unmasking the haze here in suggesting the adoption of a bargaining perspective. In asking a series of questions involving surveillance subjects and the machines and human agents behind them, buried claims may be surfaced, such as how unequal the bargaining setting is. That also applies to the design of a tool, with regard to 10 principles suggested by the Design Justice Network, such as: are the voices of the subjects/communities impacted by the tool considered in its design? and, are traditional ways that may be currently working considered?

The distancing of the technology from its creators and its automatic (rather than direct human) application do not mean that the technology has equal impacts across society or that moral responsibility has been eliminated. The fallacy can mask power relations and draw attention from social and ethical questions involving fairness, reciprocity and accountability and racist and sexist outcomes.1111 Singer Tom Lehrer’s parody of Wernher von Braun’s rockets applies: “where they go up and what they do when they come down depends on the technology, not me.”

The neutrality fallacy denies the political character of much surveillance. If questions are merely technical, and if surveillance is neutral in its impact, then there is no need for discussion or negotiation, and the structural roots of the problem that remain unresolved.

A technology can fail to be neutral in several ways. Inequality in power and resources determines which groups are best positioned to sponsor the development of new technologies.

Regardless of who develops the technology, it is rarely equally available or useful across society. The rich have little need to shoplift or to sleep behind malls, let alone under bridges and may persons can’t afford fast track access at airports. Nor are the “equal opportunity” monitoring tools such as video-cameras and phone and computer monitoring that are applied to workers as likely to be used for executives.

Regardless of who develops the technology, it is rarely equally available or useful across society. In some ways modern means of communication and surveillance, from the printing press through to the Internet, tilt toward greater equality and participation. But the individual who can access the Internet just as a large organization can is not therefore made equal to them. To the extent that access is restricted by proprietary codes or cost, the ease or difficulty of using the tool is irrelevant. Furthermore, while the cost along with the skill required to use much of the technology has steadily decreased, the skill required to understand and fix it has in general steadily increased and is controlled by those owning the software, increasing dependence on, and trust in, distanced, unseen specialists not fully acting in the subject’s interest.1212

1.2.2The fallacy that personal information is just another kind of property or material to be bought, sold, altered and manipulated

Personal information has a special quality, something that under some conditions is sacred and inviolate. It is not the same as raw materials or office furniture.

Europe recognizes this to a greater extent than has the United States. The former’s concern to protect the dignity of the person (as broadly defined) restricts the sale of personal information, while in the U.S. the focus has been on regulating specific technologies, rather than applying broad principles such as respect for human dignity.

Those in the data warehouse and analysis business often show inconsistency and self-serving attitudes, toward personal information. On the one hand, in the collection phase, personal data is treated as a free public good, like air. It is just there waiting for whoever wants it. Yet once it is harvested, it becomes private property to be used as the possessor wishes, even including selling it back to those it pertains to (e.g., credit scores). This contrasts with other free goods such as radio transmissions or a photo of a person taken in a public place.

Even if viewing personal information as property is appropriate, there is likely a need for a safety net or equity principle guaranteeing a minimum threshold for withholding information. There must be limits on the extent to which data about persons is treated simply as a free good and commodity.

1.2.3The fallacy of implied consent and free choice

Consent and choice are very difficult concepts to assess. To be meaningful choice should imply genuine alternatives and refusal costs that are not wildly exorbitant, absent that we have trickery, double-talk, and the frequent spoiled fruit of inequitable relationships. Individuals face cognitive limits on what they can know, and factors at many levels limit the amount of freedom in a “free” or willing choice. Certainly, one can protect one’s privacy by not using a phone or computer or driving a car. But that is almost like saying if you breathe polluted air or drink contaminated water, you consent to these environmental circumstances. The conditions of modern life are often such that one can hardly avoid choosing actions that are subject to surveillance. While the surveillance may be justified on other grounds, it is disingenuous to call it a free and informed choice. Nor is it necessarily preferable because it seems less directly coercive.

1.2.4The fallacy of the velvet glove

The softer means of implied consent in the above can be beguiling. It is hard to say "no" if you are unaware of what is going on and are not inconvenienced. Just because personal data can be collected relatively silently and non-invasively, does not justify doing it apart from the goals. Judge Brandeis noted that vigilance was most needed when purposes were benign. The same might be said for the softer, non-invasive means. Soft surveillance (Marx, 2017) is still surveillance. The challenges of “mandatory voluntarism” and “voluntary servitude” (Rosen, 1996) are vital topics for a democratic society and the person as a maker of genuine choices.

The seemingly non-problematic nature of choices that are not genuine choices may take attention away from other aspects. Even when there is genuinely free consent for given types of personal data collection and technological operations, that should not be seen as an open invitation to any type of data analysis, usage and sharing. Such is often the case with AI, and the merging of big, unrelated data sets gathered under different conditions.

1.2.5The fallacy of meeting rather than creating consumer needs

This fallacy overlaps the notion of free choice and implies that consumption “needs”, including perceptions of an appropriate level of security, rise up spontaneously within the individual rather than being generated by entrepreneurs. Of course, there is always a mixture, but advocates seek to soften the harsher edges of manipulation by claiming that they are simply giving the public what it wants. Here, they deny the role played by propaganda, marketing strategies, unseen manipulation and the creation of desires, rather than the meeting of needs (however hard the latter is to define).

1.2.6The fallacy that individuals are best controlled through fear

Nineteenth-century positivist theories of law and the presumed link between rationality and conformity gave great force to the belief that the more anxiety people felt over discovery, apprehension and sanctioning for normative violations, the better their behaviour would be.

It is often impractical to watch everyone all the time. The effort to engender fear and apprehension is an important part of some contemporary surveillance rhetoric. Things are usually much more complex than the messengers claim. Democratic principles require respect for the individual and toleration of a degree of disorder as a concomitant of a free society.

Certainly, in many settings accountability increases proportionally with the visibility of compliance. But no such simple statement can be adequate for all complex human situations. Other factors being equal, good behavior is more likely to occur when individuals have participated in setting the standards, understand the reasons for them and feel respected by an organization. Climates of fear and suspicion work against innovation and adoption of new ways.

Even holding apart issues of effectiveness, means have a moral quality as well as ends. Policies to effect behavior out of fear, coercion (whether technical or social), and threat of punishment, while sometimes needed, need not be unleashed or unreflectively favored over other approaches.

1.2.7The fallacy that because it is possible to skate on thin ice, it is wise to do so

When critics of technology point out possible negative consequences, a standard response is, "that’s never happened” or “that couldn’t happen”. Yet foresight remains better than hindsight. It is unwise to wait until the dam breaks to decide to reinforce or move it. There was a time when the nuclear accident at Three Mile Island and the Alaska Exxon oil spill had not happened as well. It is not enough to show that a tactic has thus far been without disastrous consequences.

1.2.8The fallacy of re-arranging the deck chairs on the titanic instead of looking for icebergs

A cartoon shows a seated man with a knife stuck in his back. A doctor leaning over him says, “this will have to come out, but of course it doesn’t address the deeper problem”. And so it is with many quick-fix technical solutions to organizational or social problems; they are sometimes no more than Band-Aids on a haemorrhaging wound (e.g., removing benches from public areas as a response to homelessness).1313

The emphasis may be on the wrong problem as a result of bad analysis or political factors. Technical solutions are often sold as cleaner, quicker and less expensive – as something that can be done – relative to the messy business of dealing with people and trying to understand the complex cultural and organizational causes for many problems. As noted, the mindset may in turn lead to the distorted view that "if you can’t fix the real problem, fix whatever the technology permits you to fix". In such cases Thoreau’s (2015) observation in Walden holds:

but lo! Men have become the tools of their tools.

Our inventions are wont to be pretty toys

which distract our attention from serious things.

Attention to deeper causes and bigger pictures (with the implication that broad changes in a system may be required) do not play very well in the short run when the focus is on the bottom line or in a moral panic.

While symptoms must often be treated as well as broader causes (if a bathtub is overflowing, the floor needs to be mopped, as well as turning off the faucet. Deeper analysis might even lead to the pessimistic conclusion that there are no realistic solutions, or none that do not bring other problems.

1.2.9The fallacy of confusing data with knowledge and techniques with wisdom

To varying degrees, all these fallacies reflect a broad, unquestioned faith in the efficacy of science and technology and the denial of trade-offs or the legitimation of specious trade-offs. Technologies for extracting and processing personal information are neither givens nor an automatic reflection of the natural world. The social hands behind the curtains and the levers of the machine need to be scoped out. Above all, technical mastery, or even knowledge, must never be equated with wisdom.