---
title: DDIA 2-е изд. (Kleppmann & Riccomini, «Designing Data-Intensive Applications», 2nd ed.) — глава 14: Doing the Right Thing
source: materials/DDIA-2nd-edition.pdf, стр. 609–626
конспект: TASK-45.39, извлечено pymupdf 2026-09-17; ниже полный «грязный» текст главы + выжимка
приоритет: опционально
статус: выжимка перенесена в knowledge-base.md / ddia-map.md — см. _map.md
---

# Глава 14. Делать правильные вещи (этика) (Doing the Right Thing)

## Выжимка

1. Данные — о людях: privacy, consent, минимизация сбора и retention; человеческое достоинство как ограничение дизайна.
2. Инженеры отвечают за последствия систем: предвзятость ML, слежка, dark patterns; ACM Code of Ethics как рамка.
3. Security и threat modeling; законодательство (GDPR и аналоги) как внешний constraint архитектуры.
4. Практика: этические вопросы задавать на этапе дизайна, а не постфактум; dark data («собираем на всякий случай») — риск, а не актив.
5. Связь с системным дизайном: retention-политики, data minimization и deletion pipeline — такие же инженерные требования, как p99 или durability: закладываются с этапа требований, а не прикручиваются потом.

## Полный текст (грязная выгрузка)

CHAPTER 14
Doing the Right Thing
Feeding AI systems on the world’s beauty, ugliness, and cruelty, but expecting it to reflect only
the beauty is a fantasy.
—Vinay Uday Prabhu and Abeba Birhane, “Large Datasets: A Pyrrhic Win for
Computer Vision?” (2020)
In the final chapter of this book, let’s take a step back. Throughout the book we
have examined a wide range of architectures for data systems, evaluated their pros
and cons, and explored techniques for building reliable, scalable, and maintainable
applications. However, we have left out a fundamental part of the discussion, which
we should now fill in.
Every system is built for a purpose; every action we take has both intended and
unintended consequences. The purpose may be as simple as making money, but the
consequences may be far-reaching. We, the engineers building these systems, have
a responsibility to carefully consider those consequences and to ensure that our
decisions do not cause harm.
We talk about data as an abstract thing, but remember that many datasets are about
people: their behavior, their interests, their identity. We must treat such data with
humanity and respect. Users are humans too, and human dignity is paramount [1].
Software development increasingly involves making important ethical choices. There
are guidelines to help software engineers navigate these issues, such as the ACM Code
of Ethics and Professional Conduct [2], but they are rarely discussed, applied, and
enforced in practice. As a result, engineers and product managers sometimes take a
cavalier attitude to privacy and potential negative consequences of their products [3, 4].
A technology is not good or bad in itself—what matters is how it is used and how it
affects people. This is true of a software system like a search engine in much the same
way as it is of a weapon like a gun. The ethical responsibility is ours to bear; it is not
585


sufficient for software engineers to focus exclusively on the technology and ignore its
consequences.
In contrast to much of computing, however, the concepts at the heart of ethics are
not fixed or determinate in their precise meaning; they require interpretation, which
may be subjective [5]. What makes something “good” or “bad” is not well defined,
and serious discourse on the subject among computing professionals is lacking [6].
Reasoning about ethics is difficult, but it is too important to ignore. What does
this entail? Ethics is not going through a checklist to confirm you comply; it’s a
participatory and iterative process of reflection, in dialog with the people involved,
with accountability for the results [7].
Predictive Analytics
Predictive analytics is a major part of why people are excited about big data and AI.
It’s also an area that is fraught with ethical dilemmas. Using data analysis to predict
the weather, or the spread of diseases, is one thing [8]; it is another matter to predict
whether a convict is likely to reoffend, whether an applicant for a loan is likely to
default, or whether an insurance customer is likely to make expensive claims [9]. The
latter have a direct effect on individual people’s lives.
Naturally, payment networks want to prevent fraudulent transactions, banks want
to avoid bad loans, airlines want to avoid hijackings, and companies want to avoid
hiring ineffective or untrustworthy people. From their point of view, the cost of a
missed business opportunity is low, but the cost of a bad loan or a problematic
employee is much higher, so it is expected for organizations to want to be cautious. If
in doubt, they are better off saying no.
However, as algorithmic decision making becomes more widespread, someone who
has (accurately or falsely) been labeled as risky by an algorithm may suffer a large
number of those “no” decisions. Systematically being excluded from jobs, air travel,
insurance coverage, property rental, financial services, and other key aspects of
society is such a large constraint of an individual’s freedom that it has been called
“algorithmic prison” [10]. In countries that respect human rights, the criminal justice
system presumes innocence until proven guilty; on the other hand, automated sys‐
tems can systematically and arbitrarily exclude a person from participating in society
without any proof of guilt, and with little chance of appeal.
Bias and Discrimination
Decisions made by an algorithm are not necessarily any better or any worse than
those made by a human. Every person is likely to have biases, even if they actively
try to counteract them, and discriminatory practices can become culturally institu‐
tionalized. There is hope that basing decisions on data, rather than subjective and
586 
| 
Chapter 14: Doing the Right Thing


instinctive assessments by people, could be more fair and give a better chance to
people who are often overlooked or disadvantaged in the traditional system [11].
When we develop predictive analytics and AI systems, we are not merely automating
a human’s decision by using software to specify the rules for when to say yes or no;
we are leaving the rules themselves to be inferred from data. However, the patterns
learned by these systems are opaque: even if the data indicates a correlation, we may
not know why. If the input to an algorithm carries a systematic bias, the system will
most likely learn and amplify that bias in its output [12].
In many countries, anti-discrimination laws prohibit treating people differently
depending on protected traits such as ethnicity, age, gender, sexuality, disability,
or beliefs. Other features of a person’s data may be analyzed, but what happens if
they are correlated with protected traits? For example, in racially segregated neigh‐
borhoods, a person’s postal code or even their IP address is a strong predictor of race.
Put like this, it seems ridiculous to believe that an algorithm could somehow take
biased data as input and produce fair and impartial output from it [13, 14]. Yet this
belief often seems to be implied by proponents of data-driven decision making—an
attitude that has been satirized as “machine learning is like money laundering for
bias” [15].
Predictive analytics systems merely extrapolate from the past; if the past is discrimi‐
natory, they codify and amplify that discrimination [16]. If we want the future to
be better than the past, moral imagination is required, and that’s something only
humans can provide [17]. Data and models should be our tools, not our masters.
Responsibility and Accountability
Automated decision making opens the question of responsibility and accountability
[17]. If a human makes a mistake, they can be held accountable, and the person
affected by the decision can appeal. Algorithms make mistakes too, but who is
accountable if they go wrong [18]? When a self-driving car causes an accident, who
is responsible? If an automated credit scoring algorithm systematically discriminates
against people of a particular race or religion, is there any recourse? If a decision by
your ML system comes under judicial review, can you explain to the judge how the
algorithm made its decision? People should not be able to evade their responsibility
by blaming an algorithm.
Credit rating agencies are a classic example of collecting data to make decisions about
people. A bad credit score makes life difficult, but at least a credit score is normally
based on relevant facts about a person’s actual borrowing history, and any errors in
the record can be corrected (although the agencies normally do not make this easy).
Scoring algorithms based on machine learning, however, typically use a much wider
range of inputs and are much more opaque, making it harder to understand how
Predictive Analytics 
| 
587


a particular decision has come about and whether someone is being treated in an
unfair or discriminatory way [19].
A credit score summarizes “How did you behave in the past?,” whereas predictive
analytics usually work on the basis of “Who is similar to you, and how did people like
you behave in the past?” Drawing parallels to others’ behavior implies stereotyping
people—for example, based on where they live (a close proxy for race and socioeco‐
nomic class). What about people who get put in the wrong bucket? Furthermore, if a
decision is incorrect because of erroneous data, recourse is almost impossible [17].
Much data is statistical in nature, which means that even if the probability distribu‐
tion on the whole is correct, individual cases may well be wrong. For example, if
the average life expectancy in your country is 80 years, that doesn’t mean you’re
expected to drop dead on your 80th birthday. From the average and the probability
distribution, you can’t say much about the age to which one particular person will
live. Similarly, the output of a prediction system is probabilistic and may well be
wrong in individual cases.
A blind belief in the supremacy of data for making decisions is not only delusional
but also positively dangerous. As data-driven decision making becomes more wide‐
spread, we will need to figure out how to avoid reinforcing existing biases, how
to make algorithms accountable and transparent, and how to fix them when they
inevitably make mistakes.
We will also need to figure out how to realize the positive potential of data and
prevent it from being used to harm people. For example, analytics can reveal financial
and social characteristics of people’s lives. On the one hand, this power could be used
to focus aid and support to help those who need it most. On the other hand, it is
sometimes used by predatory businesses seeking to identify vulnerable people and
sell them risky products such as high-cost loans or worthless college degrees [17, 20].
Feedback Loops
Even with predictive applications that have less immediately far-reaching effects on
people, such as recommendation systems, there are difficult issues that we must
confront. When services become good at predicting the content users want to see,
they may end up showing people only opinions they already agree with, leading to
echo chambers in which stereotypes, misinformation, and polarization can breed.
We are already seeing the impact social media echo chambers can have on election
campaigns.
When predictive analytics affect people’s lives, particularly pernicious problems arise
because of self-reinforcing feedback loops. For example, consider the case of employ‐
ers using credit scores to evaluate potential hires. You may be a good worker with
a good credit score, but suddenly find yourself in financial difficulties due to a
588 
| 
Chapter 14: Doing the Right Thing


misfortune outside of your control. As you miss payments on your bills, your credit
score suffers, and you will be less likely to find work. Joblessness pushes you toward
poverty, which further worsens your score, making it even harder to find employ‐
ment [17]. It’s a downward spiral due to poisonous assumptions, hidden behind a
camouflage of mathematical rigor and data.
As another example of a feedback loop, economists found that when gas stations
in Germany introduced algorithmic prices, competition was reduced and prices for
consumers went up because the algorithms learned to collude [21].
We can’t always predict when such feedback loops may happen. However, many
consequences can be predicted by thinking about the entire system (not just the
computerized parts, but also the people interacting with it)—an approach known as
systems thinking [22]. We can try to understand how a data analysis system responds
to different behaviors, structures, or characteristics. Does the system reinforce and
amplify existing differences between people (e.g., making the rich richer or the poor
poorer), or does it try to combat injustice? Even with the best intentions, we must
beware of the possibility of unintended consequences. 
Privacy and Tracking
Besides the problems of predictive analytics—that is, using data to make automated
decisions about people—there are ethical problems with data collection itself. What is
the relationship between the organizations collecting data and the people whose data
is being collected?
When a system stores only data that a user has explicitly entered, because they want
the system to store and process it in a certain way, the system is performing a service
for the user; the user is the customer. But when a user’s activity is tracked and logged
as a side effect of other things they are doing, the relationship is less clear. The service
no longer just does what the user tells it to do; it takes on interests of its own, which
may conflict with the user’s interests.
Tracking behavioral data has become increasingly important for user-facing features
of many online services. Tracking which search results are clicked helps improve
the ranking of search results; providing recommendations (“people who liked X also
liked Y”) helps users discover interesting and useful things; A/B tests and user flow
analysis can help indicate how a UI might be improved. Those features require some
amount of tracking of user behavior, and users benefit from them.
However, depending on a company’s business model, tracking often doesn’t stop
there. If the service is funded through advertising, the advertisers are the actual
customers, and the users’ interests take second place. Tracking data becomes more
detailed, analyses become further-reaching, and data is retained for a long time in
order to build up detailed profiles of each person for marketing purposes.
Privacy and Tracking 
| 
589


Now the relationship between the company and the user whose data is being collected
starts looking quite different. The user is given a free service and is coaxed into
engaging with it as much as possible. The tracking of the user primarily serves not
that individual but rather the needs of the advertisers who are funding the service.
This relationship can be appropriately described with a word that has more sinister
connotations: surveillance.
Surveillance
As a thought experiment, try replacing the word data with surveillance, and observe
whether common phrases still sound so good [23]. How about this: “In our
surveillance-driven organization we collect real-time surveillance streams and store
them in our surveillance warehouse. Our surveillance scientists use advanced analyt‐
ics and surveillance processing in order to derive new insights.”
This thought experiment is unusually polemic for this book, Designing Surveillance-
Intensive Applications, but strong words are needed to emphasize this point. In our
attempts to make software “eat the world” [24], we have built the greatest mass
surveillance infrastructure ever seen. We are rapidly approaching a world in which
every inhabited space contains at least one internet-connected microphone, in the
form of smartphones, smart TVs, voice-controlled assistant devices, baby monitors,
and even children’s toys that use cloud-based speech recognition. Many of these
devices have a terrible security record [25].
What is new compared to the past is that digitization has made it easy to collect
large amounts of data about people. Surveillance of our location and movements,
our social relationships and communications, our purchases and payments, and our
health data has become almost unavoidable. A surveillance organization may end
up knowing more about a person than that person knows about themselves—for
example, identifying illnesses or economic problems before that individual is aware of
them.
Even the most totalitarian and repressive regimes of the past could only dream of
putting a microphone in every room and forcing every person to constantly carry a
device capable of tracking their location and movements. Yet the benefits that we get
from digital technology are so great that we now voluntarily accept this state of total
surveillance. The difference is just that the data is being collected by corporations to
provide us with services, rather than government agencies seeking control [26].
Not all data collection necessarily qualifies as surveillance, but examining it as
such can help us understand our relationship with the data collector. Why are we
seemingly happy to accept surveillance by corporations? Perhaps you feel you have
nothing to hide—in other words, you are totally in line with existing power struc‐
tures, you are not a marginalized minority, and you needn’t fear persecution [27].
Not everyone is so fortunate. Or perhaps it’s because the purpose seems benign—it’s
590 
| 
Chapter 14: Doing the Right Thing


not overt coercion and conformance, merely better recommendations and more per‐
sonalized marketing. However, combined with the discussion of predictive analytics
from the last section, that distinction seems less clear.
We are already seeing behavioral data on car driving, tracked by cars without drivers’
consent, affecting their insurance premiums [28], and health insurance coverage
that depends on people wearing a fitness tracking device. When surveillance is used
to make decisions that hold sway over important aspects of life, such as insurance
coverage or employment, it starts to appear less benign. Data analysis can also reveal
surprisingly intrusive things—for example, the movement sensor in a smartwatch or
fitness tracker can be used to work out what you are typing (e.g., passwords) with
fairly good accuracy [29]. Sensor accuracy and algorithms for analysis are only going
to get better.
Consent and Freedom of Choice
We might assert that users voluntarily choose to use services that track their activity,
agreeing to the terms of service and privacy policy and consenting to data collection.
We might even claim that users are receiving a valuable service in return for the
data they provide, and that the tracking is necessary in order to provide the service.
Undoubtedly, social networks, search engines, and various other free online services
are valuable to users—but this argument has problems.
First, we should ask why the tracking is necessary. Some forms of tracking directly
feed into improving features for users—for example, tracking the click-through rate
on search results can help improve a search engine’s result ranking and relevance,
and tracking which products customers tend to buy together can help an online
shop suggest related products. However, when tracking user interaction for content
recommendations, or to build user profiles for advertising purposes, it is less clear
whether this is genuinely in the user’s interest. Is it necessary only because the ads pay
for the service?
Second, most users have little knowledge of what data they are feeding into our
databases or how it is retained and processed—and most privacy policies do more
to obscure than to illuminate. Without understanding what happens to their data,
users cannot give meaningful consent. Often, data from one user also says things
about other people who are not users of the service and who have not agreed to
any terms. The derived datasets that we discussed in the last few chapters—in which
data from the entire user base may have been combined with behavioral tracking and
external data sources—are precisely the kinds of data that users cannot meaningfully
understand.
Moreover, data is extracted from users through a one-way process, not a relationship
with true reciprocity or a fair value exchange. There is no dialogue, no option
for users to negotiate how much data they provide and what service they receive
Privacy and Tracking 
| 
591


in return. The relationship between the service and the user is asymmetric and
one-sided; the terms are set by the service, not by the user [30, 31].
In the European Union, the General Data Protection Regulation (GDPR) requires
that consent must be “freely given, specific, informed, and unambiguous” and that
the user must be able to “refuse or withdraw consent without detriment”—otherwise,
it is not considered “freely given.” Any request for consent must be written “in an
intelligible and easily accessible form, using clear and plain language,” and “silence,
pre-ticked boxes or inactivity [do not] constitute consent” [32].
Consent is not the only basis for lawful processing of personal data under the GDPR.
There are also several other bases, including to comply with other legislation or to
protect somebody’s life. In addition, the legitimate interest basis permits certain uses
of data (e.g., for fraud prevention) [33] (which fraudsters would presumably not
consent to). Nevertheless, consent is the most frequently used basis for personal data
processing in internet services.
You might argue that a user who does not consent to surveillance can simply choose
not to use a service. But this choice is not free either. If a service is so popular that it is
“regarded by most people as essential for basic social participation” [30], then it is not
reasonable to expect people to opt out of using it—its use is effectively mandatory.
For example, in most Western social communities, it has become the norm to carry
a smartphone, to use social networks for socializing, and to use Google for finding
information. Especially when a service has network effects, there is a social cost to
people choosing not to use it.
Declining to use a service because of its user tracking policies is easier said than
done. These platforms are designed specifically to engage users. Many use game
mechanics and tactics common in gambling to keep users coming back [34]. Even if
a user gets past this, declining to engage is an option for only the small number of
people who are privileged enough to have the time and knowledge to understand its
privacy policy, and who can afford to potentially miss out on social participation or
professional opportunities that may have arisen if they had participated in the service.
For people in a less privileged position, there is no meaningful freedom of choice;
surveillance becomes inescapable.
Privacy and Use of Data
Sometimes people claim that “privacy is dead” on the grounds that some users
are willing to post all sorts of things about their lives to social media, sometimes
mundane and sometimes deeply personal. However, this claim is false and rests on a
misunderstanding of the word privacy.
Having privacy does not mean keeping everything secret; it means having the free‐
dom to choose what to reveal to whom, what to make public, and what to keep secret.
592 
| 
Chapter 14: Doing the Right Thing


The right to privacy is a decision right: it enables each person to decide where they
want to be on the spectrum between secrecy and transparency in each situation [30].
It is an important aspect of a person’s freedom and autonomy.
For example, someone who suffers from a rare medical condition might be very happy
to provide their private medical data to researchers if it might help the development
of treatments for their condition. However, this person must have a choice over who
may access this data and for what purpose. If information about their condition could
hinder their access to medical insurance or employment, for example, this person
would probably be much more cautious about sharing their data.
When data is extracted from people through surveillance infrastructure, privacy
rights are not necessarily eroded but rather transferred to the data collector. Com‐
panies that acquire data essentially say, “Trust us to do the right thing with your
data,” which means that the right to decide what to reveal and what to keep secret is
transferred from the individual to the company.
The companies in turn choose to keep much of the outcome of this surveillance
secret, because to reveal it would be perceived as creepy and would harm their
business model (which relies on knowing more about people than other companies
do). Intimate information about users is revealed only indirectly—for example, in the
form of tools for targeting advertisements to specific groups of people (such as those
suffering from a particular illness).
Even if particular users cannot be personally reidentified from the bucket of people
targeted by a particular ad, they have lost their agency about the disclosure of some
intimate information. It is not the user who decides what is revealed to whom on the
basis of their personal preferences—it is the company that exercises the privacy right
with the goal of maximizing its profit.
Many companies want to avoid being perceived as creepy, avoiding the question of
how intrusive their data collection actually is and instead focusing on managing user
perceptions. And even these perceptions are often managed poorly—for example,
something may be factually correct, but if it triggers painful memories, the user may
not want to be reminded about it [35]. With any kind of data, we should expect the
possibility that it is wrong, undesirable, or inappropriate in some way, and we need
to build mechanisms for handling those failures. Whether something is “undesirable”
or “inappropriate” is of course down to human judgment; algorithms are oblivious
to such notions unless we explicitly program them to respect human needs. As
engineers of these systems, we must be humble, accepting and planning for such
failings.
Privacy settings that allow a user of an online service to control which aspects of their
data other users can see are a starting point for handing back some control to users.
However, regardless of the setting, the service itself still has unfettered access to the
Privacy and Tracking 
| 
593


data and is free to use it in any way permitted by the privacy policy. Even if the service
promises not to sell the data to third parties, it usually grants itself unrestricted rights
to process and analyze the data internally, often going much further than what is
overtly visible to users.
This kind of large-scale transfer of privacy rights from individuals to corporations
is historically unprecedented [30]. Surveillance has always existed, but it used to be
expensive and manual, not scalable and automated. Trust relationships have always
existed—for example, between a patient and their doctor, or between a defendant and
their attorney—but in these cases the use of data has been strictly governed by ethical,
legal, and regulatory constraints. Internet services have made it much easier to amass
huge amounts of sensitive information without meaningful consent, and to use it at
massive scale without users understanding what is happening to their private data.
Data as Assets and Power
Since behavioral data is a byproduct of users interacting with a service, it is some‐
times called “data exhaust”—suggesting that the data is worthless waste material.
Viewed this way, behavioral and predictive analytics can be seen as a form of recy‐
cling that extracts value from data that would have otherwise been thrown away.
More correct would be to view it the other way around. From an economic point
of view, if targeted advertising is what pays for a service, then the user activity that
generates behavioral data could be regarded as a form of labor [36]. One could go
even further and argue that the application with which the user interacts is merely
a means to lure users into feeding more and more personal information into the
surveillance infrastructure [30]. The delightful human creativity and social relation‐
ships that often find expression in online services are cynically exploited by the data
extraction machine.
Personal data is a valuable asset, as evidenced by the existence of data brokers oper‐
ating in secrecy, purchasing, aggregating, analyzing, and reselling people’s personal
data, mostly for marketing purposes [20]. Startups are valued by their user numbers,
or “eyeballs”—that is, by their surveillance capabilities.
Because the data is valuable, many people want it. Of course, companies want it—
that’s why they collect it in the first place. But governments want it too, and they
may seek to obtain it by means of secret deals, coercion, legal compulsion, or simply
theft [37]. When a company goes bankrupt, the personal data it has collected is one
of the assets that get sold. And because data is difficult to secure, breaches happen
disconcertingly often.
These observations have led critics to say that data is not just an asset, but a “toxic
asset” [37], or at least “hazardous material” [38]. Maybe data is not the new gold, or
the new oil, but rather the new uranium [39]. Even if we think that we are capable of
594 
| 
Chapter 14: Doing the Right Thing


preventing abuse of data, whenever we collect it, we need to balance the benefits with
the risk of it falling into the wrong hands. Computer systems may be compromised
by criminals or hostile foreign intelligence services, data may be leaked by insiders,
the company may fall into the hands of unscrupulous management that does not
share our values, or the country may be taken over by a regime that has no qualms
about compelling us to hand over the data.
As that observation suggests, when collecting data, we need to consider not just
today’s political environment, but all possible future governments. There is no guar‐
antee that every government elected in the future will respect human rights and
civil liberties, and as Bruce Schneier observes, “It is poor civic hygiene to install
technologies that could someday facilitate a police state” [40].
“Knowledge is power,” as the old adage goes. And furthermore, “To scrutinize others
while avoiding scrutiny oneself is one of the most important forms of power” [41].
This is why totalitarian governments want surveillance: it gives them the power
to control the population. Although today’s technology companies are not overtly
seeking political power, the data and knowledge they have accumulated—much of it
surreptitiously, outside of public oversight—nevertheless gives them a lot of power
over our lives [42].
Remembering the Industrial Revolution
Data is the defining feature of the information age. The internet, data storage and
processing, and software-driven automation are having a major impact on the global
economy and human society. As our daily lives and social organization have been
changed by information technology, and will probably continue to radically change in
the coming decades, comparisons to the Industrial Revolution come to mind [17, 26].
The Industrial Revolution came about through major technological and agricultural
advances, and it brought sustained economic growth and significantly improved
living standards in the long run—yet it also came with major problems. Pollution of
the air (due to smoke and chemical processes) and the water (from industrial and
human waste) was dreadful. Factory owners lived in splendor, while urban workers
often lived in cramped and unsanitary housing and worked long hours in harsh
conditions. Child labor was common, including dangerous and poorly paid work in
mines.
It took a long time before safeguards were established, such as environmental protec‐
tion regulations, safety protocols for workplaces, laws prohibiting child labor, and
health inspections for food. Undoubtedly, the cost of doing business increased when
factories were no longer allowed to dump their waste into rivers, sell tainted foods, or
exploit workers. But society as a whole benefited hugely from these regulations, and
few of us would want to return to a time before [17].
Privacy and Tracking 
| 
595


Just as the Industrial Revolution had a dark side that needed to be managed, our
transition to the information age has major problems that we need to confront and
solve [43, 44]. The collection and use of data is one of those problems. In the words of
Bruce Schneier [26]:
Data is the pollution problem of the information age, and protecting privacy is the
environmental challenge. Almost all computers produce information. It stays around,
festering. How we deal with it—how we contain it and how we dispose of it—is
central to the health of our information economy. Just as we look back today at the
early decades of the industrial age and wonder how our ancestors could have ignored
pollution in their rush to build an industrial world, our grandchildren will look back
at us during these early decades of the information age and judge us on how we
addressed the challenge of data collection and misuse.
We should try to make them proud.
Legislation and Self-Regulation
Data protection laws might be able to help preserve individuals’ rights. For example,
the GDPR states that personal data must be “collected for specified, explicit and
legitimate purposes and not further processed in a manner that is incompatible
with those purposes” and be “adequate, relevant and limited to what is necessary in
relation to the purposes for which [it is] processed” [32].
However, this principle of data minimization runs directly counter to the philosophy
of big data, which is to maximize data collection, to combine the collected data with
other datasets, and to experiment and explore in order to generate new insights.
Exploration means using data for unforeseen purposes, which the GDPR states is
the opposite of the “specified and explicit” purposes for which the data must have
been collected. While this regulation has had some effect on the online advertising
industry [45], it has been weakly enforced [46] and does not seem to have led to
much of a change in culture and practices across the wider tech industry.
Companies that collect lots of data about people broadly oppose regulation as being
a burden and a hindrance to innovation. To some extent, that opposition is justified.
For example, sharing medical data creates clear risks to privacy but also potential
opportunities: how many deaths could be prevented if data analysis were able to
help us achieve better diagnostics or find better treatments [47]? Overregulation may
prevent such breakthroughs. It is difficult to balance the potential opportunities with
the risks [41].
Fundamentally, we need a culture shift in the tech industry with regard to personal
data. We should stop regarding users as metrics to be optimized, and remember that
they are humans who deserve respect, dignity, and agency. We should self-regulate
our data collection and processing practices in order to establish and maintain the
trust of the people who depend on our software [48]. And we should take it upon
596 
| 
Chapter 14: Doing the Right Thing


ourselves to educate end users about how their data is used rather than keeping them
in the dark.
We should allow each individual to maintain their privacy (i.e., their control over
their own data) and not steal that control from them through surveillance. Our
individual right to control our data is like the natural environment of a national park:
if we don’t explicitly protect and care for it, it will be destroyed. It will be the tragedy
of the commons, and we will all be worse off for it. Ubiquitous surveillance is not
inevitable. We are still able to stop it.
As a first step, we should not retain data forever, but purge it as soon as it is no
longer needed, and minimize what we collect in the first place [48, 49]. Data you don’t
have is data that can’t be leaked, stolen, or compelled by governments to be handed
over. Overall, culture and attitude changes will be necessary. As people working in
technology, if we don’t consider the societal impact of our work, we’re not doing our
job [50]. 
Summary
This brings us to the end of the book. We have covered a lot of ground:
• In Chapter 1 we contrasted analytical and operational systems, compared the
•
cloud to self-hosting, weighed up distributed and single-node systems, and dis‐
cussed balancing the needs of your business with the needs of your users.
• In Chapter 2 we saw how to define several nonfunctional requirements, such as
•
performance, reliability, scalability, and maintainability.
• In Chapter 3 we explored a spectrum of data models, including the relational,
•
document, and graph models, event sourcing, and DataFrames. We also looked at
examples of various query languages, including SQL, Cypher, SPARQL, Datalog,
and GraphQL.
• In Chapter 4 we discussed storage engines for OLTP (LSM-trees and B-trees) and
•
analytics (column-oriented storage), as well as indexes for information retrieval
(full-text and vector search).
• In Chapter 5 we examined different ways of encoding data objects as bytes and
•
how to support evolution as requirements change. We also compared several
ways that data flows between processes: via databases, service calls, workflow
engines, and event-driven architectures.
• In Chapter 6 we studied the trade-offs between single-leader, multi-leader, and
•
leaderless replication. We also looked at consistency models such as read-after-
write consistency and sync engines that allow clients to work offline.
• In Chapter 7 we looked at sharding, including strategies for rebalancing, request
•
routing, and secondary indexing.
Summary 
| 
597


• In Chapter 8 we covered transactions, considering durability, how various isola‐
•
tion levels (read committed, snapshot isolation, and serializable) can be achieved,
and how atomicity can be ensured in distributed transactions.
• In Chapter 9 we surveyed fundamental problems that occur in distributed sys‐
•
tems (network faults and delays, clock errors, process pauses, crashes) and saw
how they make it difficult to correctly implement even something seemingly
simple like a lock.
• In Chapter 10 we went on a deep dive into various forms of consensus and the
•
consistency model (linearizability) it enables.
• In Chapter 11 we dug into batch processing, building up from simple chains
•
of Unix tools to large-scale distributed batch processors using distributed filesys‐
tems or object stores.
• In Chapter 12 we generalized batch processing to stream processing and dis‐
•
cussed the underlying message brokers, CDC, fault tolerance, and processing
patterns such as streaming joins.
• In Chapter 13 we explored a philosophy of streaming systems that allows dispa‐
•
rate data systems to be integrated, systems to be evolved, and applications to be
scaled more easily.
Finally, in this last chapter, we took a step back and examined some ethical aspects
of building data-intensive applications. We saw that although data can be used to do
good, it can also do significant harm: making decisions that seriously affect people’s
lives and are difficult to appeal against, leading to discrimination and exploitation,
normalizing surveillance, and exposing intimate information. We also run the risk of
data breaches, and we may find that a well-intentioned use of data has unintended
consequences.
Given the large impact that software and data have on the world, we as engineers
must remember that we carry a responsibility to work toward the kind of world that
we want to live in: a world that treats people with humanity and respect. Let’s work
together toward that goal.
References
[1] David Schmudde. “What If Data Is a Bad Idea?” schmud.de, August 2024.
Archived at perma.cc/ZXU5-XMCT
[2] Association for Computing Machinery. “ACM Code of Ethics and Professional
Conduct.” acm.org, 2018. Archived at perma.cc/SEA8-CMB8
[3] Igor Perisic. “Making Hard Choices: The Quest for Ethics in Machine Learning.”
linkedin.com, November 2016. Archived at perma.cc/DGF8-KNT7
598 
| 
Chapter 14: Doing the Right Thing


[4] John Naughton. “Algorithm Writers Need a Code of Conduct.” theguardian.com,
December 2015. Archived at perma.cc/TBG2-3NG6
[5] Deborah G. Johnson and Mario Verdicchio. “Ethical AI Is Not About AI.”
Communications of the ACM, volume 66, issue 2, pages 32–34, January 2023.
doi:10.1145/3576932
[6] Ben Green. “‘Good’ Isn’t Good Enough.” At NeurIPS Joint Workshop on AI for
Social Good, December 2019. Archived at perma.cc/H4LN-7VY3
[7] Marc Steen. “Ethics as a Participatory and Iterative Process.” Communications of
the ACM, volume 66, issue 5, pages 27–29, April 2023. doi:10.1145/3550069
[8] Logan Kugler. “What Happens When Big Data Blunders?” Communications of the
ACM, volume 59, issue 6, pages 15–16, June 2016. doi:10.1145/2911975
[9] Miri Zilka. “Algorithms and the Criminal Justice System: Promises and Challenges
in Deployment and Research.” At University of Cambridge Security Seminar Series,
March 2023. Archived at archive.org
[10] Bill Davidow. “Welcome to Algorithmic Prison.” theatlantic.com, February 2014.
Archived at archive.org
[11] Don Peck. “They’re Watching You at Work.” theatlantic.com, December 2013.
Archived at perma.cc/YR9T-6M38
[12] Leigh Alexander. “Is an Algorithm Any Less Racist Than a Human?” theguar‐
dian.com, August 2016. Archived at perma.cc/XP93-DSVX
[13] Jesse Emspak. “How a Machine Learns Prejudice.” scientificamerican.com,
December 2016. perma.cc/R3L5-55E6
[14] Rohit Chopra, Kristen Clarke, Charlotte A. Burrows, and Lina M. Khan. “Joint
Statement on Enforcement Efforts Against Discrimination and Bias in Automated
Systems.” ftc.gov, April 2023. Archived at perma.cc/YY4Y-RCCA
[15] Maciej Cegłowski. “The Moral Economy of Tech.” idlewords.com, June 2016.
Archived at perma.cc/L8XV-BKTD
[16] Greg Nichols. “Artificial Intelligence in Healthcare Is Racist.” zdnet.com, Novem‐
ber 2020. Archived at perma.cc/3MKW-YKRS
[17] Cathy O’Neil. Weapons of Math Destruction: How Big Data Increases Inequality
and Threatens Democracy. Crown Publishing, 2016. ISBN: 9780553418811
[18] Julia Angwin. “Make Algorithms Accountable.” nytimes.com, August 2016.
Archived at archive.org
Summary 
| 
599


[19] Bryce Goodman and Seth Flaxman. “European Union Regulations on Algorith‐
mic Decision-Making and a ‘Right to Explanation.’” At ICML Workshop on Human
Interpretability in Machine Learning, June 2016. Archived at arxiv.org
[20] United States Senate Committee on Commerce, Science, and Transportation,
Office of Oversight and Investigations, Majority Staff. “A Review of the Data Broker
Industry: Collection, Use, and Sale of Consumer Data for Marketing Purposes.” Staff
Report, commerce.senate.gov, December 2013. Archived at perma.cc/32NV-YWLQ
[21] Stephanie Assad, Robert Clark, Daniel Ershov, and Lei Xu. “Algorithmic Pricing
and Competition: Empirical Evidence from the German Retail Gasoline Market.”
Journal of Political Economy, volume 132, issue 3, pages 723–771, March 2024.
doi:10.1086/726906
[22] Donella H. Meadows and Diana Wright. Thinking in Systems: A Primer. Chelsea
Green Publishing, 2008. ISBN: 9781603580557
[23] Daniel J. Bernstein. “Listening to a ‘big data’/‘data science’ talk. Mentally translat‐
ing ‘data’ to '‘surveillance’: ‘…everything starts with surveillance…’” x.com, May 2015.
Archived at perma.cc/EY3D-WBBJ
[24] Marc Andreessen. “Why Software Is Eating the World.” a16z.com, August 2011.
Archived at perma.cc/3DCC-W3G6
[25] J. M. Porup. “‘Internet of Things’ Security Is Hilariously Broken and Getting
Worse.” arstechnica.com, January 2016. Archived at archive.org
[26] Bruce Schneier. Data and Goliath: The Hidden Battles to Collect Your Data and
Control Your World. W. W. Norton, 2015. ISBN: 9780393352177
[27] The Grugq. “Nothing to Hide.” grugq.tumblr.com, April 2016. Archived at
perma.cc/BL95-8W5M
[28] Federal Trade Commission. “FTC Takes Action Against General Motors for
Sharing Drivers’ Precise Location and Driving Behavior Data Without Consent.”
ftc.gov, January 2025. Archived at perma.cc/3XGV-3HRD
[29] Tony Beltramelli. “Deep-Spying: Spying Using Smartwatch and Deep Learning.”
Masters thesis, IT University of Copenhagen, December 2015. Archived at arxiv.org
[30] Shoshana Zuboff. “Big Other: Surveillance Capitalism and the Prospects of
an Information Civilization.” Journal of Information Technology, volume 30, issue 1,
pages 75–89, April 2015. doi:10.1057/jit.2015.5
[31] Michiel Rhoen. “Beyond Consent: Improving Data Protection Through Con‐
sumer Protection Law.” Internet Policy Review, volume 5, issue 1, March 2016.
doi:10.14763/2016.1.404
600 
| 
Chapter 14: Doing the Right Thing


[32] “Regulation (EU) 2016/679 of the European Parliament and of the Council of 27
April 2016.” Official Journal of the European Union, L 119/1, May 2016.
[33] UK Information Commissioner’s Office. “What Is the ‘Legitimate Interests’
Basis?” ico.org.uk. Archived at perma.cc/W8XR-F7ML
[34] Tristan Harris. “How a Handful of Tech Companies Control Billions of Minds
Every Day.” At TED2017, April 2017. Archived at archive.org
[35] Carina C. Zona. “Consequences of an Insightful Algorithm.” At GOTO Berlin,
November 2016.
[36] Imanol Arrieta Ibarra, Leonard Goff, Diego Jiménez Hernández, Jaron Lanier,
and E. Glen Weyl. “Should We Treat Data as Labor? Moving Beyond ‘Free.’” Ameri‐
can Economic Association Papers Proceedings, volume 108, pages 38–42, May 2018.
doi:10.1257/pandp.20181003
[37] Bruce Schneier. “Data Is a Toxic Asset, So Why Not Throw It Out?” schneier.com,
March 2016. Archived at perma.cc/4GZH-WR3D
[38] Cory Scott. “Data is not toxic—which implies no benefit—but rather hazardous
material, where we must balance need vs. want.” x.com, March 2016. Archived at
perma.cc/CLV7-JF2E
[39] Mark Pesce. “Data Is The New Uranium—Incredibly Powerful And Amazingly
Dangerous.” theregister.com, November 2024. Archived at perma.cc/NV8B-GYGV
[40] Bruce Schneier. “Mission Creep: When Everything Is Terrorism.” schneier.com,
July 2013. Archived at perma.cc/QB2C-5RCE
[41] Lena Ulbricht and Maximilian von Grafenstein. “Big Data: Big Power Shifts?”
Internet Policy Review, volume 5, issue 1, March 2016. doi:10.14763/2016.1.406
[42] Ellen P. Goodman and Julia Powles. “Facebook and Google: Most Powerful and
Secretive Empires We’ve Ever Known.” theguardian.com, September 2016. Archived at
perma.cc/8UJA-43G6
[43] Judy Estrin and Sam Gill. “The World Is Choking on Digital Pollution.” washing‐
tonmonthly.com, January 2019. Archived at perma.cc/3VHF-C6UC
[44] A. Michael Froomkin. “Regulating Mass Surveillance as Privacy Pollution:
Learning from Environmental Impact Statements.” University of Illinois Law Review,
volume 2015, issue 5, August 2015. Archived at perma.cc/24ZL-VK2T
[45] Pengyuan Wang, Li Jiang, and Jian Yang. “The Early Impact of GDPR Compli‐
ance on Display Advertising: The Case of an Ad Publisher.” Journal of Marketing
Research, volume 61, issue 1, April 2023. doi:10.1177/00222437231171848
[46] Johnny Ryan. “Don’t Be Fooled by Meta’s Fine for Data Breaches.” The Economist,
May 2023. Archived at perma.cc/VCR6-55HR
Summary 
| 
601


[47] Jessica Leber. “Your Data Footprint Is Affecting Your Life in Ways You Can’t
Even Imagine.” fastcompany.com, March 2016. Archived at archive.org
[48] Maciej Cegłowski. “Haunted by Data.” idlewords.com, October 2015. Archived at
archive.org
[49] Sam Thielman. “You Are Not What You Read: Librarians Purge User Data to
Protect Privacy.” theguardian.com, January 2016. Archived at archive.org
[50] Jez Humble. “It’s a cliché that people get into tech to ‘change the world.’ So
then, you have to actually consider what the impact of your work is on the world.
The idea that you can or should exclude societal and political discussions in tech is
idiotic. It means you’re not doing your job.” x.com, April 2021. Archived at perma.cc/
3NYS-MHLC
602 
| 
Chapter 14: Doing the Right Thing