Trolley problem

Do you pull the lever? How did you decide what to do?

Link to TEDx Explainer on Youtube:

Learning Objectives


Today you will:

  • Learn about different schools of ethical thought
  • Examine case studies and examples in data ethics
  • Understand our responsibility as ethical analysts

Ethical Perspectives

3 common perspectives

Ethical Perspectives

1. Deontological Ethics (Immanuel Kant)

An ethical philosophy that says actions are good or bad according to a clear set of rules

  • Focus on rights, principles, and duties
  • Preserve human autonomy and dignity, justice, fairness, transparency and many more. (However rules can conflict with each other sometimes!)
  • This dignity creates an ethic that prevents us from acting in certain ways either toward other people or toward ourselves
  • Rules should apply to everyone (universal ethical rules)

Deontogological Ethics

“Act only according to that maxim whereby you can at the same time will that it should become a universal law.” — Immanuel Kant

Deon what?

  • Its name comes from the Greek word deon, meaning duty.

  • Actions that align with these rules are ethical, while actions that don’t aren’t.

Read more here

Ask yourself


In your project:

  • How might the dignity and fairness of each stakeholder be impacted by your project?

  • Are there any issues of trust and justice relevant to your project?

  • Does your project involve any conflicting moral duties to the participants or stakeholder rights?

Deontology: ✅ Differential Privacy

What is differential privacy?

Usage data is scrambled on-device before it ever leaves your phone. Google/apple’s servers never see raw personal data. Sharing aggregated level data is opt-in only.

Privacy is treated as a duty owed to the user — not a trade-off.

Differential Privacy \(\cdot\) Apple ML Research

Deontology — ❌ Clearview AI

What did Clearview AI do?

Scraped 3+ billion photos from social media without consent to build a facial recognition database sold to law enforcement.

Clearview violated their ethical duty and biometric privacy acts. Countries around the globe have been fined Clearview AI millions of dollars.

ACLU Illinois \(\cdot\) Clearview in Australia

Ethical Perspectives ctd.

2. Consequentialist ethics

An ethical philosophy that considers the outcomes rather than the intentions.

Two main examples of consequentialism:

  • Utilitarianism judges consequences by a greatest good for the greatest number standard.

  • Hedonism considers something is “good” if the consequence produces pleasure or avoids pain.

Ask yourself


In your project:

  • Which option will produce the most good and do the least harm?
  • Which option best serves the community as a whole?
  • Are you willing to accept difficult trade-offs?

Utilitarianism — ✅ Population Breast Cancer Screening (Voluntary)

Greater good

Early detection means reduced mortality.

Mass screening programs knowingly produce false positives (overdiagnosis).

Programs continue because of the population-level benefit.

Benefit outweighs the cost to individuals who receive incorrect results.
This is a transparent, accepted trade-off.

NCI: false positives and screening behaviour · PMC: overdiagnosis and trade-offs

Utilitarianism — ❌ Health Insurer AI Claim Denials

AI and Health

Active lawsuit alleging UnitedHealth illegally denied care by using an AI model to override determinations made by the patients’ physicians.

There are also allegations about an unacceptably high false-positive rate.

UnitedHealth has maximised an outcome related to greatest financial benefit, while also accepting a high false-positive rate that causes harm their customers.

CBS News · Healthcare Finance News

Ethical Perspectives ctd.

3. Virtue Based Ethics (Plato, Aristotle, Confucius, Mencius)

An ethical philosophy centred on the study of what behavior is morally right versus what people ought to do.

  • Virtue ethics is a philosophy developed by Aristotle and other ancient Greeks
  • It is the quest to understand and live a life of moral character
  • Act in a way that makes you a better person (culturally based)
  • Careful: The same aim can lead to conflicting actions

Ask yourself

In your project:

  • Am I honest about my findings accurately, even when they don’t support the desired conclusion?

  • Do I acknowledge uncertainty and the limitations of my model?

  • Will I raise concerns even when it’s uncomfortable ?

  • Am I handling people’s data with care and dignity?

Virtue Ethics — ✅ Māori Data Sovereignty

Data sovereignty

Te Mana Raraunga asserts that Māori data should be governed by Māori — built on whanaungatanga (trust), rangatiratanga (self-determination), and kotahitanga (collective benefit).

Data governance as an expression of identity, not compliance. Important for culturally sensitive data management.

Te Mana Raraunga · Principles of Māori Data Sovereignty

Quick Question



Indigenous Data Sovereignty

Does this work the same way in Australia?

ARDC \(\cdot\) NIAA

Virtue Ethics — ❌ Australia’s COVIDSafe App

COVIDSafe Data tracking

COVIDSafe was designed to track people who may have come into contact with an active COVID case. Large-scale act of genuine civic care and collective responsibility at a moment of national crisis.

While COVIDSafe was made with good intentions, one must consider whether governments have earned the trust for this type of large-scale data-tracing?

Australian Government · RMIT

Ethical Perspectives

Summary so far:

Differences between these school of ethics

  • Deontologist emphasizes duties or rules;
  • Consequentialist emphasizes the consequences of actions;
  • Virtue ethics emphasizes the moral character.

No single school of ethical thought is perfect!

The different forms of ethical philosophy provide a guide for living life!

Still need more examples? Here is an video explainer using Batman

O’Keefe, K., & Brien, D. O. (2018). Ethical data and information management: concepts, tools and methods. Kogan Page Publishers, chapter 2

The Avengers

Iron Man

  • Utilitarian
  • Bring the most happiness or well-being for the greatest number of people.

Captain America

  • Deontology
  • Focus on right or wrong rather than better or worse.

Thor

  • Virture ethics
  • Seek to be the best person he could be

What would an Avenger Do? by Mark D. White. https://media.wiley.com/product_data/excerpt/72/11180745/1118074572-234.pdf

Ethical Perspectives

Some further ethical perspectives for business

Ethical Perspectives

4. Common good and justice based ethics

  • Action should contribute to some greater good

5. Shareholder/stockholder theory (Milton Friedman)

  • Increase value to shareholders by maximizing profits

6. Stakeholder theory

  • Organization responsible to stakeholders
  • Employees, stockholders, society, environment

What ethical schools of thought apply?

Scenario

A large company surveys all of their current employees, measuring demographics and personality factors.

They hope to identify key personality factors that correspond with a successful time with the company.

Their hope is to use this data to identify which prospective employees they should hire.

Stakeholder theory

Shareholder theory

Deontological Ethics

Why consider ethical data practices?

Data is increasingly accessible and available

  • Can use for understanding our world
  • Can use to make decisions (data driven decision making)
  • Can use to inform policy decisions

Research ethics is often discussed in experimental settings

  • The fundamental ethical concerns do not change for data ethics

Data has an increasing impact on our lives

  • Data for good initiatives
  • Unethical uses of data
  • Open data increases the reach of data but also risks privacy

A real world example

Are rewards programs at supermarkets ethical? ABC news

And more recently, is their approach to discount pricing misleading? ABC news

Breakout discussion

What issues are raised with the data ethics of rewards programs in Australia currently?

Were you aware of all of these?

What schools of ethics are relevant?

The ACCC concluded that people can opt-out of rewards programs. What do you think given the cost of living crisis?

Ethics and Data

In your job and your responsibility

Data Ethics

Data science models affect everyone of us

  • getting a job
  • loan application
  • dating
  • social life
  • buying car insurance
  • medical screening

Ask yourself: Is your data ethically sourced? and Is the data being used ethically?

Data scientist’s responsibility

Data scientist’s responsibility

  • Informed Consent
  • Privacy
  • Fairness
  • Transparency
  • Accountability
  • Bias

A good data scientist understands ethical issues across the entire data pipeline!

This includes how data are collected, data privacy and any biases in the data. It also includes building accountable algorithms and evaluating the impact of your analysis on people!

Application of ethics for data scientist

Ethics on data

  • make sure data has informed consent from the participants
  • read the term and conditions
  • privacy screening has been done

Ethics on models/algorithm

  • make sure training data is representative
  • past population is not representative of the future population
  • algorithm that imposed fairness (no discrimination on any individual and aggregate outcome)

See more checklist

Guidelines for research involving people

Belmont report (1979)

  • It was written by the National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research.
  • It was created under the National Research Act of 1974.
  • Identifies basic ethical principles and guidelines that address ethical issues arising from the conduct of reaseach with human subject
  • Application to demonstrate the use of these principles

https://www.hhs.gov/ohrp/regulations-and-policy/belmont-report/read-the-belmont-report/index.html#xbound

What constitutes as research?

Defined in the report:

“Research” - an activity designed to test an hypothesis, permit conclusions to be drawn, and thereby to develop or contribute to generalizable knowledge”

and

“Practice” - interventions that are designed solely to enhance the well-being of an individual patient or client and that have a reasonable expectation of success”

https://www.hhs.gov/ohrp/regulations-and-policy/belmont-report/read-the-belmont-report/index.html#xbound

Basic Ethical Principles

Autonomy

Being able to deliberate and make personal goals or choices, and then act upon them

Basic Ethical Principles

  1. Respect for Persons
    • Acknowledge and respect autonomy
    • Protect those with diminished autonomy
  2. Beneficence
    • Do not cause harm
    • Minimize possible harms and maximise possible benefits
  3. Justice
    • Who ought to receive the benefits of research and bear its burdens?

https://www.hhs.gov/ohrp/regulations-and-policy/belmont-report/read-the-belmont-report/index.html#xbound

Applications

Applied ethical conduct

1. Informed consent

  • Participants have a right to understand what is going to happen
  • Application of respect for persons

2. Assessment of Risks and Benefits

  • Are the risks justified?
  • Is the study well designed?
  • Are there alternative lower risk designs possible?
  • Are potential benefits and risks communicated to participants?
  • Application of beneficence

https://www.hhs.gov/ohrp/regulations-and-policy/belmont-report/read-the-belmont-report/index.html#xbound

Applications

Applied ethical conduct ctd.

3. Selection of Subjects

  • Fairly offer research participation to all eligible
  • Some subjects preferred to as best able to bear risk (e.g., adults)
  • “injustice arises from social, racial, sexual and cultural biases institutionalized in society”
  • Careful not to overburden vulnerable subjects

https://www.hhs.gov/ohrp/regulations-and-policy/belmont-report/read-the-belmont-report/index.html#xbound

Ethical Principles — ❌ Robodebt

What happened with Robodebt

Robodebt was an Australian government welfare compliance scheme that automatically matched tax office income data against welfare recipients’ records to raise debt notices.

It was found to be unlawful because it used income averaging to calculate debts that often didn’t actually exist.

Violated all of the basic ethical principles!

Royal Commission Recommendations \(\cdot\) Class Action

Data Protection Laws and Regulations

Australia: Privacy Act 1988 which includes the Australian Privacy Principles (APPs) – principal data protection legislation.

Europe: General Data Protection Regulation (GDPR) adopted by the European Parliament which regulates “personal data”.

  • Passed in 2018 to provide strong protection on the collections, use and management of data
  • Rights of the individual
  • Information and access: you can access your data and see how it is processed
  • Right to request erasure of data
  • Need to comply if you offer goods and services in EU, collect data from EU individuals or are established in the EU

General Data Protection Regulation (GDPR)

One of the most influential data privacy regulations

It defines 8 user rights to privacy under the law.

  • The right to be informed.
  • The right of access.
  • The right to rectification.
  • The right to erasure.
  • The right to restrict processing.
  • The right to data portability.
  • The right to object.
  • The right to avoid automated decision-making.

https://www.freeprivacypolicy.com/blog/8-user-rights-gdpr/

Your turn


Scenario: Social Media and teen health

A study investigates the impact of social media content on mental health in teenagers aged 13–17.

The study uses data about screen time collected via a tracking app and periodic mood surveys over 6 months.

Discuss risks, benefits and approaches to mitigation/management.

Justice, data sources and representation

Ctd. From the Belmont report

Justice relates to the risks/benefits of the study at the same probability for all participants

  • Careful not to exhaust vulnerable populations by overstudying.

  • Big data sets often have challenges of representation not everyone in the population has an equal chance of representation

  • This can be one cause of biased predictions and unfair algorithms

  • Algorithms may be less accurate for particular subsets of the population

  • Much of big data doesn’t cost the participant. It’s created through scraping which will reflect the lack of representation in society or sociological and cultural power imbalances

Your turn

Scenario: Employment Demographics

A large company surveys all of their current employees, measuring demographics and personality factors.

They hope to identify key personality factors that correspond with a successful time with the company.

Their hope is to use this data to identify which prospective employees they should hire.

Discuss: What might be the ethical concerns with the data and this approach?

Reinforce Bias

Existing employee data reinforce any previous discriminatory hiring patterns

Counterfactual

What would have happened if we hired people who were different to those previously hired?

Focussing on statistical and data practices

  • So far we’ve focused on ethical and data practice in research generally.
  • The basic principles apply to ALL research, but many of the examples focus on experimental research
  • How can we translate this principles to a more statistical/data focus?

Ethical guidelines for statisticians

https://www.amstat.org/asa/files/pdfs/EthicalGuidelines.pdf

  1. Professional Integrity and Accountability
  1. Integrity of data and method
  1. Responsibilities to Science/Public/Funder/Client
  1. Responsibilities to Research Subjects
  1. Responsibilities to Research Team Colleagues
  1. Responsibilities to Other Statisticians or Statistics Practitioners
  1. Responsibilities Regarding Allegations of Misconduct

Alternative view

Gelman (2018). Ethics in statistical practice and communication: Five recommendations. Significance

  1. Prioritize open data and methods
  1. Be clear about the information and assumptions that go into statistical methods
  1. Respect for data
  1. Publication of criticisms
  1. Respect the limitations of statistics

Yet another perspective

https://deon.drivendata.org/

  • Much of what we spent the first half of the lecture discussing surrounds data collection and modeling
  • Identify other areas including:
    • Storage
    • Analysis
    • Deployment
  • These areas are less likely to be mentioned, but are equally important.

Wrap Up

How to be an ethical data scientist

  1. Ensure that you are following the ethical frameworks of the company you work for (e.g., I work for Monash, so I ensure that my research conforms to Australian standards through Monash procedures)
  1. Ensure you understand the different areas where unethical thinking/practices can be introduced
  1. Data ethics checklists like this can be useful!
  1. Keep learning and exploring different ethical perspectives on your own

Summary


Takeaways

  • Developed an understanding for data ethics principles
  • Developed some checks and balances for any future analysis
  • Started our journey to towards conducting ethical analytics

Example answer

RISKS

Participants may report extreme psychological stress.

App usage could reveal sensitive information about mental health struggles, identity, or relationships.

BENEFIT

Understanding how algorithmic content affects teen well-being could inform platform design regulations and school-based digital literacy programs.

MITIGATION/MANAGEMENT

  • Provide mental health referral resources throughout the study, not just at the end!
  • Obtain informed consent from both the teenager and a parent/guardian with plain-language explanations of what data is collected.
  • Allow withdrawal at any time with data deletion
  • Use appropriate data anonymisation

Extended Reference List

Deontology

  • ✅ Differential Privacy — privacy by design, opt-in consent, on-device data processing
  • ❌ Clearview AI — biometric data scraped without consent; $51.75M US settlement, €34M EU fine
  • ❌ Optus Data Breach (AU, 2022) — 10 million records exposed; over-retention, no privacy-by-design
  • ❌ DoorDash (CA, 2024) — sold customer data without notice or opt-out; CCPA violation

Extended Reference List (cont.)

Utilitarianism

  • ✅ Stats NZ IDI — linked government data for public benefit, with strong access controls
  • ❌ US Health Insurer AI Claim Denials (Cigna, UnitedHealth, Humana) — algorithmic denial of care
  • ❌ Facebook Emotional Contagion Experiment — manipulated 700,000 users’ feeds without consent for research benefit
  • ❌ OpenAI/Microsoft Copyright Lawsuits (2024) — training on copyrighted content; benefits to many, costs borne by few
  • ❌ UNHCR Rohingya Biometric Data — aid registration data shared with Myanmar military; 830,000 records

Extended Reference List (cont.)

Virtue Ethics

  • ✅ Māori Data Sovereignty / Te Mana Raraunga (NZ) — data governance as cultural expression and community self-determination
  • ✅ Helsinki AI Register (Finland) — public transparency into every AI system used by city government
  • ❌ COVIDSafe App (AU, 2020) — good intentions, poor architectural prudence; failed to achieve contact tracing goal
  • ❌ Robodebt (AU, 2015–2020) — automated welfare debt system; lack of courage, care, and practical wisdom in design and escalation
  • ❌ Queensland Police AI Domestic Violence Prediction — virtuous goal, but insufficient humility about algorithmic bias against over-policed communities

ETC5512