University of Sussex
Browse
3708.pdf (541.83 kB)

Discovering fair representations in the data domain

Download (541.83 kB)
conference contribution
posted on 2023-06-09, 17:31 authored by Novi QuadriantoNovi Quadrianto, Viktoriia SharmanskaViktoriia Sharmanska, Oliver Thomas
Interpretability and fairness are critical in computer vision and machine learning applications, in particular when dealing with human outcomes, e.g. inviting or not inviting for a job interview based on application materials that may include photographs. One promising direction to achieve fairness is by learning data representations that remove the semantics of protected characteristics, and are therefore able to mitigate unfair outcomes. All available models however learn latent embeddings which comes at the cost of being uninterpretable. We propose to cast this problem as data-to-data translation, i.e. learning a mapping from an input domain to a fair target domain, where a fairness definition is being enforced. Here the data domain can be images, or any tabular data representation. This task would be straightforward if we had fair target data available, but this is not the case. To overcome this, we learn a highly unconstrained mapping by exploiting statistics of residuals -- the difference between input data and its translated version -- and the protected characteristics. When applied to the CelebA dataset of face images with gender attribute as the protected characteristic, our model enforces equality of opportunity by adjusting the eyes and lips regions. Intriguingly, on the same dataset we arrive at similar conclusions when using semantic attribute representations of images for translation. On face images of the recent DiF dataset, with the same gender attribute, our method adjusts nose regions. In the Adult income dataset, also with protected gender attribute, our model achieves equality of opportunity by, among others, obfuscating the wife and husband relationship. Analyzing those systematic changes will allow us to scrutinize the interplay of fairness criterion, chosen protected characteristics, and prediction performance.

Funding

EthicalML: Injecting Ethical and Legal Constraints into Machine Learning Models; G2034; EPSRC-ENGINEERING & PHYSICAL SCIENCES RESEARCH COUNCIL

History

Publication status

  • Published

File Version

  • Accepted version

Journal

Conference on Computer Vision and Pattern Recognition (CVPR)

ISSN

2575-7075

Publisher

Institute of Electrical and Electronics Engineers

Volume

1

Page range

8219-8228

Event name

CVPR 2019

Event location

Long Beach, California, US

Event type

conference

Event date

June 15th - June 20th 2019

Place of publication

Los Alamitos, CA

ISBN

9781728132938

Department affiliated with

  • Informatics Publications

Research groups affiliated with

  • Data Science Research Group Publications

Full text available

  • Yes

Peer reviewed?

  • Yes

Legacy Posted Date

2019-04-08

First Open Access (FOA) Date

2019-04-08

First Compliant Deposit (FCD) Date

2019-04-08

Usage metrics

    University of Sussex (Publications)

    Categories

    No categories selected

    Exports

    RefWorks
    BibTeX
    Ref. manager
    Endnote
    DataCite
    NLM
    DC