Research Article
BibTex RIS Cite
Year 2022, Issue: 71, 265 - 307, 31.12.2022
https://doi.org/10.26650/annales.2022.71.0002

Abstract

References

  • Accesswire, ‘72% of Resumes are Never Seen by Employers’ (16 February 2016) <https://www. accesswire.com/436847/72-of-Resumes-are-Never-Seen-by-Employers> accessed 20 June 2021. google scholar
  • Ali M et al., ‘Discrimination Through Optimization: How Facebook’s ad Delivery Can Lead to Skewed Outcomes’ (2019) Proceedings of the ACM on Human-Computer Interaction. <https:// arxiv.org/pdf/1904.02095.pdf> accessed 20 June 2021. google scholar
  • Angelopoulos C et al., ‘Study Of Fundamental Rights Limitations For Online Enforcement Through Self-Regulation’ (Study Report of Institute for Information Law, University of Amsterdam) (2016) <https://scholarlypublications.universiteitleiden.nl/access/item%3A2869513/view> accessed 20 June 2021. google scholar
  • Angwin J, Mattu S and Larson J, ‘The tiger mom tax: Asians are nearly twice as likely to get a higher price from Princeton Review’. ProPublica (1 September 2015) <https://www.ProPublica.org/article/ asians-nearly-twice-as-likely-to-get-higher-price-from-princeton-review> accessed 20 June 2021. google scholar
  • Angwin J, Scheiber N and Tobin A, ‘Dozens of companies are using Facebook to exclude older workers from job ads’ ProPublica (20 December 2017) <https://www.propublica.org/article/ facebook-ads-age-discrimination-targeting> accessed 20 June 2021. google scholar
  • Angwin J, Tobin A and Varner M, ‘Facebook (still) letting housing advertisers exclude users by race’ ProPublica (21 November 2017) <https://www.propublica.org/article/facebook-advertising-discrimination-housing-race-sex-national-origin> accessed 20 June 2021. google scholar
  • Avraham R, ‘Discrimination and Insurance’ in Lippert-Rasmussen K. (ed.), The Routledge Handbook of the Ethics of Discrimination, (Routledge, UK 2018), pp. 335-347. google scholar
  • Bartlett R et al., ‘Consumer-Lending Discrimination in the FinTech Era’ (2019) (National Bureau of Economic Research Working Papers 25943). google scholar
  • Barocas S and Selbst A D, ‘Big Data’s Disparate Impact’ (2016) 104 California Law Review, 671-732. google scholar
  • Bathaee Y, ‘The Artificial Intelligence Black Box And The Failure Of Intent And Causation’ (2018) 31:2 Harvard Journal of Law & Technology, 890-938. google scholar
  • Beck S et al., ‘Künstliche Intelligenz und Diskriminierung Herausforderungen und Lösungsansatze’ (2019) (Whitepaper from Plattform Lernende Systeme) <https://www.plattform-lernende-systeme.de/publikationen-details/kuenstliche-intelligenz-und-diskriminierung-herausforderungen-und-loesungsansaetze.html> accessed 20 June 2021. google scholar
  • Biao v. Denmark (Grand Chamber) (App no 38590/10) ECHR 24 May 2016. google scholar
  • Bodo B et al., ‘Tackling the algorithmic control crisis-the technical, legal, and ethical challenges of research into algorithmic agents’ (2017) 19 Yale Journal of Law & Technology, 133-180. google scholar
  • Borgesius F Z, ‘Discrimination, Artificial Intelligence and Algorithmic Decision-making’ (Study Report) (Council of Europe, Strasbourg 2018) <https://rm.coe.int/discrimination-artificial-intelligence-and-algorithmic-decision-making/1680925d73> accessed 20 June 2021. google scholar
  • Borgesius F J Z, ‘Strengthening legal protection against discrimination by algorithms and artificial intelligence’ (2020) 24 10 The International Journal of Human Rights, 1572- 1593. google scholar
  • Bundesanstalt für Finanzdienstleistungsaufsicht, ‘Big data meets artificial intelligence: Challenges and implications for the supervision and regulation of financial services’ (2018) (Study) www.bafin.de/SharedDocs/Downloads/EN/dl_bdai_studie_en.html> accessed 20 June 2021. google scholar
  • Burrell J, ‘How the machine ‘thinks’: understanding opacity in machine learning algorithms’ (2016) Big Data & Society Doi:10.1177/2053951715622512. google scholar
  • Case 170/84, Bilka-Kaufhaus GmbH and Karin Weber von Hartz, (Judgment of the Court of 13 May 1986.) google scholar
  • Case C-83/14, “CHEZ Razpredelenie Bulgaria” AD v Komisia za zashtita ot diskriminatsia, (Judgment of the Court (Grand Chamber) of 16 July 2015). google scholar
  • Case C443/15, David L. Parris v Trinity College Dublin and Others (Judgment of the Court (First Chamber) of 24 November 2016). google scholar
  • Case C-414/16, Vera Egenberger v Evangelisches Werk für Diakonie und Entwicklung e.V. (Judgment of the Court (Grand Chamber) of 17 April 2018). google scholar
  • Chander S and Jakubowska E, ‘EU’s AI law needs major changes to prevent discrimination and mass surveillance’ EDRI (2021) <https://edri.org/our-work/eus-ai-law-needs-major-changes-to-prevent-discrimination-and-mass-surveillance/> accessed 20 June 2021. google scholar
  • Charter of Fundamental Rights of the European Union, 2012/C 326/02 OJ C 326, [2012]. google scholar
  • Commission (EC) ‘Proposal for a Regulation Of The European Parliament And Of The Council Laying Down Harmonised Rules On Artificial Intelligence (Artificial Intelligence Act) And Amending Certain Union Legislative Acts’ COM(2021) 206 final, 21 April 2021. google scholar
  • Corner N, ‘Revealed: How your name, your ethnicity, and even your e-mail address could be adding hundreds of pounds to your car insurance premium’ Dailymail (29 June 2018) www.dailymail.co.uk/femail/article-5896561/How-email-address-add-hundreds-car-insurance-premium.html> accessed 20 June 2021. google scholar
  • Council Directive (EC) 2000/43 of 29 June 2000 implementing the principle of equal treatment between persons irrespective of racial or ethnic origin [2000] OJ L 180. google scholar
  • Council Directive (EC) 2000/78 of 27 November 2000 establishing a general framework for equal treatment in employment and occupation [2000] OJ L 303. google scholar
  • Council Directive (EC) 2001/95 of the European Parliament and of the Council of 3 December 2001 on general product safety [2001] OJ L 11. google scholar
  • Council of Europe, ‘Explanatory Report to the Protocol No. 12 to the Convention for the Protection of Human Rights and Fundamental Freedoms’ (2000) <https://rm.coe.int/09000016800cce48> accessed 20 June 2021. google scholar
  • Council of Europe, ‘Algorithms and Human Rights: Study On The Human Rights Dimensions Of Automated Data Processing Techniques (In Particular Algorithms) And Possible Regulatory Implications’ (Study Report) (DGI(2017)12) (2018). google scholar
  • Council Regulation (EC) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation) [2016] OJ L 119. google scholar
  • Craglia M et al., ‘Artificial Intelligence: A European Perspective’ (Joint Research Centre Working Papers) (2018). google scholar
  • Danks D and London A J, ‘Algorithmic Bias in Autonomous Systems’ (2017) IJCAI’17: Proceedings of the 26th International Joint Conference on Artificial Intelligence, 4691-4697. google scholar
  • Dastin J, ‘Amazon scraps secret AI recruiting tool that showed bias against women’ Reuters (11 October 2018) <https://www.reuters.com/article/us-amazon-com-jobs-automation-in...-airecruiting-tool-that-showed-bias-against-women-idUSKCN1MK08G> accessed 20 June 2021. google scholar
  • Data Protection Working Party Guidelines on Automated individual decision-making and Profiling for the purposes of Regulation 2016/679 Wp251rev.01. [2018]. google scholar
  • Datta A, Tschantz M C and Datta A, ‘Automated Experiments on Ad Privacy Settings A Tale of Opacity, Choice, and Discrimination’ (2015) Proceedings on Privacy Enhancing Technologies <https://www.andrew.cmu.edu/user/danupam/dtd-pets15.pdf> accessed 20 June 2021. google scholar
  • Delipetrev B, Tsinarakii C and Kostic U, ‘Historical Evolution of Artificial Intelligence’ (Technical Report) (Publications Office of the European Union, Luxembourg 2020) https://publications.jrc. ec.europa.eu/repository/handle/JRC120469 accessed 20 June 2021. google scholar
  • Dwork C et al., ‘Fairness Through Awareness, Proceedings of the 3rd Innovations in Theoretical Computer Science Conference’ (2012) ITCS ‘12 (Association for Computing Machinery), 214226. google scholar
  • Edwards, L and Veale M, ‘Slave to the algorithm: Why a right to an explanation is probably not the remedy you are looking for’ (2017) 16 Duke Law & Technology Review, 18-84. google scholar
  • Espinoza J and Murgia M, ‘Europe attempts to take leading role in regulating uses of AI’ Financial Times (24 April 2021) <https://www.ft.com/content/360faa3e-4110-4f38-b618-dd695deece90> accessed 20 June 2021. google scholar
  • European Commission, Proposal for a Council Directive on implementing the principle of equal treatment between persons irrespective of religion or belief, disability, age or sexual orientation [2008] {SEC(2008) 2180} {SEC(2008) 2181} /* COM/2008/0426 final - CNS 2008/0140 */. google scholar
  • European Commission, ‘Special Eurobarometer Discrimination in the EU in 2015’ (Report) (2015). google scholar
  • European Commission’s High-Level Expert Group on Artificial Intelligence, ‘A Definition of AI: Main Capabilities and Disciplines’ (2019) (Report) <https://digital-strategy.ec.europa.eu/en/ library/definition-artificial-intelligence-main-capabilities-and-scientific-disciplines> accessed 20 June 2021. google scholar
  • European Commission, ‘A Union of Equality: Gender Equality Strategy 2020-2025’ [2020] COM (2020) 152. google scholar
  • European Commission, ‘Internal Market, Industry, Entrepreneurship and SMEs’ <https://ec.europa. eu/growth/single-market/ce-marking/manufacturers_en> accessed 20 June 2021. google scholar
  • European Commission, ‘Harmonised Standards’ <https://ec.europa.eu/growth/single-market/ european-standards/harmonised-standards_en> accessed 20 June 2021. google scholar
  • European Convention for the Protection of Human Rights and Fundamental Freedoms (1950) <https://www.echr.coe.int/Documents/Convention_ENG.pdf> accessed 20 June 2021. google scholar
  • European Union Agency for Fundamental Rights, ‘#BigData: Discrimination in data-supported decision making’ (Report) (2018) Doi:10.2811/343905. google scholar
  • European Union Agency for Fundamental Rights, ‘Getting the Future Right-Artificial Intelligence and Fundamental Rights’ (2020) Doi:10.2811/774118. google scholar
  • Fairness, Accountability, and Transparency in Machine Learning <https://www.fatml.org/> accessed 20 June 2021. google scholar
  • Favaretto M, De Clercq E and Elger B S, ‘Big Data and discrimination: perils, promises and solutions. A systematic review’ (2019) 6 Journal of Big Data <https://doi.org/10.1186/s40537-019-0177-4> accessed 20 June 2021. google scholar
  • Financial Conduct Authority, ‘Pricing practices in the retail general insurance sector: Household insurance’ (2018) <https://www.fca.org.uk/publication/thematic-reviews/tr18-4.pdf> accessed 20 June 2021. google scholar
  • Stan F, ‘History, Motivations, and Core Themes’ in Keith Frankish and William W. Ramsey, (eds.), The Cambridge Handbook Of Artificial Intelligence (Cambridge University Press, UK 2014). google scholar
  • Fuster A et al., ‘Predictably Unequal? The Effects of Machine Learning on Credit Markets’ (2020). SSRN <https://ssrn.com/abstract=3072038> accessed 20 June 2021. google scholar
  • Future of Life Institute, ‘The Asilomar AI principles’ (2017) <https://futureoflife.org/ai-principles/> accessed 20 June 2021. google scholar
  • Gillis T B and Spiess J L, ‘Big Data and Discrimination’ (2019) 86 The University of Chicago Law Review, pp. 459-487. google scholar
  • Hacker P., ‘Teaching Fairness to Artificial Intelligence: Existing and Novel Strategies against Algorithmic Discrimination under EU Law’ (Pre-print version) (2019) <https://papers.ssrn.com/ sol3/papers.cfm?abstract_id=3164973&download=yes> accessed 20 June 2021. google scholar
  • Heinrichs B, ‘Discrimination in the age of artificial intelligence’ (2021) AI & Society Springer. <https://doi.org/10.1007/s00146-021-01192-2> accessed 20 June 2021. google scholar
  • Hildebrandt M and Gutwirth S, ‘Concise Conclusions: Citizens Out of Control’ in Hildebrandt M and Gutwirth S (eds.) Profiling the European Citizen: Cross-Disciplinary Perspectives (Springer, 2008). google scholar
  • Hill K, ‘How Target Figured Out A Teen Girl Was Pregnant Before Her Father Did’ Forbes (16 February 2012) <https://www.forbes.com/sites/kashmirhill/2012/02/16/how-target-figured-out-a-teen-girl-was-pregnant-before-her-father-did/?sh=415c8bfe6668> accessed 20 June 2021. google scholar
  • Humphreys P, ‘The philosophical novelty of computer simulation methods’ (2008) 169 Synthese, pp. 615-626 <https://doi.org/10.1007/s11229-008-9435-2> accessed 20 June 2021. google scholar
  • International Association of Insurance Supervisors, ‘Issues Paper on the Use of Big Data Analytics in Insurance’ (2020) <https://www.iaisweb.org/page/supervisory-material/issues-papers// file/89244/issues-paper-on-use-of-big-data-analytics-in-insurance> accessed 20 June 2021. google scholar
  • International Covenant on Civil and Political Rights adopted and opened for signature, ratification and accession by General Assembly resolution 2200A (XXI) of 16 December 1966 entry into force 23 March 1976. google scholar
  • Izzidien R, ‘Higher insurance if you’re called Mohammed? That’s just the start of institutionalized Islamophobia’ NewStatesman (23 January 2018) <https://www.newstatesman.com/politics/ uk/2018/01/higher-insurance-if-you-re-called-mohammed-s-just-start-institutionalised> accessed 20 June 2021. google scholar
  • Johnson K, Pasquale F and Chapman J, ‘Artificial Intelligence, Machine Learning, and Bias in Finance: Toward Responsible Innovation’ (2019) 88 2 Fordham Law Review 499-529. google scholar
  • Jones H, ‘When the wrong postcode puts insurance out of your reach’ The Guardian (30 October 2016) <https://www.theguardian.com/money/2016/oct/30/wrong-postcode-puts-insurance-out-of-reach> accessed 20 June 2021. google scholar
  • Kelion L, ‘Apple’s ‘sexist’ credit card investigated by US regulator’ BBC (11 November 2019) <https://www.bbc.co.uk/news/business-50365609> accessed 20 June 2021. google scholar
  • Kim P T, ‘Data-Driven Discrimination at Work’ (2017) 58 3 William & Mary Law Review, pp. 857-936. google scholar
  • Kleinberg J et al., ‘Discrimination in the Age of Algorithms’ (2018) 10 Journal of Legal Analysis pp. 113-174. google scholar
  • Kloza D et al., ‘Data protection impact assessments in the European Union: complementing the new legal framework towards a more robust protection of individuals’ (d.pia.lab Policy Brief) (2017) <https://cris.vub.be/files/32009890/dpialab_pb2017_1_final.pdf> accessed 20 June 2021. google scholar
  • Knutson T, ‘AI Lending Discrimination Needs To Be Tackled With Legislation Says House Financial Services Chair’ Forbes (7 May 2021) <https://www.forbes.com/sites/tedknutson/2021/05/07/ ai-lending-discrimination-needs-to-be-tackled-with-legislation-says-house-financial-services-chair/?sh=7d233f205c7d> accessed 20 June 2021. google scholar
  • Kretchmer H, ‘Insurers ‘risk breaking racism laws’’ BBC (9 February 2018) <https://www.bbc. com/news/business-43011882> accessed 20 June 2021. google scholar
  • Kroll J A et al., ‘Accountable Algorithms’ (2017) 165 University of Pennsylvania Law Review, 633-705. google scholar
  • Lacroix C, ‘Preventing discrimination caused by the use of artificial intelligence’ (Report of Committee on Equality and Non-Discrimination at Council of Europe) (2020). google scholar
  • Lambrecht A and Tucker C E, ‘Algorithmic Bias? An Empirical Study of Apparent Gender-Based Discrimination in the Display of STEM Career Ads’ (2019) 65 7 Management Science, 2966-2981. google scholar
  • Liboreiro J, ‘The higher the risk, the stricter the rule’: Brussels’ new draft rules on artificial intelligence’ Euronews (21.4.2021) <https://www.euronews.com/2021/04/21/the-higher-the-risk-the-stricter-the-rule-brussels-new-draft-rules-on-artificial-intellige> accessed 20 June 2021. google scholar
  • Loi M and Christen M, ‘Choosing how to discriminate: navigating ethical trade-offs in fair algorithmic design for the insurance sector’ (2021) Philosophy & Technology <https://doi. org/10.1007/s13347-021-00444-9> accessed 20 June 2021. google scholar
  • Lucci S and Kopec D, Artificial Intelligence in the 21st Century: A Living Introduction (Mercury Learning and Information, USA 2016). google scholar
  • MacCarthy M and Propp K, ‘Machines Learn That Brussels Writes the Rules: The EU’s New AI Regulation’ Lawfare Blog (28 April 2021) <https://www.lawfareblog.com/machines-learn-brussels-writes-rules-eus-new-ai-regulation> accessed 20 June 2021. google scholar
  • Makinen J, ‘The EU’s AI Regulation Proposal - How to Fix It?’ (2021) <https://teknologiateollisuus. fi/en/ajankohtaista/eus-ai-regulation-proposal-how-fix-it> accessed 20 June 2021. google scholar
  • Meneceur Y, ‘European Commission’s AI regulation proposal: between too much and too little?’ LinkedIn (23 April 2021) <https://www.linkedin.com/pulse/european-commissions-ai-regulation-proposal-between-too-meneceur/> accessed 20 June 2021. google scholar
  • Modernised Convention for the Protection of Individuals with Regard to the Processing of Personal Data CM/Inf(2018)15-final [2018]. google scholar
  • Möslein F, ‘Robots in the Boardroom: Artificial Intelligence and Corporate Law’ in Barfield W and Pagallo U (eds.), Research Handbook on the Law of Artificial Intelligence (Edward Elgar, UK 2018) 649-669. google scholar
  • Newell S and Marabelli M, ‘Strategic opportunities (and challenges) of algorithmic decision-making: A call for action on the long-term societal effects of ‘datification’’ (2015) 24 Journal of Strategic Information Systems, 3-15. google scholar
  • Peck D, ‘They’re Watching You at Work’ Atlantic (December 2013) <http://www.theatlantic.com/ magazine/archive/2013/12/theyre-watching-you-at-work/354681> accessed 20 June 2021. google scholar
  • Peppet S R, ‘Regulating the Internet of Things: first steps toward managing discrimination, privacy, security, and consent’ (2014) 93 Texas Law Review, 85-176. google scholar
  • Pettinger-Kearney N et al., ‘The EU’s proposed AI Regulation’ (2021) <https://www.freshfields. com/en-gb/our-thinking/campaigns/digital/artificial-intelligence/the-eus-proposed-ai-regulation/> accessed 20 June 2021. google scholar
  • Pope D G and Sydnor J R, ‘Implementing Anti-Discrimination Policies in Statistical Profiling Models’ (2011) 3 American Economic Journal: Economic Policy, 206-231. google scholar
  • Prince A E R and Schwarcz D, ‘Proxy Discrimination in the Age of Artificial Intelligence and Big Data’ (2020) 105 Iowa Law Review, 1257-1318. google scholar
  • Probasco J, ‘The Insurance Industry Confronts Its Own Racism’ Investopedia (1 September 2020) <https://www.investopedia.com/race-and-insurance-5075141#citation-9> accessed 20 June 2021. google scholar
  • Reinhold F and Müller A, ‘Algorithm Watch’s response to the European Commission’s proposed regulation on Artificial Intelligence - a major step with major gaps’ (Report) (22 April 2021) <https:// algorithmwatch.org/en/response-to-eu-ai-regulation-proposal-2021/> accessed 20 June 2021. google scholar
  • Rieke A, Bogen M and Robinson D G, ‘Public scrutiny of automated decisions: Early lessons and emerging methods’ (Upturn and Omidyar Network Report) (2018) <https://www.data.govt.nz/ assets/Uploads/Public-Scrutiny-of-Automated-Decisions.pdf> accessed 20 June 2021. google scholar
  • Rodrigues R, ‘Legal and human rights issues of AI: Gaps, challenges and vulnerabilities’ (2020) 4 Journal of Responsible Technology <https://doi.org/10.1016/j.jrt.2020.100005> accessed 20 June 2021. google scholar
  • Samoili S et al., ‘AI Watch. Defining Artificial Intelligence. Towards An Operational Definition And Taxonomy Of Artificial Intelligence’ (Joint Research Centre Technical Reports) (Luxembourg 2020). google scholar
  • Schauer F, ‘Statistical (and non-statistical) discrimination’ in Kasper Lippert-Rasmussen (ed.), The Routledge Handbook of the Ethics of Discrimination, (Routledge, UK 2018), 42-53. google scholar
  • Swedloff R, ‘Risk classification’s Big Data (r)evolution’ (2014) 21 Connecticut Insurance Law Journal, 339-373. google scholar
  • The European Consumer Organization, ‘Automated Decision Making And Artificial Intelligence - A Consumer Perspective’ (Report) (2018) <https://www.beuc.eu/publications/beuc-x-2018-058_ automated_decision_making_and_artificial_intelligence.pdf> accessed 20 June 2021. google scholar
  • The European Consumer Organization, ‘AI Rights for Consumers’ (Report) (2019) <https://www. beuc.eu/publications/beuc-x-2019-063_ai_rights_for_consumers.pdf> accessed 20 June 2021. google scholar
  • The European Consumer Organization, ‘The Use Of Big Data And Artificial Intelligence In Insurance’ (Report) (2020) <https://www.beuc.eu/publications/beuc-x-2020-039_beuc_ position_paper_big_data_and_ai_in_insurances.pdf> accessed 20 June 2021. google scholar
  • The European Consumer Organization, ‘EU proposal for artificial intelligence law is weak on consumer protection’ (Report) (2021) <https://www.beuc.eu/publications/eu-proposal-artificial-intelligence-law-weak-consumer-protection/html> accessed 20 June 2021. google scholar
  • The Federal Government of Germany, ‘Artificial Intelligence Strategy’ (2018) <https://www.ki-strategie-deutschland.de/home.html?file=files/downloads/Nationale_KI-Strategie_engl.pdf> accessed 20 June 2021. google scholar
  • The Montreal Declaration for Responsible AI (2017) <https://www.montrealdeclaration-responsibleai.com/the-declaration> accessed 20 June 2021. google scholar
  • The Partnership on AI to Benefit People and Society <https://www.partnershiponai.org/about/> accessed 20 June 2021. google scholar
  • The Royal Society, ‘Machine Learning: The Power And Promise Of Computers That Learn By Example’ (Report) (April 2017) <royalsociety.org/machine-learning> accessed 20 June 2021. google scholar
  • The Royal Society, ‘Explainable AI: the basics’ (Policy briefing) (2019) <royalsociety.org/ai-interpretability> accessed 20 June 2021. google scholar
  • The White House, ‘Big Data: Seizing Opportunities, Preserving Values’ (Report) (2014). google scholar
  • Universal Declaration of Human Rights, UNGA Res 217/A (10 December 1948). google scholar
  • Valentino-Devries J, Singer-Vine J, and Soltani A, ‘Websites vary prices, deals based on users’ information’ Wall Street Journal (24 December 2012) <https://www.wsj.com/articles/SB10001 424127887323777204578189391813881534> accessed June 2021. google scholar
  • Vigdor N, ‘Apple Card Investigated After Gender Discrimination Complaints’ The New York Times (10 November 2019) <https://www.nytimes.com/2019/11/10/business/Apple-credit-card-investigation.html> accessed June May 2021. google scholar
  • Wachter S, Mittelstadt B and Russell C, ‘Counterfactual explanations without opening the black box: automated decisions and the GDPR’ (2018) 31 2 Harvard Journal of Law & Technology, 842-887. google scholar
  • Wachter S, Mittelstadt B and Russell C, ‘Why Fairness Cannot Be Automated: Bridging The Gap Between EU Non-Discrimination Law And AI’ (Draft) (2020) <https://papers.ssrn.com/sol3/ papers.cfm?abstract_id=3547922> accessed 20 June 2021. google scholar
  • Wagner B et al., ‘Algorithms and human rights. Study on the human rights dimensions of automated data processing techniques and possible regulatory implications’ (Study Report) (DGI(2017)12) (2018) <https://rm.coe.int/algorithms-and-human-rights-en-rev/16807956b5> accessed 20 June 2021. google scholar
  • Wahlster W and Winterhalter C, ‘German Standardization Roadmap On Artificial Intelligence’ (German Institute for Standardization and DKE German Commission for Electrical, Electronic & Information Technologies of DIN and VDE November 2020) <https://www.din.de/resource/ blob/772610/e96c34dd6b12900ea75b460538805349/normungsroadmap-en-data.pdf> accessed 20 June 2021. google scholar
  • Walch K, ‘Is There A Difference Between Assisted Intelligence Vs. Augmented Intelligence?’ Forbes. (12 January 2020) <https://www.forbes.com/sites/cognitiveworld/2020/01/12/is-there-a-difference-between-assisted-intelligence-vs-augmented-intelligence/?sh=3300950426ab> accessed 20 June 2021. google scholar
  • Wang P, ‘On the Working Definition of Intelligence’ (1995) <http://citeseerx.ist.psu.edu/viewdoc/ summary?doi=10.1.1.55.5012> accessed 20 May 2021. google scholar
  • Weisbaum H, ‘Data Mining Is Now Used to Set Insurance Rates; Critics Cry Foul’ CNBC (16 April 2014) <https://www.cnbc.com/2014/04/16/data-mining-is-now-used-to-set-insurance-rates-critics-cry-fowl.html#:~:text=You%20might%20not%20like%20the,to%20provide%20 you%20that%20coverage> accessed 20 June 2021. google scholar
  • Whitehead D, ‘AI & Algorithms (Part 3): Why the EU’s AI regulation is a ground breaking proposal’ Hogan Lovells Engage (3 May 2021) <https://www.engage.hoganlovells.com/ knowledgeservices/viewContent. google scholar action?key=Ec8teaJ9VaqLasZjnzVtGsxgHJMKLFEppVpbbVX%2B3OXcP3PYxlq7sZUjdb-Sm5FIetvAtgf1eVU8%3D&nav=FRbANEucS95NMLRN47z%2BeeOgEFCt8EGQ0qF-foEM4UR4%3D&emailtofriendview=true&freeviewlink=true> accessed 20 June 2021. google scholar
  • Williams B et al., ‘How Algorithms Discriminate Based on Data they Lack: Challenges, Solutions, and Policy Implications’ (2018) 8 Journal of Information Policy, 78-115. google scholar
  • World Economic Forum, ‘Transforming Paradigms A Global AI in Financial Services Survey’ (2020) (Report) <http://www3.weforum.org/docs/WEF_AI_in_Financial_Services_Survey. pdf> accessed 20 June 2021. google scholar
  • Xenidis R, ‘Tuning EU equality law to algorithmic discrimination: Three pathways to resilience’ (2020) 27(6) Maastricht Journal of European and Comparative Law, 736-758. google scholar
  • Xenidis R and Senden L, ‘EU non-discrimination law in the era of artificial intelligence: Mapping the challenges of algorithmic discrimination’ in Ulf Bernitz et al. (eds.), General principles of EU law and the EU digital order (Kluwer Law International 2020), 151-182. google scholar
Year 2022, Issue: 71, 265 - 307, 31.12.2022
https://doi.org/10.26650/annales.2022.71.0002

Abstract

References

  • Accesswire, ‘72% of Resumes are Never Seen by Employers’ (16 February 2016) <https://www. accesswire.com/436847/72-of-Resumes-are-Never-Seen-by-Employers> accessed 20 June 2021. google scholar
  • Ali M et al., ‘Discrimination Through Optimization: How Facebook’s ad Delivery Can Lead to Skewed Outcomes’ (2019) Proceedings of the ACM on Human-Computer Interaction. <https:// arxiv.org/pdf/1904.02095.pdf> accessed 20 June 2021. google scholar
  • Angelopoulos C et al., ‘Study Of Fundamental Rights Limitations For Online Enforcement Through Self-Regulation’ (Study Report of Institute for Information Law, University of Amsterdam) (2016) <https://scholarlypublications.universiteitleiden.nl/access/item%3A2869513/view> accessed 20 June 2021. google scholar
  • Angwin J, Mattu S and Larson J, ‘The tiger mom tax: Asians are nearly twice as likely to get a higher price from Princeton Review’. ProPublica (1 September 2015) <https://www.ProPublica.org/article/ asians-nearly-twice-as-likely-to-get-higher-price-from-princeton-review> accessed 20 June 2021. google scholar
  • Angwin J, Scheiber N and Tobin A, ‘Dozens of companies are using Facebook to exclude older workers from job ads’ ProPublica (20 December 2017) <https://www.propublica.org/article/ facebook-ads-age-discrimination-targeting> accessed 20 June 2021. google scholar
  • Angwin J, Tobin A and Varner M, ‘Facebook (still) letting housing advertisers exclude users by race’ ProPublica (21 November 2017) <https://www.propublica.org/article/facebook-advertising-discrimination-housing-race-sex-national-origin> accessed 20 June 2021. google scholar
  • Avraham R, ‘Discrimination and Insurance’ in Lippert-Rasmussen K. (ed.), The Routledge Handbook of the Ethics of Discrimination, (Routledge, UK 2018), pp. 335-347. google scholar
  • Bartlett R et al., ‘Consumer-Lending Discrimination in the FinTech Era’ (2019) (National Bureau of Economic Research Working Papers 25943). google scholar
  • Barocas S and Selbst A D, ‘Big Data’s Disparate Impact’ (2016) 104 California Law Review, 671-732. google scholar
  • Bathaee Y, ‘The Artificial Intelligence Black Box And The Failure Of Intent And Causation’ (2018) 31:2 Harvard Journal of Law & Technology, 890-938. google scholar
  • Beck S et al., ‘Künstliche Intelligenz und Diskriminierung Herausforderungen und Lösungsansatze’ (2019) (Whitepaper from Plattform Lernende Systeme) <https://www.plattform-lernende-systeme.de/publikationen-details/kuenstliche-intelligenz-und-diskriminierung-herausforderungen-und-loesungsansaetze.html> accessed 20 June 2021. google scholar
  • Biao v. Denmark (Grand Chamber) (App no 38590/10) ECHR 24 May 2016. google scholar
  • Bodo B et al., ‘Tackling the algorithmic control crisis-the technical, legal, and ethical challenges of research into algorithmic agents’ (2017) 19 Yale Journal of Law & Technology, 133-180. google scholar
  • Borgesius F Z, ‘Discrimination, Artificial Intelligence and Algorithmic Decision-making’ (Study Report) (Council of Europe, Strasbourg 2018) <https://rm.coe.int/discrimination-artificial-intelligence-and-algorithmic-decision-making/1680925d73> accessed 20 June 2021. google scholar
  • Borgesius F J Z, ‘Strengthening legal protection against discrimination by algorithms and artificial intelligence’ (2020) 24 10 The International Journal of Human Rights, 1572- 1593. google scholar
  • Bundesanstalt für Finanzdienstleistungsaufsicht, ‘Big data meets artificial intelligence: Challenges and implications for the supervision and regulation of financial services’ (2018) (Study) www.bafin.de/SharedDocs/Downloads/EN/dl_bdai_studie_en.html> accessed 20 June 2021. google scholar
  • Burrell J, ‘How the machine ‘thinks’: understanding opacity in machine learning algorithms’ (2016) Big Data & Society Doi:10.1177/2053951715622512. google scholar
  • Case 170/84, Bilka-Kaufhaus GmbH and Karin Weber von Hartz, (Judgment of the Court of 13 May 1986.) google scholar
  • Case C-83/14, “CHEZ Razpredelenie Bulgaria” AD v Komisia za zashtita ot diskriminatsia, (Judgment of the Court (Grand Chamber) of 16 July 2015). google scholar
  • Case C443/15, David L. Parris v Trinity College Dublin and Others (Judgment of the Court (First Chamber) of 24 November 2016). google scholar
  • Case C-414/16, Vera Egenberger v Evangelisches Werk für Diakonie und Entwicklung e.V. (Judgment of the Court (Grand Chamber) of 17 April 2018). google scholar
  • Chander S and Jakubowska E, ‘EU’s AI law needs major changes to prevent discrimination and mass surveillance’ EDRI (2021) <https://edri.org/our-work/eus-ai-law-needs-major-changes-to-prevent-discrimination-and-mass-surveillance/> accessed 20 June 2021. google scholar
  • Charter of Fundamental Rights of the European Union, 2012/C 326/02 OJ C 326, [2012]. google scholar
  • Commission (EC) ‘Proposal for a Regulation Of The European Parliament And Of The Council Laying Down Harmonised Rules On Artificial Intelligence (Artificial Intelligence Act) And Amending Certain Union Legislative Acts’ COM(2021) 206 final, 21 April 2021. google scholar
  • Corner N, ‘Revealed: How your name, your ethnicity, and even your e-mail address could be adding hundreds of pounds to your car insurance premium’ Dailymail (29 June 2018) www.dailymail.co.uk/femail/article-5896561/How-email-address-add-hundreds-car-insurance-premium.html> accessed 20 June 2021. google scholar
  • Council Directive (EC) 2000/43 of 29 June 2000 implementing the principle of equal treatment between persons irrespective of racial or ethnic origin [2000] OJ L 180. google scholar
  • Council Directive (EC) 2000/78 of 27 November 2000 establishing a general framework for equal treatment in employment and occupation [2000] OJ L 303. google scholar
  • Council Directive (EC) 2001/95 of the European Parliament and of the Council of 3 December 2001 on general product safety [2001] OJ L 11. google scholar
  • Council of Europe, ‘Explanatory Report to the Protocol No. 12 to the Convention for the Protection of Human Rights and Fundamental Freedoms’ (2000) <https://rm.coe.int/09000016800cce48> accessed 20 June 2021. google scholar
  • Council of Europe, ‘Algorithms and Human Rights: Study On The Human Rights Dimensions Of Automated Data Processing Techniques (In Particular Algorithms) And Possible Regulatory Implications’ (Study Report) (DGI(2017)12) (2018). google scholar
  • Council Regulation (EC) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation) [2016] OJ L 119. google scholar
  • Craglia M et al., ‘Artificial Intelligence: A European Perspective’ (Joint Research Centre Working Papers) (2018). google scholar
  • Danks D and London A J, ‘Algorithmic Bias in Autonomous Systems’ (2017) IJCAI’17: Proceedings of the 26th International Joint Conference on Artificial Intelligence, 4691-4697. google scholar
  • Dastin J, ‘Amazon scraps secret AI recruiting tool that showed bias against women’ Reuters (11 October 2018) <https://www.reuters.com/article/us-amazon-com-jobs-automation-in...-airecruiting-tool-that-showed-bias-against-women-idUSKCN1MK08G> accessed 20 June 2021. google scholar
  • Data Protection Working Party Guidelines on Automated individual decision-making and Profiling for the purposes of Regulation 2016/679 Wp251rev.01. [2018]. google scholar
  • Datta A, Tschantz M C and Datta A, ‘Automated Experiments on Ad Privacy Settings A Tale of Opacity, Choice, and Discrimination’ (2015) Proceedings on Privacy Enhancing Technologies <https://www.andrew.cmu.edu/user/danupam/dtd-pets15.pdf> accessed 20 June 2021. google scholar
  • Delipetrev B, Tsinarakii C and Kostic U, ‘Historical Evolution of Artificial Intelligence’ (Technical Report) (Publications Office of the European Union, Luxembourg 2020) https://publications.jrc. ec.europa.eu/repository/handle/JRC120469 accessed 20 June 2021. google scholar
  • Dwork C et al., ‘Fairness Through Awareness, Proceedings of the 3rd Innovations in Theoretical Computer Science Conference’ (2012) ITCS ‘12 (Association for Computing Machinery), 214226. google scholar
  • Edwards, L and Veale M, ‘Slave to the algorithm: Why a right to an explanation is probably not the remedy you are looking for’ (2017) 16 Duke Law & Technology Review, 18-84. google scholar
  • Espinoza J and Murgia M, ‘Europe attempts to take leading role in regulating uses of AI’ Financial Times (24 April 2021) <https://www.ft.com/content/360faa3e-4110-4f38-b618-dd695deece90> accessed 20 June 2021. google scholar
  • European Commission, Proposal for a Council Directive on implementing the principle of equal treatment between persons irrespective of religion or belief, disability, age or sexual orientation [2008] {SEC(2008) 2180} {SEC(2008) 2181} /* COM/2008/0426 final - CNS 2008/0140 */. google scholar
  • European Commission, ‘Special Eurobarometer Discrimination in the EU in 2015’ (Report) (2015). google scholar
  • European Commission’s High-Level Expert Group on Artificial Intelligence, ‘A Definition of AI: Main Capabilities and Disciplines’ (2019) (Report) <https://digital-strategy.ec.europa.eu/en/ library/definition-artificial-intelligence-main-capabilities-and-scientific-disciplines> accessed 20 June 2021. google scholar
  • European Commission, ‘A Union of Equality: Gender Equality Strategy 2020-2025’ [2020] COM (2020) 152. google scholar
  • European Commission, ‘Internal Market, Industry, Entrepreneurship and SMEs’ <https://ec.europa. eu/growth/single-market/ce-marking/manufacturers_en> accessed 20 June 2021. google scholar
  • European Commission, ‘Harmonised Standards’ <https://ec.europa.eu/growth/single-market/ european-standards/harmonised-standards_en> accessed 20 June 2021. google scholar
  • European Convention for the Protection of Human Rights and Fundamental Freedoms (1950) <https://www.echr.coe.int/Documents/Convention_ENG.pdf> accessed 20 June 2021. google scholar
  • European Union Agency for Fundamental Rights, ‘#BigData: Discrimination in data-supported decision making’ (Report) (2018) Doi:10.2811/343905. google scholar
  • European Union Agency for Fundamental Rights, ‘Getting the Future Right-Artificial Intelligence and Fundamental Rights’ (2020) Doi:10.2811/774118. google scholar
  • Fairness, Accountability, and Transparency in Machine Learning <https://www.fatml.org/> accessed 20 June 2021. google scholar
  • Favaretto M, De Clercq E and Elger B S, ‘Big Data and discrimination: perils, promises and solutions. A systematic review’ (2019) 6 Journal of Big Data <https://doi.org/10.1186/s40537-019-0177-4> accessed 20 June 2021. google scholar
  • Financial Conduct Authority, ‘Pricing practices in the retail general insurance sector: Household insurance’ (2018) <https://www.fca.org.uk/publication/thematic-reviews/tr18-4.pdf> accessed 20 June 2021. google scholar
  • Stan F, ‘History, Motivations, and Core Themes’ in Keith Frankish and William W. Ramsey, (eds.), The Cambridge Handbook Of Artificial Intelligence (Cambridge University Press, UK 2014). google scholar
  • Fuster A et al., ‘Predictably Unequal? The Effects of Machine Learning on Credit Markets’ (2020). SSRN <https://ssrn.com/abstract=3072038> accessed 20 June 2021. google scholar
  • Future of Life Institute, ‘The Asilomar AI principles’ (2017) <https://futureoflife.org/ai-principles/> accessed 20 June 2021. google scholar
  • Gillis T B and Spiess J L, ‘Big Data and Discrimination’ (2019) 86 The University of Chicago Law Review, pp. 459-487. google scholar
  • Hacker P., ‘Teaching Fairness to Artificial Intelligence: Existing and Novel Strategies against Algorithmic Discrimination under EU Law’ (Pre-print version) (2019) <https://papers.ssrn.com/ sol3/papers.cfm?abstract_id=3164973&download=yes> accessed 20 June 2021. google scholar
  • Heinrichs B, ‘Discrimination in the age of artificial intelligence’ (2021) AI & Society Springer. <https://doi.org/10.1007/s00146-021-01192-2> accessed 20 June 2021. google scholar
  • Hildebrandt M and Gutwirth S, ‘Concise Conclusions: Citizens Out of Control’ in Hildebrandt M and Gutwirth S (eds.) Profiling the European Citizen: Cross-Disciplinary Perspectives (Springer, 2008). google scholar
  • Hill K, ‘How Target Figured Out A Teen Girl Was Pregnant Before Her Father Did’ Forbes (16 February 2012) <https://www.forbes.com/sites/kashmirhill/2012/02/16/how-target-figured-out-a-teen-girl-was-pregnant-before-her-father-did/?sh=415c8bfe6668> accessed 20 June 2021. google scholar
  • Humphreys P, ‘The philosophical novelty of computer simulation methods’ (2008) 169 Synthese, pp. 615-626 <https://doi.org/10.1007/s11229-008-9435-2> accessed 20 June 2021. google scholar
  • International Association of Insurance Supervisors, ‘Issues Paper on the Use of Big Data Analytics in Insurance’ (2020) <https://www.iaisweb.org/page/supervisory-material/issues-papers// file/89244/issues-paper-on-use-of-big-data-analytics-in-insurance> accessed 20 June 2021. google scholar
  • International Covenant on Civil and Political Rights adopted and opened for signature, ratification and accession by General Assembly resolution 2200A (XXI) of 16 December 1966 entry into force 23 March 1976. google scholar
  • Izzidien R, ‘Higher insurance if you’re called Mohammed? That’s just the start of institutionalized Islamophobia’ NewStatesman (23 January 2018) <https://www.newstatesman.com/politics/ uk/2018/01/higher-insurance-if-you-re-called-mohammed-s-just-start-institutionalised> accessed 20 June 2021. google scholar
  • Johnson K, Pasquale F and Chapman J, ‘Artificial Intelligence, Machine Learning, and Bias in Finance: Toward Responsible Innovation’ (2019) 88 2 Fordham Law Review 499-529. google scholar
  • Jones H, ‘When the wrong postcode puts insurance out of your reach’ The Guardian (30 October 2016) <https://www.theguardian.com/money/2016/oct/30/wrong-postcode-puts-insurance-out-of-reach> accessed 20 June 2021. google scholar
  • Kelion L, ‘Apple’s ‘sexist’ credit card investigated by US regulator’ BBC (11 November 2019) <https://www.bbc.co.uk/news/business-50365609> accessed 20 June 2021. google scholar
  • Kim P T, ‘Data-Driven Discrimination at Work’ (2017) 58 3 William & Mary Law Review, pp. 857-936. google scholar
  • Kleinberg J et al., ‘Discrimination in the Age of Algorithms’ (2018) 10 Journal of Legal Analysis pp. 113-174. google scholar
  • Kloza D et al., ‘Data protection impact assessments in the European Union: complementing the new legal framework towards a more robust protection of individuals’ (d.pia.lab Policy Brief) (2017) <https://cris.vub.be/files/32009890/dpialab_pb2017_1_final.pdf> accessed 20 June 2021. google scholar
  • Knutson T, ‘AI Lending Discrimination Needs To Be Tackled With Legislation Says House Financial Services Chair’ Forbes (7 May 2021) <https://www.forbes.com/sites/tedknutson/2021/05/07/ ai-lending-discrimination-needs-to-be-tackled-with-legislation-says-house-financial-services-chair/?sh=7d233f205c7d> accessed 20 June 2021. google scholar
  • Kretchmer H, ‘Insurers ‘risk breaking racism laws’’ BBC (9 February 2018) <https://www.bbc. com/news/business-43011882> accessed 20 June 2021. google scholar
  • Kroll J A et al., ‘Accountable Algorithms’ (2017) 165 University of Pennsylvania Law Review, 633-705. google scholar
  • Lacroix C, ‘Preventing discrimination caused by the use of artificial intelligence’ (Report of Committee on Equality and Non-Discrimination at Council of Europe) (2020). google scholar
  • Lambrecht A and Tucker C E, ‘Algorithmic Bias? An Empirical Study of Apparent Gender-Based Discrimination in the Display of STEM Career Ads’ (2019) 65 7 Management Science, 2966-2981. google scholar
  • Liboreiro J, ‘The higher the risk, the stricter the rule’: Brussels’ new draft rules on artificial intelligence’ Euronews (21.4.2021) <https://www.euronews.com/2021/04/21/the-higher-the-risk-the-stricter-the-rule-brussels-new-draft-rules-on-artificial-intellige> accessed 20 June 2021. google scholar
  • Loi M and Christen M, ‘Choosing how to discriminate: navigating ethical trade-offs in fair algorithmic design for the insurance sector’ (2021) Philosophy & Technology <https://doi. org/10.1007/s13347-021-00444-9> accessed 20 June 2021. google scholar
  • Lucci S and Kopec D, Artificial Intelligence in the 21st Century: A Living Introduction (Mercury Learning and Information, USA 2016). google scholar
  • MacCarthy M and Propp K, ‘Machines Learn That Brussels Writes the Rules: The EU’s New AI Regulation’ Lawfare Blog (28 April 2021) <https://www.lawfareblog.com/machines-learn-brussels-writes-rules-eus-new-ai-regulation> accessed 20 June 2021. google scholar
  • Makinen J, ‘The EU’s AI Regulation Proposal - How to Fix It?’ (2021) <https://teknologiateollisuus. fi/en/ajankohtaista/eus-ai-regulation-proposal-how-fix-it> accessed 20 June 2021. google scholar
  • Meneceur Y, ‘European Commission’s AI regulation proposal: between too much and too little?’ LinkedIn (23 April 2021) <https://www.linkedin.com/pulse/european-commissions-ai-regulation-proposal-between-too-meneceur/> accessed 20 June 2021. google scholar
  • Modernised Convention for the Protection of Individuals with Regard to the Processing of Personal Data CM/Inf(2018)15-final [2018]. google scholar
  • Möslein F, ‘Robots in the Boardroom: Artificial Intelligence and Corporate Law’ in Barfield W and Pagallo U (eds.), Research Handbook on the Law of Artificial Intelligence (Edward Elgar, UK 2018) 649-669. google scholar
  • Newell S and Marabelli M, ‘Strategic opportunities (and challenges) of algorithmic decision-making: A call for action on the long-term societal effects of ‘datification’’ (2015) 24 Journal of Strategic Information Systems, 3-15. google scholar
  • Peck D, ‘They’re Watching You at Work’ Atlantic (December 2013) <http://www.theatlantic.com/ magazine/archive/2013/12/theyre-watching-you-at-work/354681> accessed 20 June 2021. google scholar
  • Peppet S R, ‘Regulating the Internet of Things: first steps toward managing discrimination, privacy, security, and consent’ (2014) 93 Texas Law Review, 85-176. google scholar
  • Pettinger-Kearney N et al., ‘The EU’s proposed AI Regulation’ (2021) <https://www.freshfields. com/en-gb/our-thinking/campaigns/digital/artificial-intelligence/the-eus-proposed-ai-regulation/> accessed 20 June 2021. google scholar
  • Pope D G and Sydnor J R, ‘Implementing Anti-Discrimination Policies in Statistical Profiling Models’ (2011) 3 American Economic Journal: Economic Policy, 206-231. google scholar
  • Prince A E R and Schwarcz D, ‘Proxy Discrimination in the Age of Artificial Intelligence and Big Data’ (2020) 105 Iowa Law Review, 1257-1318. google scholar
  • Probasco J, ‘The Insurance Industry Confronts Its Own Racism’ Investopedia (1 September 2020) <https://www.investopedia.com/race-and-insurance-5075141#citation-9> accessed 20 June 2021. google scholar
  • Reinhold F and Müller A, ‘Algorithm Watch’s response to the European Commission’s proposed regulation on Artificial Intelligence - a major step with major gaps’ (Report) (22 April 2021) <https:// algorithmwatch.org/en/response-to-eu-ai-regulation-proposal-2021/> accessed 20 June 2021. google scholar
  • Rieke A, Bogen M and Robinson D G, ‘Public scrutiny of automated decisions: Early lessons and emerging methods’ (Upturn and Omidyar Network Report) (2018) <https://www.data.govt.nz/ assets/Uploads/Public-Scrutiny-of-Automated-Decisions.pdf> accessed 20 June 2021. google scholar
  • Rodrigues R, ‘Legal and human rights issues of AI: Gaps, challenges and vulnerabilities’ (2020) 4 Journal of Responsible Technology <https://doi.org/10.1016/j.jrt.2020.100005> accessed 20 June 2021. google scholar
  • Samoili S et al., ‘AI Watch. Defining Artificial Intelligence. Towards An Operational Definition And Taxonomy Of Artificial Intelligence’ (Joint Research Centre Technical Reports) (Luxembourg 2020). google scholar
  • Schauer F, ‘Statistical (and non-statistical) discrimination’ in Kasper Lippert-Rasmussen (ed.), The Routledge Handbook of the Ethics of Discrimination, (Routledge, UK 2018), 42-53. google scholar
  • Swedloff R, ‘Risk classification’s Big Data (r)evolution’ (2014) 21 Connecticut Insurance Law Journal, 339-373. google scholar
  • The European Consumer Organization, ‘Automated Decision Making And Artificial Intelligence - A Consumer Perspective’ (Report) (2018) <https://www.beuc.eu/publications/beuc-x-2018-058_ automated_decision_making_and_artificial_intelligence.pdf> accessed 20 June 2021. google scholar
  • The European Consumer Organization, ‘AI Rights for Consumers’ (Report) (2019) <https://www. beuc.eu/publications/beuc-x-2019-063_ai_rights_for_consumers.pdf> accessed 20 June 2021. google scholar
  • The European Consumer Organization, ‘The Use Of Big Data And Artificial Intelligence In Insurance’ (Report) (2020) <https://www.beuc.eu/publications/beuc-x-2020-039_beuc_ position_paper_big_data_and_ai_in_insurances.pdf> accessed 20 June 2021. google scholar
  • The European Consumer Organization, ‘EU proposal for artificial intelligence law is weak on consumer protection’ (Report) (2021) <https://www.beuc.eu/publications/eu-proposal-artificial-intelligence-law-weak-consumer-protection/html> accessed 20 June 2021. google scholar
  • The Federal Government of Germany, ‘Artificial Intelligence Strategy’ (2018) <https://www.ki-strategie-deutschland.de/home.html?file=files/downloads/Nationale_KI-Strategie_engl.pdf> accessed 20 June 2021. google scholar
  • The Montreal Declaration for Responsible AI (2017) <https://www.montrealdeclaration-responsibleai.com/the-declaration> accessed 20 June 2021. google scholar
  • The Partnership on AI to Benefit People and Society <https://www.partnershiponai.org/about/> accessed 20 June 2021. google scholar
  • The Royal Society, ‘Machine Learning: The Power And Promise Of Computers That Learn By Example’ (Report) (April 2017) <royalsociety.org/machine-learning> accessed 20 June 2021. google scholar
  • The Royal Society, ‘Explainable AI: the basics’ (Policy briefing) (2019) <royalsociety.org/ai-interpretability> accessed 20 June 2021. google scholar
  • The White House, ‘Big Data: Seizing Opportunities, Preserving Values’ (Report) (2014). google scholar
  • Universal Declaration of Human Rights, UNGA Res 217/A (10 December 1948). google scholar
  • Valentino-Devries J, Singer-Vine J, and Soltani A, ‘Websites vary prices, deals based on users’ information’ Wall Street Journal (24 December 2012) <https://www.wsj.com/articles/SB10001 424127887323777204578189391813881534> accessed June 2021. google scholar
  • Vigdor N, ‘Apple Card Investigated After Gender Discrimination Complaints’ The New York Times (10 November 2019) <https://www.nytimes.com/2019/11/10/business/Apple-credit-card-investigation.html> accessed June May 2021. google scholar
  • Wachter S, Mittelstadt B and Russell C, ‘Counterfactual explanations without opening the black box: automated decisions and the GDPR’ (2018) 31 2 Harvard Journal of Law & Technology, 842-887. google scholar
  • Wachter S, Mittelstadt B and Russell C, ‘Why Fairness Cannot Be Automated: Bridging The Gap Between EU Non-Discrimination Law And AI’ (Draft) (2020) <https://papers.ssrn.com/sol3/ papers.cfm?abstract_id=3547922> accessed 20 June 2021. google scholar
  • Wagner B et al., ‘Algorithms and human rights. Study on the human rights dimensions of automated data processing techniques and possible regulatory implications’ (Study Report) (DGI(2017)12) (2018) <https://rm.coe.int/algorithms-and-human-rights-en-rev/16807956b5> accessed 20 June 2021. google scholar
  • Wahlster W and Winterhalter C, ‘German Standardization Roadmap On Artificial Intelligence’ (German Institute for Standardization and DKE German Commission for Electrical, Electronic & Information Technologies of DIN and VDE November 2020) <https://www.din.de/resource/ blob/772610/e96c34dd6b12900ea75b460538805349/normungsroadmap-en-data.pdf> accessed 20 June 2021. google scholar
  • Walch K, ‘Is There A Difference Between Assisted Intelligence Vs. Augmented Intelligence?’ Forbes. (12 January 2020) <https://www.forbes.com/sites/cognitiveworld/2020/01/12/is-there-a-difference-between-assisted-intelligence-vs-augmented-intelligence/?sh=3300950426ab> accessed 20 June 2021. google scholar
  • Wang P, ‘On the Working Definition of Intelligence’ (1995) <http://citeseerx.ist.psu.edu/viewdoc/ summary?doi=10.1.1.55.5012> accessed 20 May 2021. google scholar
  • Weisbaum H, ‘Data Mining Is Now Used to Set Insurance Rates; Critics Cry Foul’ CNBC (16 April 2014) <https://www.cnbc.com/2014/04/16/data-mining-is-now-used-to-set-insurance-rates-critics-cry-fowl.html#:~:text=You%20might%20not%20like%20the,to%20provide%20 you%20that%20coverage> accessed 20 June 2021. google scholar
  • Whitehead D, ‘AI & Algorithms (Part 3): Why the EU’s AI regulation is a ground breaking proposal’ Hogan Lovells Engage (3 May 2021) <https://www.engage.hoganlovells.com/ knowledgeservices/viewContent. google scholar action?key=Ec8teaJ9VaqLasZjnzVtGsxgHJMKLFEppVpbbVX%2B3OXcP3PYxlq7sZUjdb-Sm5FIetvAtgf1eVU8%3D&nav=FRbANEucS95NMLRN47z%2BeeOgEFCt8EGQ0qF-foEM4UR4%3D&emailtofriendview=true&freeviewlink=true> accessed 20 June 2021. google scholar
  • Williams B et al., ‘How Algorithms Discriminate Based on Data they Lack: Challenges, Solutions, and Policy Implications’ (2018) 8 Journal of Information Policy, 78-115. google scholar
  • World Economic Forum, ‘Transforming Paradigms A Global AI in Financial Services Survey’ (2020) (Report) <http://www3.weforum.org/docs/WEF_AI_in_Financial_Services_Survey. pdf> accessed 20 June 2021. google scholar
  • Xenidis R, ‘Tuning EU equality law to algorithmic discrimination: Three pathways to resilience’ (2020) 27(6) Maastricht Journal of European and Comparative Law, 736-758. google scholar
  • Xenidis R and Senden L, ‘EU non-discrimination law in the era of artificial intelligence: Mapping the challenges of algorithmic discrimination’ in Ulf Bernitz et al. (eds.), General principles of EU law and the EU digital order (Kluwer Law International 2020), 151-182. google scholar
Year 2022, Issue: 71, 265 - 307, 31.12.2022
https://doi.org/10.26650/annales.2022.71.0002

Abstract

References

  • Accesswire, ‘72% of Resumes are Never Seen by Employers’ (16 February 2016) <https://www. accesswire.com/436847/72-of-Resumes-are-Never-Seen-by-Employers> accessed 20 June 2021. google scholar
  • Ali M et al., ‘Discrimination Through Optimization: How Facebook’s ad Delivery Can Lead to Skewed Outcomes’ (2019) Proceedings of the ACM on Human-Computer Interaction. <https:// arxiv.org/pdf/1904.02095.pdf> accessed 20 June 2021. google scholar
  • Angelopoulos C et al., ‘Study Of Fundamental Rights Limitations For Online Enforcement Through Self-Regulation’ (Study Report of Institute for Information Law, University of Amsterdam) (2016) <https://scholarlypublications.universiteitleiden.nl/access/item%3A2869513/view> accessed 20 June 2021. google scholar
  • Angwin J, Mattu S and Larson J, ‘The tiger mom tax: Asians are nearly twice as likely to get a higher price from Princeton Review’. ProPublica (1 September 2015) <https://www.ProPublica.org/article/ asians-nearly-twice-as-likely-to-get-higher-price-from-princeton-review> accessed 20 June 2021. google scholar
  • Angwin J, Scheiber N and Tobin A, ‘Dozens of companies are using Facebook to exclude older workers from job ads’ ProPublica (20 December 2017) <https://www.propublica.org/article/ facebook-ads-age-discrimination-targeting> accessed 20 June 2021. google scholar
  • Angwin J, Tobin A and Varner M, ‘Facebook (still) letting housing advertisers exclude users by race’ ProPublica (21 November 2017) <https://www.propublica.org/article/facebook-advertising-discrimination-housing-race-sex-national-origin> accessed 20 June 2021. google scholar
  • Avraham R, ‘Discrimination and Insurance’ in Lippert-Rasmussen K. (ed.), The Routledge Handbook of the Ethics of Discrimination, (Routledge, UK 2018), pp. 335-347. google scholar
  • Bartlett R et al., ‘Consumer-Lending Discrimination in the FinTech Era’ (2019) (National Bureau of Economic Research Working Papers 25943). google scholar
  • Barocas S and Selbst A D, ‘Big Data’s Disparate Impact’ (2016) 104 California Law Review, 671-732. google scholar
  • Bathaee Y, ‘The Artificial Intelligence Black Box And The Failure Of Intent And Causation’ (2018) 31:2 Harvard Journal of Law & Technology, 890-938. google scholar
  • Beck S et al., ‘Künstliche Intelligenz und Diskriminierung Herausforderungen und Lösungsansatze’ (2019) (Whitepaper from Plattform Lernende Systeme) <https://www.plattform-lernende-systeme.de/publikationen-details/kuenstliche-intelligenz-und-diskriminierung-herausforderungen-und-loesungsansaetze.html> accessed 20 June 2021. google scholar
  • Biao v. Denmark (Grand Chamber) (App no 38590/10) ECHR 24 May 2016. google scholar
  • Bodo B et al., ‘Tackling the algorithmic control crisis-the technical, legal, and ethical challenges of research into algorithmic agents’ (2017) 19 Yale Journal of Law & Technology, 133-180. google scholar
  • Borgesius F Z, ‘Discrimination, Artificial Intelligence and Algorithmic Decision-making’ (Study Report) (Council of Europe, Strasbourg 2018) <https://rm.coe.int/discrimination-artificial-intelligence-and-algorithmic-decision-making/1680925d73> accessed 20 June 2021. google scholar
  • Borgesius F J Z, ‘Strengthening legal protection against discrimination by algorithms and artificial intelligence’ (2020) 24 10 The International Journal of Human Rights, 1572- 1593. google scholar
  • Bundesanstalt für Finanzdienstleistungsaufsicht, ‘Big data meets artificial intelligence: Challenges and implications for the supervision and regulation of financial services’ (2018) (Study) www.bafin.de/SharedDocs/Downloads/EN/dl_bdai_studie_en.html> accessed 20 June 2021. google scholar
  • Burrell J, ‘How the machine ‘thinks’: understanding opacity in machine learning algorithms’ (2016) Big Data & Society Doi:10.1177/2053951715622512. google scholar
  • Case 170/84, Bilka-Kaufhaus GmbH and Karin Weber von Hartz, (Judgment of the Court of 13 May 1986.) google scholar
  • Case C-83/14, “CHEZ Razpredelenie Bulgaria” AD v Komisia za zashtita ot diskriminatsia, (Judgment of the Court (Grand Chamber) of 16 July 2015). google scholar
  • Case C443/15, David L. Parris v Trinity College Dublin and Others (Judgment of the Court (First Chamber) of 24 November 2016). google scholar
  • Case C-414/16, Vera Egenberger v Evangelisches Werk für Diakonie und Entwicklung e.V. (Judgment of the Court (Grand Chamber) of 17 April 2018). google scholar
  • Chander S and Jakubowska E, ‘EU’s AI law needs major changes to prevent discrimination and mass surveillance’ EDRI (2021) <https://edri.org/our-work/eus-ai-law-needs-major-changes-to-prevent-discrimination-and-mass-surveillance/> accessed 20 June 2021. google scholar
  • Charter of Fundamental Rights of the European Union, 2012/C 326/02 OJ C 326, [2012]. google scholar
  • Commission (EC) ‘Proposal for a Regulation Of The European Parliament And Of The Council Laying Down Harmonised Rules On Artificial Intelligence (Artificial Intelligence Act) And Amending Certain Union Legislative Acts’ COM(2021) 206 final, 21 April 2021. google scholar
  • Corner N, ‘Revealed: How your name, your ethnicity, and even your e-mail address could be adding hundreds of pounds to your car insurance premium’ Dailymail (29 June 2018) www.dailymail.co.uk/femail/article-5896561/How-email-address-add-hundreds-car-insurance-premium.html> accessed 20 June 2021. google scholar
  • Council Directive (EC) 2000/43 of 29 June 2000 implementing the principle of equal treatment between persons irrespective of racial or ethnic origin [2000] OJ L 180. google scholar
  • Council Directive (EC) 2000/78 of 27 November 2000 establishing a general framework for equal treatment in employment and occupation [2000] OJ L 303. google scholar
  • Council Directive (EC) 2001/95 of the European Parliament and of the Council of 3 December 2001 on general product safety [2001] OJ L 11. google scholar
  • Council of Europe, ‘Explanatory Report to the Protocol No. 12 to the Convention for the Protection of Human Rights and Fundamental Freedoms’ (2000) <https://rm.coe.int/09000016800cce48> accessed 20 June 2021. google scholar
  • Council of Europe, ‘Algorithms and Human Rights: Study On The Human Rights Dimensions Of Automated Data Processing Techniques (In Particular Algorithms) And Possible Regulatory Implications’ (Study Report) (DGI(2017)12) (2018). google scholar
  • Council Regulation (EC) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation) [2016] OJ L 119. google scholar
  • Craglia M et al., ‘Artificial Intelligence: A European Perspective’ (Joint Research Centre Working Papers) (2018). google scholar
  • Danks D and London A J, ‘Algorithmic Bias in Autonomous Systems’ (2017) IJCAI’17: Proceedings of the 26th International Joint Conference on Artificial Intelligence, 4691-4697. google scholar
  • Dastin J, ‘Amazon scraps secret AI recruiting tool that showed bias against women’ Reuters (11 October 2018) <https://www.reuters.com/article/us-amazon-com-jobs-automation-in...-airecruiting-tool-that-showed-bias-against-women-idUSKCN1MK08G> accessed 20 June 2021. google scholar
  • Data Protection Working Party Guidelines on Automated individual decision-making and Profiling for the purposes of Regulation 2016/679 Wp251rev.01. [2018]. google scholar
  • Datta A, Tschantz M C and Datta A, ‘Automated Experiments on Ad Privacy Settings A Tale of Opacity, Choice, and Discrimination’ (2015) Proceedings on Privacy Enhancing Technologies <https://www.andrew.cmu.edu/user/danupam/dtd-pets15.pdf> accessed 20 June 2021. google scholar
  • Delipetrev B, Tsinarakii C and Kostic U, ‘Historical Evolution of Artificial Intelligence’ (Technical Report) (Publications Office of the European Union, Luxembourg 2020) https://publications.jrc. ec.europa.eu/repository/handle/JRC120469 accessed 20 June 2021. google scholar
  • Dwork C et al., ‘Fairness Through Awareness, Proceedings of the 3rd Innovations in Theoretical Computer Science Conference’ (2012) ITCS ‘12 (Association for Computing Machinery), 214226. google scholar
  • Edwards, L and Veale M, ‘Slave to the algorithm: Why a right to an explanation is probably not the remedy you are looking for’ (2017) 16 Duke Law & Technology Review, 18-84. google scholar
  • Espinoza J and Murgia M, ‘Europe attempts to take leading role in regulating uses of AI’ Financial Times (24 April 2021) <https://www.ft.com/content/360faa3e-4110-4f38-b618-dd695deece90> accessed 20 June 2021. google scholar
  • European Commission, Proposal for a Council Directive on implementing the principle of equal treatment between persons irrespective of religion or belief, disability, age or sexual orientation [2008] {SEC(2008) 2180} {SEC(2008) 2181} /* COM/2008/0426 final - CNS 2008/0140 */. google scholar
  • European Commission, ‘Special Eurobarometer Discrimination in the EU in 2015’ (Report) (2015). google scholar
  • European Commission’s High-Level Expert Group on Artificial Intelligence, ‘A Definition of AI: Main Capabilities and Disciplines’ (2019) (Report) <https://digital-strategy.ec.europa.eu/en/ library/definition-artificial-intelligence-main-capabilities-and-scientific-disciplines> accessed 20 June 2021. google scholar
  • European Commission, ‘A Union of Equality: Gender Equality Strategy 2020-2025’ [2020] COM (2020) 152. google scholar
  • European Commission, ‘Internal Market, Industry, Entrepreneurship and SMEs’ <https://ec.europa. eu/growth/single-market/ce-marking/manufacturers_en> accessed 20 June 2021. google scholar
  • European Commission, ‘Harmonised Standards’ <https://ec.europa.eu/growth/single-market/ european-standards/harmonised-standards_en> accessed 20 June 2021. google scholar
  • European Convention for the Protection of Human Rights and Fundamental Freedoms (1950) <https://www.echr.coe.int/Documents/Convention_ENG.pdf> accessed 20 June 2021. google scholar
  • European Union Agency for Fundamental Rights, ‘#BigData: Discrimination in data-supported decision making’ (Report) (2018) Doi:10.2811/343905. google scholar
  • European Union Agency for Fundamental Rights, ‘Getting the Future Right-Artificial Intelligence and Fundamental Rights’ (2020) Doi:10.2811/774118. google scholar
  • Fairness, Accountability, and Transparency in Machine Learning <https://www.fatml.org/> accessed 20 June 2021. google scholar
  • Favaretto M, De Clercq E and Elger B S, ‘Big Data and discrimination: perils, promises and solutions. A systematic review’ (2019) 6 Journal of Big Data <https://doi.org/10.1186/s40537-019-0177-4> accessed 20 June 2021. google scholar
  • Financial Conduct Authority, ‘Pricing practices in the retail general insurance sector: Household insurance’ (2018) <https://www.fca.org.uk/publication/thematic-reviews/tr18-4.pdf> accessed 20 June 2021. google scholar
  • Stan F, ‘History, Motivations, and Core Themes’ in Keith Frankish and William W. Ramsey, (eds.), The Cambridge Handbook Of Artificial Intelligence (Cambridge University Press, UK 2014). google scholar
  • Fuster A et al., ‘Predictably Unequal? The Effects of Machine Learning on Credit Markets’ (2020). SSRN <https://ssrn.com/abstract=3072038> accessed 20 June 2021. google scholar
  • Future of Life Institute, ‘The Asilomar AI principles’ (2017) <https://futureoflife.org/ai-principles/> accessed 20 June 2021. google scholar
  • Gillis T B and Spiess J L, ‘Big Data and Discrimination’ (2019) 86 The University of Chicago Law Review, pp. 459-487. google scholar
  • Hacker P., ‘Teaching Fairness to Artificial Intelligence: Existing and Novel Strategies against Algorithmic Discrimination under EU Law’ (Pre-print version) (2019) <https://papers.ssrn.com/ sol3/papers.cfm?abstract_id=3164973&download=yes> accessed 20 June 2021. google scholar
  • Heinrichs B, ‘Discrimination in the age of artificial intelligence’ (2021) AI & Society Springer. <https://doi.org/10.1007/s00146-021-01192-2> accessed 20 June 2021. google scholar
  • Hildebrandt M and Gutwirth S, ‘Concise Conclusions: Citizens Out of Control’ in Hildebrandt M and Gutwirth S (eds.) Profiling the European Citizen: Cross-Disciplinary Perspectives (Springer, 2008). google scholar
  • Hill K, ‘How Target Figured Out A Teen Girl Was Pregnant Before Her Father Did’ Forbes (16 February 2012) <https://www.forbes.com/sites/kashmirhill/2012/02/16/how-target-figured-out-a-teen-girl-was-pregnant-before-her-father-did/?sh=415c8bfe6668> accessed 20 June 2021. google scholar
  • Humphreys P, ‘The philosophical novelty of computer simulation methods’ (2008) 169 Synthese, pp. 615-626 <https://doi.org/10.1007/s11229-008-9435-2> accessed 20 June 2021. google scholar
  • International Association of Insurance Supervisors, ‘Issues Paper on the Use of Big Data Analytics in Insurance’ (2020) <https://www.iaisweb.org/page/supervisory-material/issues-papers// file/89244/issues-paper-on-use-of-big-data-analytics-in-insurance> accessed 20 June 2021. google scholar
  • International Covenant on Civil and Political Rights adopted and opened for signature, ratification and accession by General Assembly resolution 2200A (XXI) of 16 December 1966 entry into force 23 March 1976. google scholar
  • Izzidien R, ‘Higher insurance if you’re called Mohammed? That’s just the start of institutionalized Islamophobia’ NewStatesman (23 January 2018) <https://www.newstatesman.com/politics/ uk/2018/01/higher-insurance-if-you-re-called-mohammed-s-just-start-institutionalised> accessed 20 June 2021. google scholar
  • Johnson K, Pasquale F and Chapman J, ‘Artificial Intelligence, Machine Learning, and Bias in Finance: Toward Responsible Innovation’ (2019) 88 2 Fordham Law Review 499-529. google scholar
  • Jones H, ‘When the wrong postcode puts insurance out of your reach’ The Guardian (30 October 2016) <https://www.theguardian.com/money/2016/oct/30/wrong-postcode-puts-insurance-out-of-reach> accessed 20 June 2021. google scholar
  • Kelion L, ‘Apple’s ‘sexist’ credit card investigated by US regulator’ BBC (11 November 2019) <https://www.bbc.co.uk/news/business-50365609> accessed 20 June 2021. google scholar
  • Kim P T, ‘Data-Driven Discrimination at Work’ (2017) 58 3 William & Mary Law Review, pp. 857-936. google scholar
  • Kleinberg J et al., ‘Discrimination in the Age of Algorithms’ (2018) 10 Journal of Legal Analysis pp. 113-174. google scholar
  • Kloza D et al., ‘Data protection impact assessments in the European Union: complementing the new legal framework towards a more robust protection of individuals’ (d.pia.lab Policy Brief) (2017) <https://cris.vub.be/files/32009890/dpialab_pb2017_1_final.pdf> accessed 20 June 2021. google scholar
  • Knutson T, ‘AI Lending Discrimination Needs To Be Tackled With Legislation Says House Financial Services Chair’ Forbes (7 May 2021) <https://www.forbes.com/sites/tedknutson/2021/05/07/ ai-lending-discrimination-needs-to-be-tackled-with-legislation-says-house-financial-services-chair/?sh=7d233f205c7d> accessed 20 June 2021. google scholar
  • Kretchmer H, ‘Insurers ‘risk breaking racism laws’’ BBC (9 February 2018) <https://www.bbc. com/news/business-43011882> accessed 20 June 2021. google scholar
  • Kroll J A et al., ‘Accountable Algorithms’ (2017) 165 University of Pennsylvania Law Review, 633-705. google scholar
  • Lacroix C, ‘Preventing discrimination caused by the use of artificial intelligence’ (Report of Committee on Equality and Non-Discrimination at Council of Europe) (2020). google scholar
  • Lambrecht A and Tucker C E, ‘Algorithmic Bias? An Empirical Study of Apparent Gender-Based Discrimination in the Display of STEM Career Ads’ (2019) 65 7 Management Science, 2966-2981. google scholar
  • Liboreiro J, ‘The higher the risk, the stricter the rule’: Brussels’ new draft rules on artificial intelligence’ Euronews (21.4.2021) <https://www.euronews.com/2021/04/21/the-higher-the-risk-the-stricter-the-rule-brussels-new-draft-rules-on-artificial-intellige> accessed 20 June 2021. google scholar
  • Loi M and Christen M, ‘Choosing how to discriminate: navigating ethical trade-offs in fair algorithmic design for the insurance sector’ (2021) Philosophy & Technology <https://doi. org/10.1007/s13347-021-00444-9> accessed 20 June 2021. google scholar
  • Lucci S and Kopec D, Artificial Intelligence in the 21st Century: A Living Introduction (Mercury Learning and Information, USA 2016). google scholar
  • MacCarthy M and Propp K, ‘Machines Learn That Brussels Writes the Rules: The EU’s New AI Regulation’ Lawfare Blog (28 April 2021) <https://www.lawfareblog.com/machines-learn-brussels-writes-rules-eus-new-ai-regulation> accessed 20 June 2021. google scholar
  • Makinen J, ‘The EU’s AI Regulation Proposal - How to Fix It?’ (2021) <https://teknologiateollisuus. fi/en/ajankohtaista/eus-ai-regulation-proposal-how-fix-it> accessed 20 June 2021. google scholar
  • Meneceur Y, ‘European Commission’s AI regulation proposal: between too much and too little?’ LinkedIn (23 April 2021) <https://www.linkedin.com/pulse/european-commissions-ai-regulation-proposal-between-too-meneceur/> accessed 20 June 2021. google scholar
  • Modernised Convention for the Protection of Individuals with Regard to the Processing of Personal Data CM/Inf(2018)15-final [2018]. google scholar
  • Möslein F, ‘Robots in the Boardroom: Artificial Intelligence and Corporate Law’ in Barfield W and Pagallo U (eds.), Research Handbook on the Law of Artificial Intelligence (Edward Elgar, UK 2018) 649-669. google scholar
  • Newell S and Marabelli M, ‘Strategic opportunities (and challenges) of algorithmic decision-making: A call for action on the long-term societal effects of ‘datification’’ (2015) 24 Journal of Strategic Information Systems, 3-15. google scholar
  • Peck D, ‘They’re Watching You at Work’ Atlantic (December 2013) <http://www.theatlantic.com/ magazine/archive/2013/12/theyre-watching-you-at-work/354681> accessed 20 June 2021. google scholar
  • Peppet S R, ‘Regulating the Internet of Things: first steps toward managing discrimination, privacy, security, and consent’ (2014) 93 Texas Law Review, 85-176. google scholar
  • Pettinger-Kearney N et al., ‘The EU’s proposed AI Regulation’ (2021) <https://www.freshfields. com/en-gb/our-thinking/campaigns/digital/artificial-intelligence/the-eus-proposed-ai-regulation/> accessed 20 June 2021. google scholar
  • Pope D G and Sydnor J R, ‘Implementing Anti-Discrimination Policies in Statistical Profiling Models’ (2011) 3 American Economic Journal: Economic Policy, 206-231. google scholar
  • Prince A E R and Schwarcz D, ‘Proxy Discrimination in the Age of Artificial Intelligence and Big Data’ (2020) 105 Iowa Law Review, 1257-1318. google scholar
  • Probasco J, ‘The Insurance Industry Confronts Its Own Racism’ Investopedia (1 September 2020) <https://www.investopedia.com/race-and-insurance-5075141#citation-9> accessed 20 June 2021. google scholar
  • Reinhold F and Müller A, ‘Algorithm Watch’s response to the European Commission’s proposed regulation on Artificial Intelligence - a major step with major gaps’ (Report) (22 April 2021) <https:// algorithmwatch.org/en/response-to-eu-ai-regulation-proposal-2021/> accessed 20 June 2021. google scholar
  • Rieke A, Bogen M and Robinson D G, ‘Public scrutiny of automated decisions: Early lessons and emerging methods’ (Upturn and Omidyar Network Report) (2018) <https://www.data.govt.nz/ assets/Uploads/Public-Scrutiny-of-Automated-Decisions.pdf> accessed 20 June 2021. google scholar
  • Rodrigues R, ‘Legal and human rights issues of AI: Gaps, challenges and vulnerabilities’ (2020) 4 Journal of Responsible Technology <https://doi.org/10.1016/j.jrt.2020.100005> accessed 20 June 2021. google scholar
  • Samoili S et al., ‘AI Watch. Defining Artificial Intelligence. Towards An Operational Definition And Taxonomy Of Artificial Intelligence’ (Joint Research Centre Technical Reports) (Luxembourg 2020). google scholar
  • Schauer F, ‘Statistical (and non-statistical) discrimination’ in Kasper Lippert-Rasmussen (ed.), The Routledge Handbook of the Ethics of Discrimination, (Routledge, UK 2018), 42-53. google scholar
  • Swedloff R, ‘Risk classification’s Big Data (r)evolution’ (2014) 21 Connecticut Insurance Law Journal, 339-373. google scholar
  • The European Consumer Organization, ‘Automated Decision Making And Artificial Intelligence - A Consumer Perspective’ (Report) (2018) <https://www.beuc.eu/publications/beuc-x-2018-058_ automated_decision_making_and_artificial_intelligence.pdf> accessed 20 June 2021. google scholar
  • The European Consumer Organization, ‘AI Rights for Consumers’ (Report) (2019) <https://www. beuc.eu/publications/beuc-x-2019-063_ai_rights_for_consumers.pdf> accessed 20 June 2021. google scholar
  • The European Consumer Organization, ‘The Use Of Big Data And Artificial Intelligence In Insurance’ (Report) (2020) <https://www.beuc.eu/publications/beuc-x-2020-039_beuc_ position_paper_big_data_and_ai_in_insurances.pdf> accessed 20 June 2021. google scholar
  • The European Consumer Organization, ‘EU proposal for artificial intelligence law is weak on consumer protection’ (Report) (2021) <https://www.beuc.eu/publications/eu-proposal-artificial-intelligence-law-weak-consumer-protection/html> accessed 20 June 2021. google scholar
  • The Federal Government of Germany, ‘Artificial Intelligence Strategy’ (2018) <https://www.ki-strategie-deutschland.de/home.html?file=files/downloads/Nationale_KI-Strategie_engl.pdf> accessed 20 June 2021. google scholar
  • The Montreal Declaration for Responsible AI (2017) <https://www.montrealdeclaration-responsibleai.com/the-declaration> accessed 20 June 2021. google scholar
  • The Partnership on AI to Benefit People and Society <https://www.partnershiponai.org/about/> accessed 20 June 2021. google scholar
  • The Royal Society, ‘Machine Learning: The Power And Promise Of Computers That Learn By Example’ (Report) (April 2017) <royalsociety.org/machine-learning> accessed 20 June 2021. google scholar
  • The Royal Society, ‘Explainable AI: the basics’ (Policy briefing) (2019) <royalsociety.org/ai-interpretability> accessed 20 June 2021. google scholar
  • The White House, ‘Big Data: Seizing Opportunities, Preserving Values’ (Report) (2014). google scholar
  • Universal Declaration of Human Rights, UNGA Res 217/A (10 December 1948). google scholar
  • Valentino-Devries J, Singer-Vine J, and Soltani A, ‘Websites vary prices, deals based on users’ information’ Wall Street Journal (24 December 2012) <https://www.wsj.com/articles/SB10001 424127887323777204578189391813881534> accessed June 2021. google scholar
  • Vigdor N, ‘Apple Card Investigated After Gender Discrimination Complaints’ The New York Times (10 November 2019) <https://www.nytimes.com/2019/11/10/business/Apple-credit-card-investigation.html> accessed June May 2021. google scholar
  • Wachter S, Mittelstadt B and Russell C, ‘Counterfactual explanations without opening the black box: automated decisions and the GDPR’ (2018) 31 2 Harvard Journal of Law & Technology, 842-887. google scholar
  • Wachter S, Mittelstadt B and Russell C, ‘Why Fairness Cannot Be Automated: Bridging The Gap Between EU Non-Discrimination Law And AI’ (Draft) (2020) <https://papers.ssrn.com/sol3/ papers.cfm?abstract_id=3547922> accessed 20 June 2021. google scholar
  • Wagner B et al., ‘Algorithms and human rights. Study on the human rights dimensions of automated data processing techniques and possible regulatory implications’ (Study Report) (DGI(2017)12) (2018) <https://rm.coe.int/algorithms-and-human-rights-en-rev/16807956b5> accessed 20 June 2021. google scholar
  • Wahlster W and Winterhalter C, ‘German Standardization Roadmap On Artificial Intelligence’ (German Institute for Standardization and DKE German Commission for Electrical, Electronic & Information Technologies of DIN and VDE November 2020) <https://www.din.de/resource/ blob/772610/e96c34dd6b12900ea75b460538805349/normungsroadmap-en-data.pdf> accessed 20 June 2021. google scholar
  • Walch K, ‘Is There A Difference Between Assisted Intelligence Vs. Augmented Intelligence?’ Forbes. (12 January 2020) <https://www.forbes.com/sites/cognitiveworld/2020/01/12/is-there-a-difference-between-assisted-intelligence-vs-augmented-intelligence/?sh=3300950426ab> accessed 20 June 2021. google scholar
  • Wang P, ‘On the Working Definition of Intelligence’ (1995) <http://citeseerx.ist.psu.edu/viewdoc/ summary?doi=10.1.1.55.5012> accessed 20 May 2021. google scholar
  • Weisbaum H, ‘Data Mining Is Now Used to Set Insurance Rates; Critics Cry Foul’ CNBC (16 April 2014) <https://www.cnbc.com/2014/04/16/data-mining-is-now-used-to-set-insurance-rates-critics-cry-fowl.html#:~:text=You%20might%20not%20like%20the,to%20provide%20 you%20that%20coverage> accessed 20 June 2021. google scholar
  • Whitehead D, ‘AI & Algorithms (Part 3): Why the EU’s AI regulation is a ground breaking proposal’ Hogan Lovells Engage (3 May 2021) <https://www.engage.hoganlovells.com/ knowledgeservices/viewContent. google scholar action?key=Ec8teaJ9VaqLasZjnzVtGsxgHJMKLFEppVpbbVX%2B3OXcP3PYxlq7sZUjdb-Sm5FIetvAtgf1eVU8%3D&nav=FRbANEucS95NMLRN47z%2BeeOgEFCt8EGQ0qF-foEM4UR4%3D&emailtofriendview=true&freeviewlink=true> accessed 20 June 2021. google scholar
  • Williams B et al., ‘How Algorithms Discriminate Based on Data they Lack: Challenges, Solutions, and Policy Implications’ (2018) 8 Journal of Information Policy, 78-115. google scholar
  • World Economic Forum, ‘Transforming Paradigms A Global AI in Financial Services Survey’ (2020) (Report) <http://www3.weforum.org/docs/WEF_AI_in_Financial_Services_Survey. pdf> accessed 20 June 2021. google scholar
  • Xenidis R, ‘Tuning EU equality law to algorithmic discrimination: Three pathways to resilience’ (2020) 27(6) Maastricht Journal of European and Comparative Law, 736-758. google scholar
  • Xenidis R and Senden L, ‘EU non-discrimination law in the era of artificial intelligence: Mapping the challenges of algorithmic discrimination’ in Ulf Bernitz et al. (eds.), General principles of EU law and the EU digital order (Kluwer Law International 2020), 151-182. google scholar

European Union Law and Mitigation of Artificial Intelligence-Related Discrimination Risks in the Private Sector: With Special Focus on the Proposed Artificial Intelligence Act

Year 2022, Issue: 71, 265 - 307, 31.12.2022
https://doi.org/10.26650/annales.2022.71.0002

Abstract

Integrating AI systems into decision-making processes in the private sector may place the right to non-discrimination in danger. In order to illustrate this threat, risky fields in the private sector, namely employment, banking, advertising, pricing and insurance, were investigated in this paper with authentic examples of AI-related discrimination. Then, the current EU non-discrimination laws and data protection laws were examined, and it was found out that these EU laws do not have the necessary tools to tackle specific risks arising from AI-related discrimination in the private sector. Therefore, there is an immediate need for new EU legislation equipped with tools which explicitly target AI-related discrimination risks in the private sector. The proposed AI Act may provide new tools against AI-related discrimination in the private sector. Thus, this paper analyzes the proposed AI Act in terms of its potential impacts on mitigating AI-related discrimination risks. Due to the cradle-to-grave approach adopted by the proposed AI Act, providers and users of high-risk AI systems are required to comply with various specific ex-ante and ex-post obligations. It is found out that these obligations can contribute to the mitigation of AI-related discrimination by providing new legal tools. However, these tools are not sufficient in the face of AI-related discrimination risks. Therefore, it is concluded that the crucial need for specific legislation to mitigate AI-related discrimination risks in the private sector is still present.

References

  • Accesswire, ‘72% of Resumes are Never Seen by Employers’ (16 February 2016) <https://www. accesswire.com/436847/72-of-Resumes-are-Never-Seen-by-Employers> accessed 20 June 2021. google scholar
  • Ali M et al., ‘Discrimination Through Optimization: How Facebook’s ad Delivery Can Lead to Skewed Outcomes’ (2019) Proceedings of the ACM on Human-Computer Interaction. <https:// arxiv.org/pdf/1904.02095.pdf> accessed 20 June 2021. google scholar
  • Angelopoulos C et al., ‘Study Of Fundamental Rights Limitations For Online Enforcement Through Self-Regulation’ (Study Report of Institute for Information Law, University of Amsterdam) (2016) <https://scholarlypublications.universiteitleiden.nl/access/item%3A2869513/view> accessed 20 June 2021. google scholar
  • Angwin J, Mattu S and Larson J, ‘The tiger mom tax: Asians are nearly twice as likely to get a higher price from Princeton Review’. ProPublica (1 September 2015) <https://www.ProPublica.org/article/ asians-nearly-twice-as-likely-to-get-higher-price-from-princeton-review> accessed 20 June 2021. google scholar
  • Angwin J, Scheiber N and Tobin A, ‘Dozens of companies are using Facebook to exclude older workers from job ads’ ProPublica (20 December 2017) <https://www.propublica.org/article/ facebook-ads-age-discrimination-targeting> accessed 20 June 2021. google scholar
  • Angwin J, Tobin A and Varner M, ‘Facebook (still) letting housing advertisers exclude users by race’ ProPublica (21 November 2017) <https://www.propublica.org/article/facebook-advertising-discrimination-housing-race-sex-national-origin> accessed 20 June 2021. google scholar
  • Avraham R, ‘Discrimination and Insurance’ in Lippert-Rasmussen K. (ed.), The Routledge Handbook of the Ethics of Discrimination, (Routledge, UK 2018), pp. 335-347. google scholar
  • Bartlett R et al., ‘Consumer-Lending Discrimination in the FinTech Era’ (2019) (National Bureau of Economic Research Working Papers 25943). google scholar
  • Barocas S and Selbst A D, ‘Big Data’s Disparate Impact’ (2016) 104 California Law Review, 671-732. google scholar
  • Bathaee Y, ‘The Artificial Intelligence Black Box And The Failure Of Intent And Causation’ (2018) 31:2 Harvard Journal of Law & Technology, 890-938. google scholar
  • Beck S et al., ‘Künstliche Intelligenz und Diskriminierung Herausforderungen und Lösungsansatze’ (2019) (Whitepaper from Plattform Lernende Systeme) <https://www.plattform-lernende-systeme.de/publikationen-details/kuenstliche-intelligenz-und-diskriminierung-herausforderungen-und-loesungsansaetze.html> accessed 20 June 2021. google scholar
  • Biao v. Denmark (Grand Chamber) (App no 38590/10) ECHR 24 May 2016. google scholar
  • Bodo B et al., ‘Tackling the algorithmic control crisis-the technical, legal, and ethical challenges of research into algorithmic agents’ (2017) 19 Yale Journal of Law & Technology, 133-180. google scholar
  • Borgesius F Z, ‘Discrimination, Artificial Intelligence and Algorithmic Decision-making’ (Study Report) (Council of Europe, Strasbourg 2018) <https://rm.coe.int/discrimination-artificial-intelligence-and-algorithmic-decision-making/1680925d73> accessed 20 June 2021. google scholar
  • Borgesius F J Z, ‘Strengthening legal protection against discrimination by algorithms and artificial intelligence’ (2020) 24 10 The International Journal of Human Rights, 1572- 1593. google scholar
  • Bundesanstalt für Finanzdienstleistungsaufsicht, ‘Big data meets artificial intelligence: Challenges and implications for the supervision and regulation of financial services’ (2018) (Study) www.bafin.de/SharedDocs/Downloads/EN/dl_bdai_studie_en.html> accessed 20 June 2021. google scholar
  • Burrell J, ‘How the machine ‘thinks’: understanding opacity in machine learning algorithms’ (2016) Big Data & Society Doi:10.1177/2053951715622512. google scholar
  • Case 170/84, Bilka-Kaufhaus GmbH and Karin Weber von Hartz, (Judgment of the Court of 13 May 1986.) google scholar
  • Case C-83/14, “CHEZ Razpredelenie Bulgaria” AD v Komisia za zashtita ot diskriminatsia, (Judgment of the Court (Grand Chamber) of 16 July 2015). google scholar
  • Case C443/15, David L. Parris v Trinity College Dublin and Others (Judgment of the Court (First Chamber) of 24 November 2016). google scholar
  • Case C-414/16, Vera Egenberger v Evangelisches Werk für Diakonie und Entwicklung e.V. (Judgment of the Court (Grand Chamber) of 17 April 2018). google scholar
  • Chander S and Jakubowska E, ‘EU’s AI law needs major changes to prevent discrimination and mass surveillance’ EDRI (2021) <https://edri.org/our-work/eus-ai-law-needs-major-changes-to-prevent-discrimination-and-mass-surveillance/> accessed 20 June 2021. google scholar
  • Charter of Fundamental Rights of the European Union, 2012/C 326/02 OJ C 326, [2012]. google scholar
  • Commission (EC) ‘Proposal for a Regulation Of The European Parliament And Of The Council Laying Down Harmonised Rules On Artificial Intelligence (Artificial Intelligence Act) And Amending Certain Union Legislative Acts’ COM(2021) 206 final, 21 April 2021. google scholar
  • Corner N, ‘Revealed: How your name, your ethnicity, and even your e-mail address could be adding hundreds of pounds to your car insurance premium’ Dailymail (29 June 2018) www.dailymail.co.uk/femail/article-5896561/How-email-address-add-hundreds-car-insurance-premium.html> accessed 20 June 2021. google scholar
  • Council Directive (EC) 2000/43 of 29 June 2000 implementing the principle of equal treatment between persons irrespective of racial or ethnic origin [2000] OJ L 180. google scholar
  • Council Directive (EC) 2000/78 of 27 November 2000 establishing a general framework for equal treatment in employment and occupation [2000] OJ L 303. google scholar
  • Council Directive (EC) 2001/95 of the European Parliament and of the Council of 3 December 2001 on general product safety [2001] OJ L 11. google scholar
  • Council of Europe, ‘Explanatory Report to the Protocol No. 12 to the Convention for the Protection of Human Rights and Fundamental Freedoms’ (2000) <https://rm.coe.int/09000016800cce48> accessed 20 June 2021. google scholar
  • Council of Europe, ‘Algorithms and Human Rights: Study On The Human Rights Dimensions Of Automated Data Processing Techniques (In Particular Algorithms) And Possible Regulatory Implications’ (Study Report) (DGI(2017)12) (2018). google scholar
  • Council Regulation (EC) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation) [2016] OJ L 119. google scholar
  • Craglia M et al., ‘Artificial Intelligence: A European Perspective’ (Joint Research Centre Working Papers) (2018). google scholar
  • Danks D and London A J, ‘Algorithmic Bias in Autonomous Systems’ (2017) IJCAI’17: Proceedings of the 26th International Joint Conference on Artificial Intelligence, 4691-4697. google scholar
  • Dastin J, ‘Amazon scraps secret AI recruiting tool that showed bias against women’ Reuters (11 October 2018) <https://www.reuters.com/article/us-amazon-com-jobs-automation-in...-airecruiting-tool-that-showed-bias-against-women-idUSKCN1MK08G> accessed 20 June 2021. google scholar
  • Data Protection Working Party Guidelines on Automated individual decision-making and Profiling for the purposes of Regulation 2016/679 Wp251rev.01. [2018]. google scholar
  • Datta A, Tschantz M C and Datta A, ‘Automated Experiments on Ad Privacy Settings A Tale of Opacity, Choice, and Discrimination’ (2015) Proceedings on Privacy Enhancing Technologies <https://www.andrew.cmu.edu/user/danupam/dtd-pets15.pdf> accessed 20 June 2021. google scholar
  • Delipetrev B, Tsinarakii C and Kostic U, ‘Historical Evolution of Artificial Intelligence’ (Technical Report) (Publications Office of the European Union, Luxembourg 2020) https://publications.jrc. ec.europa.eu/repository/handle/JRC120469 accessed 20 June 2021. google scholar
  • Dwork C et al., ‘Fairness Through Awareness, Proceedings of the 3rd Innovations in Theoretical Computer Science Conference’ (2012) ITCS ‘12 (Association for Computing Machinery), 214226. google scholar
  • Edwards, L and Veale M, ‘Slave to the algorithm: Why a right to an explanation is probably not the remedy you are looking for’ (2017) 16 Duke Law & Technology Review, 18-84. google scholar
  • Espinoza J and Murgia M, ‘Europe attempts to take leading role in regulating uses of AI’ Financial Times (24 April 2021) <https://www.ft.com/content/360faa3e-4110-4f38-b618-dd695deece90> accessed 20 June 2021. google scholar
  • European Commission, Proposal for a Council Directive on implementing the principle of equal treatment between persons irrespective of religion or belief, disability, age or sexual orientation [2008] {SEC(2008) 2180} {SEC(2008) 2181} /* COM/2008/0426 final - CNS 2008/0140 */. google scholar
  • European Commission, ‘Special Eurobarometer Discrimination in the EU in 2015’ (Report) (2015). google scholar
  • European Commission’s High-Level Expert Group on Artificial Intelligence, ‘A Definition of AI: Main Capabilities and Disciplines’ (2019) (Report) <https://digital-strategy.ec.europa.eu/en/ library/definition-artificial-intelligence-main-capabilities-and-scientific-disciplines> accessed 20 June 2021. google scholar
  • European Commission, ‘A Union of Equality: Gender Equality Strategy 2020-2025’ [2020] COM (2020) 152. google scholar
  • European Commission, ‘Internal Market, Industry, Entrepreneurship and SMEs’ <https://ec.europa. eu/growth/single-market/ce-marking/manufacturers_en> accessed 20 June 2021. google scholar
  • European Commission, ‘Harmonised Standards’ <https://ec.europa.eu/growth/single-market/ european-standards/harmonised-standards_en> accessed 20 June 2021. google scholar
  • European Convention for the Protection of Human Rights and Fundamental Freedoms (1950) <https://www.echr.coe.int/Documents/Convention_ENG.pdf> accessed 20 June 2021. google scholar
  • European Union Agency for Fundamental Rights, ‘#BigData: Discrimination in data-supported decision making’ (Report) (2018) Doi:10.2811/343905. google scholar
  • European Union Agency for Fundamental Rights, ‘Getting the Future Right-Artificial Intelligence and Fundamental Rights’ (2020) Doi:10.2811/774118. google scholar
  • Fairness, Accountability, and Transparency in Machine Learning <https://www.fatml.org/> accessed 20 June 2021. google scholar
  • Favaretto M, De Clercq E and Elger B S, ‘Big Data and discrimination: perils, promises and solutions. A systematic review’ (2019) 6 Journal of Big Data <https://doi.org/10.1186/s40537-019-0177-4> accessed 20 June 2021. google scholar
  • Financial Conduct Authority, ‘Pricing practices in the retail general insurance sector: Household insurance’ (2018) <https://www.fca.org.uk/publication/thematic-reviews/tr18-4.pdf> accessed 20 June 2021. google scholar
  • Stan F, ‘History, Motivations, and Core Themes’ in Keith Frankish and William W. Ramsey, (eds.), The Cambridge Handbook Of Artificial Intelligence (Cambridge University Press, UK 2014). google scholar
  • Fuster A et al., ‘Predictably Unequal? The Effects of Machine Learning on Credit Markets’ (2020). SSRN <https://ssrn.com/abstract=3072038> accessed 20 June 2021. google scholar
  • Future of Life Institute, ‘The Asilomar AI principles’ (2017) <https://futureoflife.org/ai-principles/> accessed 20 June 2021. google scholar
  • Gillis T B and Spiess J L, ‘Big Data and Discrimination’ (2019) 86 The University of Chicago Law Review, pp. 459-487. google scholar
  • Hacker P., ‘Teaching Fairness to Artificial Intelligence: Existing and Novel Strategies against Algorithmic Discrimination under EU Law’ (Pre-print version) (2019) <https://papers.ssrn.com/ sol3/papers.cfm?abstract_id=3164973&download=yes> accessed 20 June 2021. google scholar
  • Heinrichs B, ‘Discrimination in the age of artificial intelligence’ (2021) AI & Society Springer. <https://doi.org/10.1007/s00146-021-01192-2> accessed 20 June 2021. google scholar
  • Hildebrandt M and Gutwirth S, ‘Concise Conclusions: Citizens Out of Control’ in Hildebrandt M and Gutwirth S (eds.) Profiling the European Citizen: Cross-Disciplinary Perspectives (Springer, 2008). google scholar
  • Hill K, ‘How Target Figured Out A Teen Girl Was Pregnant Before Her Father Did’ Forbes (16 February 2012) <https://www.forbes.com/sites/kashmirhill/2012/02/16/how-target-figured-out-a-teen-girl-was-pregnant-before-her-father-did/?sh=415c8bfe6668> accessed 20 June 2021. google scholar
  • Humphreys P, ‘The philosophical novelty of computer simulation methods’ (2008) 169 Synthese, pp. 615-626 <https://doi.org/10.1007/s11229-008-9435-2> accessed 20 June 2021. google scholar
  • International Association of Insurance Supervisors, ‘Issues Paper on the Use of Big Data Analytics in Insurance’ (2020) <https://www.iaisweb.org/page/supervisory-material/issues-papers// file/89244/issues-paper-on-use-of-big-data-analytics-in-insurance> accessed 20 June 2021. google scholar
  • International Covenant on Civil and Political Rights adopted and opened for signature, ratification and accession by General Assembly resolution 2200A (XXI) of 16 December 1966 entry into force 23 March 1976. google scholar
  • Izzidien R, ‘Higher insurance if you’re called Mohammed? That’s just the start of institutionalized Islamophobia’ NewStatesman (23 January 2018) <https://www.newstatesman.com/politics/ uk/2018/01/higher-insurance-if-you-re-called-mohammed-s-just-start-institutionalised> accessed 20 June 2021. google scholar
  • Johnson K, Pasquale F and Chapman J, ‘Artificial Intelligence, Machine Learning, and Bias in Finance: Toward Responsible Innovation’ (2019) 88 2 Fordham Law Review 499-529. google scholar
  • Jones H, ‘When the wrong postcode puts insurance out of your reach’ The Guardian (30 October 2016) <https://www.theguardian.com/money/2016/oct/30/wrong-postcode-puts-insurance-out-of-reach> accessed 20 June 2021. google scholar
  • Kelion L, ‘Apple’s ‘sexist’ credit card investigated by US regulator’ BBC (11 November 2019) <https://www.bbc.co.uk/news/business-50365609> accessed 20 June 2021. google scholar
  • Kim P T, ‘Data-Driven Discrimination at Work’ (2017) 58 3 William & Mary Law Review, pp. 857-936. google scholar
  • Kleinberg J et al., ‘Discrimination in the Age of Algorithms’ (2018) 10 Journal of Legal Analysis pp. 113-174. google scholar
  • Kloza D et al., ‘Data protection impact assessments in the European Union: complementing the new legal framework towards a more robust protection of individuals’ (d.pia.lab Policy Brief) (2017) <https://cris.vub.be/files/32009890/dpialab_pb2017_1_final.pdf> accessed 20 June 2021. google scholar
  • Knutson T, ‘AI Lending Discrimination Needs To Be Tackled With Legislation Says House Financial Services Chair’ Forbes (7 May 2021) <https://www.forbes.com/sites/tedknutson/2021/05/07/ ai-lending-discrimination-needs-to-be-tackled-with-legislation-says-house-financial-services-chair/?sh=7d233f205c7d> accessed 20 June 2021. google scholar
  • Kretchmer H, ‘Insurers ‘risk breaking racism laws’’ BBC (9 February 2018) <https://www.bbc. com/news/business-43011882> accessed 20 June 2021. google scholar
  • Kroll J A et al., ‘Accountable Algorithms’ (2017) 165 University of Pennsylvania Law Review, 633-705. google scholar
  • Lacroix C, ‘Preventing discrimination caused by the use of artificial intelligence’ (Report of Committee on Equality and Non-Discrimination at Council of Europe) (2020). google scholar
  • Lambrecht A and Tucker C E, ‘Algorithmic Bias? An Empirical Study of Apparent Gender-Based Discrimination in the Display of STEM Career Ads’ (2019) 65 7 Management Science, 2966-2981. google scholar
  • Liboreiro J, ‘The higher the risk, the stricter the rule’: Brussels’ new draft rules on artificial intelligence’ Euronews (21.4.2021) <https://www.euronews.com/2021/04/21/the-higher-the-risk-the-stricter-the-rule-brussels-new-draft-rules-on-artificial-intellige> accessed 20 June 2021. google scholar
  • Loi M and Christen M, ‘Choosing how to discriminate: navigating ethical trade-offs in fair algorithmic design for the insurance sector’ (2021) Philosophy & Technology <https://doi. org/10.1007/s13347-021-00444-9> accessed 20 June 2021. google scholar
  • Lucci S and Kopec D, Artificial Intelligence in the 21st Century: A Living Introduction (Mercury Learning and Information, USA 2016). google scholar
  • MacCarthy M and Propp K, ‘Machines Learn That Brussels Writes the Rules: The EU’s New AI Regulation’ Lawfare Blog (28 April 2021) <https://www.lawfareblog.com/machines-learn-brussels-writes-rules-eus-new-ai-regulation> accessed 20 June 2021. google scholar
  • Makinen J, ‘The EU’s AI Regulation Proposal - How to Fix It?’ (2021) <https://teknologiateollisuus. fi/en/ajankohtaista/eus-ai-regulation-proposal-how-fix-it> accessed 20 June 2021. google scholar
  • Meneceur Y, ‘European Commission’s AI regulation proposal: between too much and too little?’ LinkedIn (23 April 2021) <https://www.linkedin.com/pulse/european-commissions-ai-regulation-proposal-between-too-meneceur/> accessed 20 June 2021. google scholar
  • Modernised Convention for the Protection of Individuals with Regard to the Processing of Personal Data CM/Inf(2018)15-final [2018]. google scholar
  • Möslein F, ‘Robots in the Boardroom: Artificial Intelligence and Corporate Law’ in Barfield W and Pagallo U (eds.), Research Handbook on the Law of Artificial Intelligence (Edward Elgar, UK 2018) 649-669. google scholar
  • Newell S and Marabelli M, ‘Strategic opportunities (and challenges) of algorithmic decision-making: A call for action on the long-term societal effects of ‘datification’’ (2015) 24 Journal of Strategic Information Systems, 3-15. google scholar
  • Peck D, ‘They’re Watching You at Work’ Atlantic (December 2013) <http://www.theatlantic.com/ magazine/archive/2013/12/theyre-watching-you-at-work/354681> accessed 20 June 2021. google scholar
  • Peppet S R, ‘Regulating the Internet of Things: first steps toward managing discrimination, privacy, security, and consent’ (2014) 93 Texas Law Review, 85-176. google scholar
  • Pettinger-Kearney N et al., ‘The EU’s proposed AI Regulation’ (2021) <https://www.freshfields. com/en-gb/our-thinking/campaigns/digital/artificial-intelligence/the-eus-proposed-ai-regulation/> accessed 20 June 2021. google scholar
  • Pope D G and Sydnor J R, ‘Implementing Anti-Discrimination Policies in Statistical Profiling Models’ (2011) 3 American Economic Journal: Economic Policy, 206-231. google scholar
  • Prince A E R and Schwarcz D, ‘Proxy Discrimination in the Age of Artificial Intelligence and Big Data’ (2020) 105 Iowa Law Review, 1257-1318. google scholar
  • Probasco J, ‘The Insurance Industry Confronts Its Own Racism’ Investopedia (1 September 2020) <https://www.investopedia.com/race-and-insurance-5075141#citation-9> accessed 20 June 2021. google scholar
  • Reinhold F and Müller A, ‘Algorithm Watch’s response to the European Commission’s proposed regulation on Artificial Intelligence - a major step with major gaps’ (Report) (22 April 2021) <https:// algorithmwatch.org/en/response-to-eu-ai-regulation-proposal-2021/> accessed 20 June 2021. google scholar
  • Rieke A, Bogen M and Robinson D G, ‘Public scrutiny of automated decisions: Early lessons and emerging methods’ (Upturn and Omidyar Network Report) (2018) <https://www.data.govt.nz/ assets/Uploads/Public-Scrutiny-of-Automated-Decisions.pdf> accessed 20 June 2021. google scholar
  • Rodrigues R, ‘Legal and human rights issues of AI: Gaps, challenges and vulnerabilities’ (2020) 4 Journal of Responsible Technology <https://doi.org/10.1016/j.jrt.2020.100005> accessed 20 June 2021. google scholar
  • Samoili S et al., ‘AI Watch. Defining Artificial Intelligence. Towards An Operational Definition And Taxonomy Of Artificial Intelligence’ (Joint Research Centre Technical Reports) (Luxembourg 2020). google scholar
  • Schauer F, ‘Statistical (and non-statistical) discrimination’ in Kasper Lippert-Rasmussen (ed.), The Routledge Handbook of the Ethics of Discrimination, (Routledge, UK 2018), 42-53. google scholar
  • Swedloff R, ‘Risk classification’s Big Data (r)evolution’ (2014) 21 Connecticut Insurance Law Journal, 339-373. google scholar
  • The European Consumer Organization, ‘Automated Decision Making And Artificial Intelligence - A Consumer Perspective’ (Report) (2018) <https://www.beuc.eu/publications/beuc-x-2018-058_ automated_decision_making_and_artificial_intelligence.pdf> accessed 20 June 2021. google scholar
  • The European Consumer Organization, ‘AI Rights for Consumers’ (Report) (2019) <https://www. beuc.eu/publications/beuc-x-2019-063_ai_rights_for_consumers.pdf> accessed 20 June 2021. google scholar
  • The European Consumer Organization, ‘The Use Of Big Data And Artificial Intelligence In Insurance’ (Report) (2020) <https://www.beuc.eu/publications/beuc-x-2020-039_beuc_ position_paper_big_data_and_ai_in_insurances.pdf> accessed 20 June 2021. google scholar
  • The European Consumer Organization, ‘EU proposal for artificial intelligence law is weak on consumer protection’ (Report) (2021) <https://www.beuc.eu/publications/eu-proposal-artificial-intelligence-law-weak-consumer-protection/html> accessed 20 June 2021. google scholar
  • The Federal Government of Germany, ‘Artificial Intelligence Strategy’ (2018) <https://www.ki-strategie-deutschland.de/home.html?file=files/downloads/Nationale_KI-Strategie_engl.pdf> accessed 20 June 2021. google scholar
  • The Montreal Declaration for Responsible AI (2017) <https://www.montrealdeclaration-responsibleai.com/the-declaration> accessed 20 June 2021. google scholar
  • The Partnership on AI to Benefit People and Society <https://www.partnershiponai.org/about/> accessed 20 June 2021. google scholar
  • The Royal Society, ‘Machine Learning: The Power And Promise Of Computers That Learn By Example’ (Report) (April 2017) <royalsociety.org/machine-learning> accessed 20 June 2021. google scholar
  • The Royal Society, ‘Explainable AI: the basics’ (Policy briefing) (2019) <royalsociety.org/ai-interpretability> accessed 20 June 2021. google scholar
  • The White House, ‘Big Data: Seizing Opportunities, Preserving Values’ (Report) (2014). google scholar
  • Universal Declaration of Human Rights, UNGA Res 217/A (10 December 1948). google scholar
  • Valentino-Devries J, Singer-Vine J, and Soltani A, ‘Websites vary prices, deals based on users’ information’ Wall Street Journal (24 December 2012) <https://www.wsj.com/articles/SB10001 424127887323777204578189391813881534> accessed June 2021. google scholar
  • Vigdor N, ‘Apple Card Investigated After Gender Discrimination Complaints’ The New York Times (10 November 2019) <https://www.nytimes.com/2019/11/10/business/Apple-credit-card-investigation.html> accessed June May 2021. google scholar
  • Wachter S, Mittelstadt B and Russell C, ‘Counterfactual explanations without opening the black box: automated decisions and the GDPR’ (2018) 31 2 Harvard Journal of Law & Technology, 842-887. google scholar
  • Wachter S, Mittelstadt B and Russell C, ‘Why Fairness Cannot Be Automated: Bridging The Gap Between EU Non-Discrimination Law And AI’ (Draft) (2020) <https://papers.ssrn.com/sol3/ papers.cfm?abstract_id=3547922> accessed 20 June 2021. google scholar
  • Wagner B et al., ‘Algorithms and human rights. Study on the human rights dimensions of automated data processing techniques and possible regulatory implications’ (Study Report) (DGI(2017)12) (2018) <https://rm.coe.int/algorithms-and-human-rights-en-rev/16807956b5> accessed 20 June 2021. google scholar
  • Wahlster W and Winterhalter C, ‘German Standardization Roadmap On Artificial Intelligence’ (German Institute for Standardization and DKE German Commission for Electrical, Electronic & Information Technologies of DIN and VDE November 2020) <https://www.din.de/resource/ blob/772610/e96c34dd6b12900ea75b460538805349/normungsroadmap-en-data.pdf> accessed 20 June 2021. google scholar
  • Walch K, ‘Is There A Difference Between Assisted Intelligence Vs. Augmented Intelligence?’ Forbes. (12 January 2020) <https://www.forbes.com/sites/cognitiveworld/2020/01/12/is-there-a-difference-between-assisted-intelligence-vs-augmented-intelligence/?sh=3300950426ab> accessed 20 June 2021. google scholar
  • Wang P, ‘On the Working Definition of Intelligence’ (1995) <http://citeseerx.ist.psu.edu/viewdoc/ summary?doi=10.1.1.55.5012> accessed 20 May 2021. google scholar
  • Weisbaum H, ‘Data Mining Is Now Used to Set Insurance Rates; Critics Cry Foul’ CNBC (16 April 2014) <https://www.cnbc.com/2014/04/16/data-mining-is-now-used-to-set-insurance-rates-critics-cry-fowl.html#:~:text=You%20might%20not%20like%20the,to%20provide%20 you%20that%20coverage> accessed 20 June 2021. google scholar
  • Whitehead D, ‘AI & Algorithms (Part 3): Why the EU’s AI regulation is a ground breaking proposal’ Hogan Lovells Engage (3 May 2021) <https://www.engage.hoganlovells.com/ knowledgeservices/viewContent. google scholar action?key=Ec8teaJ9VaqLasZjnzVtGsxgHJMKLFEppVpbbVX%2B3OXcP3PYxlq7sZUjdb-Sm5FIetvAtgf1eVU8%3D&nav=FRbANEucS95NMLRN47z%2BeeOgEFCt8EGQ0qF-foEM4UR4%3D&emailtofriendview=true&freeviewlink=true> accessed 20 June 2021. google scholar
  • Williams B et al., ‘How Algorithms Discriminate Based on Data they Lack: Challenges, Solutions, and Policy Implications’ (2018) 8 Journal of Information Policy, 78-115. google scholar
  • World Economic Forum, ‘Transforming Paradigms A Global AI in Financial Services Survey’ (2020) (Report) <http://www3.weforum.org/docs/WEF_AI_in_Financial_Services_Survey. pdf> accessed 20 June 2021. google scholar
  • Xenidis R, ‘Tuning EU equality law to algorithmic discrimination: Three pathways to resilience’ (2020) 27(6) Maastricht Journal of European and Comparative Law, 736-758. google scholar
  • Xenidis R and Senden L, ‘EU non-discrimination law in the era of artificial intelligence: Mapping the challenges of algorithmic discrimination’ in Ulf Bernitz et al. (eds.), General principles of EU law and the EU digital order (Kluwer Law International 2020), 151-182. google scholar
There are 121 citations in total.

Details

Primary Language English
Subjects Law in Context
Journal Section Makaleler
Authors

Salih Tayfun İnce This is me 0000-0002-0750-8152

Publication Date December 31, 2022
Submission Date August 6, 2021
Published in Issue Year 2022 Issue: 71

Cite

APA İnce, S. T. (2022). European Union Law and Mitigation of Artificial Intelligence-Related Discrimination Risks in the Private Sector: With Special Focus on the Proposed Artificial Intelligence Act. Annales De La Faculté De Droit d’Istanbul(71), 265-307. https://doi.org/10.26650/annales.2022.71.0002
AMA İnce ST. European Union Law and Mitigation of Artificial Intelligence-Related Discrimination Risks in the Private Sector: With Special Focus on the Proposed Artificial Intelligence Act. Annales de la Faculté de Droit d’Istanbul. December 2022;(71):265-307. doi:10.26650/annales.2022.71.0002
Chicago İnce, Salih Tayfun. “European Union Law and Mitigation of Artificial Intelligence-Related Discrimination Risks in the Private Sector: With Special Focus on the Proposed Artificial Intelligence Act”. Annales De La Faculté De Droit d’Istanbul, no. 71 (December 2022): 265-307. https://doi.org/10.26650/annales.2022.71.0002.
EndNote İnce ST (December 1, 2022) European Union Law and Mitigation of Artificial Intelligence-Related Discrimination Risks in the Private Sector: With Special Focus on the Proposed Artificial Intelligence Act. Annales de la Faculté de Droit d’Istanbul 71 265–307.
IEEE S. T. İnce, “European Union Law and Mitigation of Artificial Intelligence-Related Discrimination Risks in the Private Sector: With Special Focus on the Proposed Artificial Intelligence Act”, Annales de la Faculté de Droit d’Istanbul, no. 71, pp. 265–307, December 2022, doi: 10.26650/annales.2022.71.0002.
ISNAD İnce, Salih Tayfun. “European Union Law and Mitigation of Artificial Intelligence-Related Discrimination Risks in the Private Sector: With Special Focus on the Proposed Artificial Intelligence Act”. Annales de la Faculté de Droit d’Istanbul 71 (December 2022), 265-307. https://doi.org/10.26650/annales.2022.71.0002.
JAMA İnce ST. European Union Law and Mitigation of Artificial Intelligence-Related Discrimination Risks in the Private Sector: With Special Focus on the Proposed Artificial Intelligence Act. Annales de la Faculté de Droit d’Istanbul. 2022;:265–307.
MLA İnce, Salih Tayfun. “European Union Law and Mitigation of Artificial Intelligence-Related Discrimination Risks in the Private Sector: With Special Focus on the Proposed Artificial Intelligence Act”. Annales De La Faculté De Droit d’Istanbul, no. 71, 2022, pp. 265-07, doi:10.26650/annales.2022.71.0002.
Vancouver İnce ST. European Union Law and Mitigation of Artificial Intelligence-Related Discrimination Risks in the Private Sector: With Special Focus on the Proposed Artificial Intelligence Act. Annales de la Faculté de Droit d’Istanbul. 2022(71):265-307.