Research Article
BibTex RIS Cite

Avrupa Birliği Genel Veri Koruma Tüzüğü Kapsamında Gerçek Kişilerin Kişisel Sosyal Robot Kullanımından Doğabilecek Sorumlulukları

Year 2020, Volume: 2 Issue: 2, 19 - 29, 31.12.2020

Abstract

Yapay zekâ ve robotik gibi ileri teknolojilerin Makine Öğrenmesi aracılığıyla elde ettiği otonom öğrenme ve çalışabilme yeteneği sayesinde bu teknolojilerden en üst seviyede fayda sağlanabileceği düşünülmektedir. Yapay zekânın yaşam kaynağı olan verinin ve beyni olan algoritmaların vücut bulduğu robotlar, insansı tasarımlarının bir parçası olan sosyal etkileşim kurabilme yetenekleri sayesinde hem kamusal hem de kişisel yaşam alanlarında yerini alacaktır. Sosyal robotların kişisel kullanımıyla ilgili gerçek kişiler üzerindeki muhtemel sorumluluğun tartışıldığı bu makalede, senaryo yöntemiyle ortaya konulan bir durum üzerinde gerçek kişilerin rıza ve bilgilendirme sorumluluklarına ilişkin sorular sorulacak, bu sorular Avrupa Birliği’nin Genel Veri Koruma Tüzüğü ve Avrupa Adalet Divanı içtihadı kapsamında analiz edilecektir. Evlerinde çeşitli sebeplerle bir sosyal robot kullanan gerçek kişilerin ev içerisinde robotla etkileşimde bulunan diğer ilgili kişilerin rızasını alma ve onları bilgilendirme sorumluluğu bulunmakta; ancak bu durum robotun üreticileri gibi diğer veri sorumluları ve veri işleyenlerin asıl sorumluluğunu değiştirmemektedir.

References

  • Ahonen, P., Alahuhta, P., Daskala, B., Delaitre, S., de Hert, P., Lindner, R., Maghiros, I., Moscibroda, A., Schreurs, W., Verlinden, M. (2008). Dark scenarios. Wright D., Friedewald M., Punie Y., Gutwirth S., Vildjiounaite E. (ed.) Safeguards in a World of Ambient Intelligence. The International Library of Ethics, Law and Technology, vol 1. Springer, Dordrecht, 33-142.
  • Article 29 Working Party (2011). 15/2011 Opinion on the definition of consent adopted on 13 July 2011.
  • Article 29 Working Party (2016). Guidelines on consent under Regulation 2016/679.
  • Arzt S. (2019). AI Learns to Park-Deep Reinforcement Learning <https://www.youtube.com/watch?v=VMp6pq6_QjI> adresinden 10.05.2020 tarihinde alınmıştır.
  • Astor, M. (2017). ‘Your Roomba May Be Mapping Your Home, Collecting Data That Could Be Shared’, The New York Times <https://www.nytimes.com/2017/07/25/technology/roomba-irobot-data-privacy.html> adresinden 09.05.2020 tarihinde alınmıştır.
  • Avrupa Komisyonu. (2018). Commission Staff Working Document Impact Assessment Accompanying the document Proposal for a Regulation of the European Parliament and of the Council establishing the Digital Europe programme for the period 2021-2027, SWD(2018) 305 final. <https://eur-lex.europa.eu/legal-content/EN/TXT/HTML/?uri=CELEX:52018SC0305&from=EN> adresinden 09.05.2020 tarihinde alınmıştır.
  • Avrupa Komisyonu. (2020). White Paper on Artificial Intelligence-A European approach to excellence and trust. COM(2020) 65 final. <https://ec.europa.eu/info/sites/info/files/commission-white-paper-artificial-intelligence-feb2020_en.pdf> adresinden 09.05.2020 tarihinde alınmıştır.
  • Avrupa Parlamentosu. (2015). European Parliament resolution of 16 February 2017 with recommendations to the Commission on Civil Law Rules on Robotics procedure, 2015/2103(INL). <https://www.europarl.europa.eu/doceo/document/TA-8-2017-0051_EN.html?redirect> adresinden 09.05.2020 tarihinde alınmıştır.
  • Barfield, W. (2018). Liability for Autonomous and Artificially Intelligent Robots, Paladyn Journal of Behavioral Robotics, 9(1), 193-203.
  • Report of COMEST on Robotics Ethics (2017). United Nations World Commission on the Ethics of Scientific Knowledge and Technology. <https://unesdoc.unesco.org/ark:/48223/pf0000253952> adresinden 10.05.2020 tarihinde alınmıştır.
  • Breazeal, C. (2002). Designing Sociable Robots. Cambridge, MA, USA: MIT Press.
  • Broman, M. M., Finckenberg-Broman, P. (2017). Human-Robotics&AI interaction: The Robotics/AI legal entity (RAiLE©), 2017 IEEE International Symposium on Technology and Society (ISTAS) içinde, 1–7.
  • CloudMinds (2020). XR-1 Intelligent Service Robot. <https://www.en.cloudminds.com/home-new/cloud-robots/xr-1/> adresine 09.05.2020 tarihinde erişilmiştir.
  • CARESSES (2020). Culture-Aware Robots and Environmental Sensor Systems for Elderly Support <http://caressesrobot.org/en/> adresine 09.05.2020 tarihinde erişilmiştir.
  • Custers B.H.M., Ursic H. (2016). Big data and data reuse: a taxonomy of data reuse for balancing big data benefits and personal data protection, International Data Privacy Law 6(1): 4-15.
  • Çetin, S. (Ed.). (2019). Yapay Zekâ Çağında Hukuk: İstanbul, Ankara ve İzmir Baroları Çalıştay Raporu. <https://www.istanbulbarosu.org.tr/files/docs/Yapay_Zekâ_Caginda_Hukuk2019.pdf> adresinden 12.05.2020 tarihinde alınmıştır.
  • Darling, K. (2017). “Who’s Johnny?” Anthropomorphic Framing in Human–Robot Interaction, Integration, and Policy. In Robot Ethics 2.0: From Autonomous Cars to Artificial Intelligence, Oxford University Press.
  • Edwards, C., Edwards, A., Spence, P. R., Xialing, L. (2018). I, teacher: using artificial intelligence (AI) and social robots in communication and instruction, Communication Education. Routledge, 67(4), 473–480.
  • European Data Protection Board. (2020). Guidelines 05/2020 on consent under Regulation 2016/679 adopted on 4 May 2020.<https://edpb.europa.eu/sites/edpb/files/files/file1/edpb_guidelines_202005_consent_en.pdf> adresinden 10.05.2020 tarihinde alınmıştır.
  • Fong, T., Nourbakhsh, I., Dautenhahn, K. (2003). A survey of socially interactive robots, Robotics and Autonomous Systems, 42, 143–166.
  • Glenn, J. C., Theodore, J. G. (2009). Futures Research Methodology Version 3.0, The Millennium Project; 3.0 edition.
  • Francis, Jr. et. al. (2015). U.S. Patent No. 8996429B1, USPTO, United States.
  • Gültekin Várkonyi, G. (2019). Operability of the GDPR’s Consent Rule in Intelligent Systems: Evaluating the Transparency Rule and the Right to Be Forgotten, Intelligent Environments, Andrés Muñoz, Sofia Ouhbi, Wolfgang Minker, Loubna Echabbi, Miguel Navarro-Cía (ed.), IOS Press.
  • Hallevy, G., (2010). The Criminal Liabiliıty of Artificial Intelligence Entities-From Science Fiction to Legal Social Control. Akron Intellectual Property Journal, 4(2).
  • International Federation of Robotics (2017). Executive Summary World Robotics 2017 Industrial Robots, <https://ifr.org/downloads/press/Executive_Summary_WR_2017_Industrial_Robots.pdf> adresinden 09.05.2020 tarihinde alınmıştır.
  • International Federation of Robotics (2019). Executive Summary World Robotics 2019: Service Robots. <https://ifr.org/downloads/press2018/Executive_Summary_WR_Service_Robots_2019.pdf> adresinden 09.05.2020 tarihinde alınmıştır.
  • ISO 8373:2012(en). ‘Robots and robotic devices -Vocabulary’.
  • Istituto Italiano di Tecnologia (2018). IIT 2018-2023 Technical Annex. <https://multimedia.iit.it/asset-bank/assetfile/11121.pdf> adresinden 11.05.2020 tarihinde alınmıştır.
  • Leroux, C., Labruto, R., Boscarato, C., Caroleo, F., Günther, J.P., Löffler, S., Münch, F., Beck, S., May, E., Huebert-Saintot, C., de Cock Buning, M., Belder, L., de Bruin, R., Bonarini, A., Matteucci, M., Salvini, P., Schafer, B., Santosuosso, A., Hilgendorf, E. (2012). Suggestion for a green paper on legal issues in robotics. euRobotics, The European Robotics Coordination Action, 7th Framework Programme.
  • Leyzberg, D., Ramachandran, A., Scassellati, B. (2018). The Effect of Personalization in Longer-Term Robot Tutoring, ACM Transactions on Human-Robot Interaction, Article no. 19, 7 (3), 19.
  • Lo Scalzo, F. (2020). Tommy the robot nurse helps keep Italy doctors safe from coronavirus, Reuters Technology News <https://www.reuters.com/article/us-health-coronavirus-italy-robots/tommy-the-robot-nurse-helps-keep-italy-doctors-safe-from-coronavirus-idUSKBN21J67Y> adresinden 09.05.2020 tarihinde alınmıştır.
  • Millar, J., Kerr, I. (2016). Delegation, relinquishment, and responsibility: The prospect of expert robots. Robot Law, Cheltenham, Edward Elgar Publishing, UK.
  • Minkkinen, M. (2015). Futures of privacy protection: A framework for creating scenarios of institutional change, Futures, 73, 48–60.
  • MIT Technology Review (2020). ‘Facebook has trained an AI to navigate without needing a map’. <https://www.technologyreview.com/2020/01/22/276139/facebook-has-trained-an-ai-to-navigate-without-needing-a-map/> adresinden 10.05.2020 tarihinde alınmıştır.
  • Mori, M. (1970). The Uncanny Valley, Energy, vol. 7, no. 4, 33–35 (Japonca). Eserin İngilizce tercümesi <https://spectrum.ieee.org/automaton/robotics/humanoids/the-uncanny-valley> adresinden 11.05.2020 tarihinde alınmıştır.
  • Mulligan, C. (2018). Revenge against Robots, 69 S. C. L. Rev. 579.
  • Shadana IMC (t.y.) Robotics in the Netherlands, report prepared for the State Agency for Enterprising <https://www.araneo-magna.nl/images/pdfs/Robotics-in-the-Netherlandsfinal.pdf> adresinden 10.05.2020 tarihinde alınmıştır.
  • Social Cognitive Robotics in the European Society (SOCRATES) (2020). <http://www.socrates-project.eu adresine 11.05.2020 tarihinde erişilmiştir.
  • X Company (2020). <https://x.company/projects/everyday-robots> adresine 09.05.2020 tarihinde erişilmiştir.
  • Zornoza, A., Moreno, J.C., Guzmán, J.L., Rodríguez, F., Sánchez-Hermosilla, J. (2017). Robots Liability: A Use Case and a Potential Solution. Robotics—Legal, Ethical and Socioeconomic Impacts içinde, InTech, London.

Possible Responsibilities of the Natural Persons Using Personal Household Social Robots under the General Data Protection Regulation

Year 2020, Volume: 2 Issue: 2, 19 - 29, 31.12.2020

Abstract

It has been claimed that advanced technologies, such as artificial intelligence and robotic technologies, will be at the maximum benefit of humans due to the Machine Learning methods enabling these technologies learn and work autonomously. The data, as the blood of AI, and the algorithm, as the brain of AI, embodied in robotic bodies will take place both in personal and professional lives, thanks to their ability to socially interact with human as part of their anthropomorphized design. This paper uses the scenario method to be able to analyze the responsibilities on natural persons using personal household social robots in frame of the General Data Protection Regulation and the case law of the European Court of Justice. The paper gives more weight on the questions of natural persons’ possible duties on obtaining consent and informing obligations. Natural persons using a personal household social robot in their home for several reasons indeed have a responsibility to obtain consent from and fulfill their informing obligations towards other persons who interact with the robot, however, this does anyhow not change the actual responsibility of the data controllers, such as the manufacturers of the robot, and/or data processors.

References

  • Ahonen, P., Alahuhta, P., Daskala, B., Delaitre, S., de Hert, P., Lindner, R., Maghiros, I., Moscibroda, A., Schreurs, W., Verlinden, M. (2008). Dark scenarios. Wright D., Friedewald M., Punie Y., Gutwirth S., Vildjiounaite E. (ed.) Safeguards in a World of Ambient Intelligence. The International Library of Ethics, Law and Technology, vol 1. Springer, Dordrecht, 33-142.
  • Article 29 Working Party (2011). 15/2011 Opinion on the definition of consent adopted on 13 July 2011.
  • Article 29 Working Party (2016). Guidelines on consent under Regulation 2016/679.
  • Arzt S. (2019). AI Learns to Park-Deep Reinforcement Learning <https://www.youtube.com/watch?v=VMp6pq6_QjI> adresinden 10.05.2020 tarihinde alınmıştır.
  • Astor, M. (2017). ‘Your Roomba May Be Mapping Your Home, Collecting Data That Could Be Shared’, The New York Times <https://www.nytimes.com/2017/07/25/technology/roomba-irobot-data-privacy.html> adresinden 09.05.2020 tarihinde alınmıştır.
  • Avrupa Komisyonu. (2018). Commission Staff Working Document Impact Assessment Accompanying the document Proposal for a Regulation of the European Parliament and of the Council establishing the Digital Europe programme for the period 2021-2027, SWD(2018) 305 final. <https://eur-lex.europa.eu/legal-content/EN/TXT/HTML/?uri=CELEX:52018SC0305&from=EN> adresinden 09.05.2020 tarihinde alınmıştır.
  • Avrupa Komisyonu. (2020). White Paper on Artificial Intelligence-A European approach to excellence and trust. COM(2020) 65 final. <https://ec.europa.eu/info/sites/info/files/commission-white-paper-artificial-intelligence-feb2020_en.pdf> adresinden 09.05.2020 tarihinde alınmıştır.
  • Avrupa Parlamentosu. (2015). European Parliament resolution of 16 February 2017 with recommendations to the Commission on Civil Law Rules on Robotics procedure, 2015/2103(INL). <https://www.europarl.europa.eu/doceo/document/TA-8-2017-0051_EN.html?redirect> adresinden 09.05.2020 tarihinde alınmıştır.
  • Barfield, W. (2018). Liability for Autonomous and Artificially Intelligent Robots, Paladyn Journal of Behavioral Robotics, 9(1), 193-203.
  • Report of COMEST on Robotics Ethics (2017). United Nations World Commission on the Ethics of Scientific Knowledge and Technology. <https://unesdoc.unesco.org/ark:/48223/pf0000253952> adresinden 10.05.2020 tarihinde alınmıştır.
  • Breazeal, C. (2002). Designing Sociable Robots. Cambridge, MA, USA: MIT Press.
  • Broman, M. M., Finckenberg-Broman, P. (2017). Human-Robotics&AI interaction: The Robotics/AI legal entity (RAiLE©), 2017 IEEE International Symposium on Technology and Society (ISTAS) içinde, 1–7.
  • CloudMinds (2020). XR-1 Intelligent Service Robot. <https://www.en.cloudminds.com/home-new/cloud-robots/xr-1/> adresine 09.05.2020 tarihinde erişilmiştir.
  • CARESSES (2020). Culture-Aware Robots and Environmental Sensor Systems for Elderly Support <http://caressesrobot.org/en/> adresine 09.05.2020 tarihinde erişilmiştir.
  • Custers B.H.M., Ursic H. (2016). Big data and data reuse: a taxonomy of data reuse for balancing big data benefits and personal data protection, International Data Privacy Law 6(1): 4-15.
  • Çetin, S. (Ed.). (2019). Yapay Zekâ Çağında Hukuk: İstanbul, Ankara ve İzmir Baroları Çalıştay Raporu. <https://www.istanbulbarosu.org.tr/files/docs/Yapay_Zekâ_Caginda_Hukuk2019.pdf> adresinden 12.05.2020 tarihinde alınmıştır.
  • Darling, K. (2017). “Who’s Johnny?” Anthropomorphic Framing in Human–Robot Interaction, Integration, and Policy. In Robot Ethics 2.0: From Autonomous Cars to Artificial Intelligence, Oxford University Press.
  • Edwards, C., Edwards, A., Spence, P. R., Xialing, L. (2018). I, teacher: using artificial intelligence (AI) and social robots in communication and instruction, Communication Education. Routledge, 67(4), 473–480.
  • European Data Protection Board. (2020). Guidelines 05/2020 on consent under Regulation 2016/679 adopted on 4 May 2020.<https://edpb.europa.eu/sites/edpb/files/files/file1/edpb_guidelines_202005_consent_en.pdf> adresinden 10.05.2020 tarihinde alınmıştır.
  • Fong, T., Nourbakhsh, I., Dautenhahn, K. (2003). A survey of socially interactive robots, Robotics and Autonomous Systems, 42, 143–166.
  • Glenn, J. C., Theodore, J. G. (2009). Futures Research Methodology Version 3.0, The Millennium Project; 3.0 edition.
  • Francis, Jr. et. al. (2015). U.S. Patent No. 8996429B1, USPTO, United States.
  • Gültekin Várkonyi, G. (2019). Operability of the GDPR’s Consent Rule in Intelligent Systems: Evaluating the Transparency Rule and the Right to Be Forgotten, Intelligent Environments, Andrés Muñoz, Sofia Ouhbi, Wolfgang Minker, Loubna Echabbi, Miguel Navarro-Cía (ed.), IOS Press.
  • Hallevy, G., (2010). The Criminal Liabiliıty of Artificial Intelligence Entities-From Science Fiction to Legal Social Control. Akron Intellectual Property Journal, 4(2).
  • International Federation of Robotics (2017). Executive Summary World Robotics 2017 Industrial Robots, <https://ifr.org/downloads/press/Executive_Summary_WR_2017_Industrial_Robots.pdf> adresinden 09.05.2020 tarihinde alınmıştır.
  • International Federation of Robotics (2019). Executive Summary World Robotics 2019: Service Robots. <https://ifr.org/downloads/press2018/Executive_Summary_WR_Service_Robots_2019.pdf> adresinden 09.05.2020 tarihinde alınmıştır.
  • ISO 8373:2012(en). ‘Robots and robotic devices -Vocabulary’.
  • Istituto Italiano di Tecnologia (2018). IIT 2018-2023 Technical Annex. <https://multimedia.iit.it/asset-bank/assetfile/11121.pdf> adresinden 11.05.2020 tarihinde alınmıştır.
  • Leroux, C., Labruto, R., Boscarato, C., Caroleo, F., Günther, J.P., Löffler, S., Münch, F., Beck, S., May, E., Huebert-Saintot, C., de Cock Buning, M., Belder, L., de Bruin, R., Bonarini, A., Matteucci, M., Salvini, P., Schafer, B., Santosuosso, A., Hilgendorf, E. (2012). Suggestion for a green paper on legal issues in robotics. euRobotics, The European Robotics Coordination Action, 7th Framework Programme.
  • Leyzberg, D., Ramachandran, A., Scassellati, B. (2018). The Effect of Personalization in Longer-Term Robot Tutoring, ACM Transactions on Human-Robot Interaction, Article no. 19, 7 (3), 19.
  • Lo Scalzo, F. (2020). Tommy the robot nurse helps keep Italy doctors safe from coronavirus, Reuters Technology News <https://www.reuters.com/article/us-health-coronavirus-italy-robots/tommy-the-robot-nurse-helps-keep-italy-doctors-safe-from-coronavirus-idUSKBN21J67Y> adresinden 09.05.2020 tarihinde alınmıştır.
  • Millar, J., Kerr, I. (2016). Delegation, relinquishment, and responsibility: The prospect of expert robots. Robot Law, Cheltenham, Edward Elgar Publishing, UK.
  • Minkkinen, M. (2015). Futures of privacy protection: A framework for creating scenarios of institutional change, Futures, 73, 48–60.
  • MIT Technology Review (2020). ‘Facebook has trained an AI to navigate without needing a map’. <https://www.technologyreview.com/2020/01/22/276139/facebook-has-trained-an-ai-to-navigate-without-needing-a-map/> adresinden 10.05.2020 tarihinde alınmıştır.
  • Mori, M. (1970). The Uncanny Valley, Energy, vol. 7, no. 4, 33–35 (Japonca). Eserin İngilizce tercümesi <https://spectrum.ieee.org/automaton/robotics/humanoids/the-uncanny-valley> adresinden 11.05.2020 tarihinde alınmıştır.
  • Mulligan, C. (2018). Revenge against Robots, 69 S. C. L. Rev. 579.
  • Shadana IMC (t.y.) Robotics in the Netherlands, report prepared for the State Agency for Enterprising <https://www.araneo-magna.nl/images/pdfs/Robotics-in-the-Netherlandsfinal.pdf> adresinden 10.05.2020 tarihinde alınmıştır.
  • Social Cognitive Robotics in the European Society (SOCRATES) (2020). <http://www.socrates-project.eu adresine 11.05.2020 tarihinde erişilmiştir.
  • X Company (2020). <https://x.company/projects/everyday-robots> adresine 09.05.2020 tarihinde erişilmiştir.
  • Zornoza, A., Moreno, J.C., Guzmán, J.L., Rodríguez, F., Sánchez-Hermosilla, J. (2017). Robots Liability: A Use Case and a Potential Solution. Robotics—Legal, Ethical and Socioeconomic Impacts içinde, InTech, London.
There are 40 citations in total.

Details

Primary Language Turkish
Subjects Law in Context
Journal Section Articles
Authors

Gizem Gültekin Várkonyi 0000-0001-9586-9179

Publication Date December 31, 2020
Submission Date May 12, 2020
Acceptance Date December 15, 2020
Published in Issue Year 2020 Volume: 2 Issue: 2

Cite

APA Gültekin Várkonyi, G. (2020). Avrupa Birliği Genel Veri Koruma Tüzüğü Kapsamında Gerçek Kişilerin Kişisel Sosyal Robot Kullanımından Doğabilecek Sorumlulukları. Kişisel Verileri Koruma Dergisi, 2(2), 19-29.
AMA Gültekin Várkonyi G. Avrupa Birliği Genel Veri Koruma Tüzüğü Kapsamında Gerçek Kişilerin Kişisel Sosyal Robot Kullanımından Doğabilecek Sorumlulukları. Kişisel Verileri Koruma Dergisi. December 2020;2(2):19-29.
Chicago Gültekin Várkonyi, Gizem. “Avrupa Birliği Genel Veri Koruma Tüzüğü Kapsamında Gerçek Kişilerin Kişisel Sosyal Robot Kullanımından Doğabilecek Sorumlulukları”. Kişisel Verileri Koruma Dergisi 2, no. 2 (December 2020): 19-29.
EndNote Gültekin Várkonyi G (December 1, 2020) Avrupa Birliği Genel Veri Koruma Tüzüğü Kapsamında Gerçek Kişilerin Kişisel Sosyal Robot Kullanımından Doğabilecek Sorumlulukları. Kişisel Verileri Koruma Dergisi 2 2 19–29.
IEEE G. Gültekin Várkonyi, “Avrupa Birliği Genel Veri Koruma Tüzüğü Kapsamında Gerçek Kişilerin Kişisel Sosyal Robot Kullanımından Doğabilecek Sorumlulukları”, Kişisel Verileri Koruma Dergisi, vol. 2, no. 2, pp. 19–29, 2020.
ISNAD Gültekin Várkonyi, Gizem. “Avrupa Birliği Genel Veri Koruma Tüzüğü Kapsamında Gerçek Kişilerin Kişisel Sosyal Robot Kullanımından Doğabilecek Sorumlulukları”. Kişisel Verileri Koruma Dergisi 2/2 (December 2020), 19-29.
JAMA Gültekin Várkonyi G. Avrupa Birliği Genel Veri Koruma Tüzüğü Kapsamında Gerçek Kişilerin Kişisel Sosyal Robot Kullanımından Doğabilecek Sorumlulukları. Kişisel Verileri Koruma Dergisi. 2020;2:19–29.
MLA Gültekin Várkonyi, Gizem. “Avrupa Birliği Genel Veri Koruma Tüzüğü Kapsamında Gerçek Kişilerin Kişisel Sosyal Robot Kullanımından Doğabilecek Sorumlulukları”. Kişisel Verileri Koruma Dergisi, vol. 2, no. 2, 2020, pp. 19-29.
Vancouver Gültekin Várkonyi G. Avrupa Birliği Genel Veri Koruma Tüzüğü Kapsamında Gerçek Kişilerin Kişisel Sosyal Robot Kullanımından Doğabilecek Sorumlulukları. Kişisel Verileri Koruma Dergisi. 2020;2(2):19-2.