{"id":672,"date":"2020-03-12T10:36:08","date_gmt":"2020-03-12T10:36:08","guid":{"rendered":"https:\/\/aastaraamat.riigikohus.ee\/?p=672"},"modified":"2020-05-20T15:54:26","modified_gmt":"2020-05-20T15:54:26","slug":"a-kratt-as-an-administrative-body-algorithmic-decisions-and-principles-of-administrative-law","status":"publish","type":"post","link":"https:\/\/aastaraamat.riigikohus.ee\/en\/a-kratt-as-an-administrative-body-algorithmic-decisions-and-principles-of-administrative-law\/","title":{"rendered":"A Kratt as an Administrative Body: Algorithmic Decisions and Principles of Administrative Law*"},"content":{"rendered":"

Ivo Pilving
\n<\/strong>Chairman of the Administrative Law Chamber of the Supreme Court, University of Tartu Associate Professor in Administrative Law<\/em><\/p>\n

Monika Mikiver
\n<\/strong>Adviser in the Public Law Division of the Ministry of Justice, Doctoral student at the University of Tartu Faculty of Law<\/em><\/p>\n

 <\/p>\n

Artificial intelligence applications used in the private sector \u201efourth industrial revolution\u201c are now increasingly finding their way into public sector offices. Estonia also has ambitions to use robots or kratts<\/em>**<\/a>, more widely in public administration to support or replace officials.[1]<\/a> Public sector kratts<\/em> have received rather cursory academic attention,[2]<\/a> even though the legislator has granted in some fields (tax administration, environmental fees, unemployment insurance) authorization for automated administrative decisions already in 2019.[3]<\/a> There are plans to present Riigikogu<\/em> with a bill by June of 2020 that, if adopted, would introduce the necessary changes into existing legislation, including the Administrative Procedure Act, to allow for wider use of artificial intelligence.[4]<\/a><\/p>\n

There has been talk in Estonian media about automating pension payments and unemployment registration.[5]<\/a> It is worth noting that not all e-government solutions are based on artificial intelligence and many are just simpler, automated forms of data processing. True, according to the principles of administrative law, if such human-guided solutions malfunction, they may also be transgressing against the law. The real challenge, however, is legislating artificial intelligence, especially self-learning algorithms. With sloppy or malicious implementation, kratts<\/em> may easily defy the rules of fair procedure, break the law or treat people and businesses arbitrarily. Robots cannot explain their decisions yet.<\/p>\n

In order for our kratt<\/em> project to succeed without exposing society or businesses to grave risks, the development of e-government must understand the nature of machine learning as well as its impact on administrative and judicial procedures.[6]<\/a> Acting rashly in this field is tantamount to exercising governmental authority according to a horoscope. It would be na\u00efve and dangerous to let ourselves get overcome by the illusion of a kratt that can now or will soon be able to engage in reasoned debate or comprehend the content of human language, including legal texts. Proper implementation of the law requires both rationality as well as a true understanding of the law. If risks are perceived and considered and algorithms are used for proper operations, there will be plenty of work for them in public administration and they can be beneficial to both efficiency and quality in decision making.<\/p>\n

To keep focus in this article, we will not broach questions of data protection,[7]<\/a> even though there is important commonality here when it comes to administrative law. We will consider algorithms regardless of whether decisions are made about humans or legal entities and whether these are based on personal or other data. The breadth of this article doesn\u2019t allow us to go in depth into the issue of equality. This article aims to give the reader some examples of the use of robots in public administration (1), attempts to explain the technical nature of algorithmic administrative decisions (2) and finally examines the operation of the principles of Estonian administrative law in this type of decision making (3). The main question posed by this article is if and when can a kratt<\/em> be taught to read and follow the law as the legitimacy of governmental authority cannot be sacrificed to progress.<\/p>\n

1. Algorithms in Public Administration<\/h3>\n

Artificial intelligence enthusiasts both in Estonia and abroad have pointed out that the implementation of algorithms affords wide opportunities for cost savings, productivity gains and freeing officials of routine assignments.[8]<\/a> An increasingly powerful fleet of computers and ever more intelligent software can handle crazy amounts of data and solve assignments that are too complicated for humans. The public sector has to keep up with the private sector. Among other applications, algorithms may become necessary in public administration to effectively control the use of artificial intelligence in business, such as in automated transactions on the stock exchange.[9]<\/a><\/p>\n

Smart public administration systems can be classified into the following categories: communication with people,[10]<\/a> internal activities,[11]<\/a> preparation of decisions and decision making. In the framework of this article, we are primarily interested in the latter two. In Estonia, for example, the Agricultural Registers and Information Board uses algorithms to analyse satellite imagery to check for compliance with grassland mowing obligations. The Tallinn City Government uses machine vision to measure traffic flows. The Ministry of the Interior wants to automate surveillance with a nationwide network of face and number recognition cameras. The Unemployment Insurance Fund soon hopes to implement artificial intelligence to assess the risk of unemployment.[12]<\/a><\/p>\n

In the U.S., an algorithm determines family benefits and analyses the risks that may justify separating a child from his or her family.[13]<\/a> They have also applied algorithms to bar entry into the United States, to approve pre-trial bail, to grant parole, in counter-terrorism, planning inspection visits to restaurants, etc. There are predictions that artificial intelligence will soon be implemented in the fields of airplane pilot licensing, tax refund assessment or the assignment of detainees to prisons.[14]<\/a> Algorithmic predictive policing is used in both the U.S. as well as Germany to predict the time, place and perpetrator of an offense. The system analyses crime statistics together with camera and drone surveillance records to identify occurrence patterns for certain offenses and uses the information gleaned to direct operational forces. Disputes over the use of algorithms have already reached the highest courts in European countries, such as the French and Dutch Councils of State, regarding such issues as university applications and environmental permits.[15]<\/a><\/p>\n

2. Technical Background<\/h3>\n

2.1. Basic Concepts<\/h4>\n

In order to understand artificial intelligence, we must first unveil some concepts from data science with definitions that are far from unanimous.[16]<\/a> Nevertheless, we will try to give one potential overview.<\/p>\n

Artificial intelligence<\/strong> can be understood as the ability of a computer system to perform tasks commonly associated with the human mind, such as understanding and observing information, communicating, discussing and learning. These features of artificial intelligence must be considered metaphors in the functional sense, because machine \u201clearning\u201d is not actually the same as human learning. Artificial intelligence has many branches: automated decision support, speech recognition and synthesis, image recognition and so on. A robot in our context is an artificial intelligence application \u2013 an intelligent system.[17]<\/a><\/p>\n

Data mining<\/strong> is the process of extracting new knowledge – generalization, data correlation and repeating patterns – from large amounts of data (big data) using statistical methods.[18]<\/a> Various statistical methods have previously allowed analysts to build mathematical models<\/strong> based on data sets to describe what is happening in nature or society. These can, in turn, help assess and classify new situations and predict the future, such as the weather or criminal recidivism. This becomes particularly effective if the models are built on self-learning (machine learning) algorithms.[19]<\/a><\/p>\n

An algorithm<\/strong> is an exact set of mathematical or logical instructions, more generally a step-by-step procedure for solving a given problem (e.g. including a cake recipe). The representation of an algorithm in programming language is a computer program. (1) Algorithms where the performance is entirely human-defined are distinguished from (2) algorithms that change their parameters autonomously in the course of learning.[20]<\/a> The systems that automate the traditional decision making processes in public administration (expert systems) are based on the former. Artificial intelligence applications in public administration are mostly based on learning algorithms (sometimes also on more sophisticated non-learning algorithms).<\/p>\n

Machine learning<\/strong> is the process by which an artificial intelligence system improves its service by acquiring or reorganizing new knowledge or skills. It is characterized by using the help of learning algorithms to assess situations or make predictions (e.g. making diagnoses, detecting credit card fraud, predicting crime). There are many machine learning techniques with different characteristics: linear and logistic regression, decision tree, decision forest, artificial neural networks, etc.[21]<\/a> In the most widespread \u2013supervised learning \u2013 the algorithm is first trained using training data, a large amount of data cases where the input (e.g. payment behaviour data) and output (e.g. solvency) values (features) are known.<\/p>\n

At a later stage, the application must calculate the output values of new cases based on input data on its own. These can be presented as numerical data (regression) or, for example, as yes\/no answers (classification). The core element of a learning algorithm is its optimizing or objective function. This is the mathematical expression of the algorithm\u2019s task, which contains a set of so-called weight parameters.[22]<\/a> As it learns, the robot looks for possible combinations of weights and chooses the working model that is most appropriate for the future and the one that gives solutions that deviate the least from the relationship given in the training data. These operations are repeated hundreds, thousands or even millions of times.[23]<\/a><\/p>\n

By automated administrative decisions<\/strong> we mean any administrative decision that is prepared or made using automation. This can be based on simpler or more sophisticated non-learning algorithms (expert systems) as well as on machine learning.[24]<\/a> For example, land tax statements in Estonia are made entirely according to set rules and require no cleverness on the part of a computer.<\/p>\n

An algorithmic administrative decision<\/strong> is more narrowly a decision made with the help of artificial intelligence. Automated administrative decisions can be divided into fully and semi-automated ones. The latter are approved by an official. Sometimes the computer decides, based on certain criteria provided, whether it is able to make a final decision, such as granting a tax refund claim, or if an official must decide.[25]<\/a> Sometimes the terms automated and algorithmic decisions are used synonymously or often combined. But it must be taken into account that the learning potential of artificial intelligence brings both new opportunity but also problems to public administration.[26]<\/a><\/p>\n

2.2. The Basic Characteristics of Machine Learning<\/h4>\n

Self-learning algorithms can handle trillions of data cases, each with tens of thousands of variables. For some time, institutions in Estonian have been collecting data into large data warehouses for analytical purposes.[27]<\/a> Machine learning doesn\u2019t change the main essence of data analysis, but amplifies it: machine learning (in its current capacity) is only able to discover statistical correlations<\/strong>. These are not causal, natural or legal relationships. Depending on the level of refinement of the model, the output data of machine learning may reflect the real world and anticipate the future with amazing accuracy. However, probability calculations will always retain some rate of error.[28]<\/a><\/p>\n

Learning algorithms and models created during learning are so sizeable and complex that a human \u2013 even an experienced computer scientist or the creator of the algorithm \u2013 may not always be able to observe or explain the work of a machine learning application (opacity<\/strong> or the black-box effect). The more efficient the algorithm, the opaquer it is. Individual elements of a sophisticated machine learning system can be tracked, such as individual trees in a decision forest. But this does not allow much to be inferred about the process as a whole. Sometimes the opacity of a system is actually desirable in order to protect personal data or business secrets or to prevent the addressee of a decision from deceiving the algorithm.[29]<\/a><\/p>\n

Because of the statistical nature of machine learning, very big amounts of data<\/strong> are needed. Unfortunately, or rather, fortunately, there is too little information on terrorist acts, for example, to make accurate estimates.[30]<\/a> In addition to the quantity of data, high quality<\/strong> and standardization<\/strong> are no less important: accuracy, relevance, organization, compatibility, comprehensiveness, impartiality and, above all, security. This applies to both the training data as well as the operating data used in the actual implementation of the algorithm. All machine learning predictions are based on training data and previous experience. Another golden rule of machine learning is: garbage in, garbage out. Poor data quality can result in a variety of distortions, including a failure to investigate all of the factors affecting assessment because of inability, not considering this important or finding it economically nonessential.[31]<\/a> At the same time, large numbers of decisions amplify the impact of an error rate on an absolute scale.<\/p>\n

Models developed and decisions made through machine learning cannot be completely foreseen or guided.[32]<\/a> Nonetheless, people \u2013 programmers, analysts, data scientists, system developers and, ultimately, the end user \u2013 have a huge role and responsibility in the quality of machine learning outcomes. The end result is influenced by all kinds of strategic decisions and fine-tuning: defining the relevant output value (target features),[33]<\/a> creating an objective function, selecting and developing the type of algorithm, fine-tuning the algorithm to be more cautious or bolder, testing and auditing. We must take into account that two different types of algorithms, both of which might be very precise on their own, may give completely different answers to the same case.[34]<\/a><\/p>\n

3. Rule of Robots or Smart Rule of Law?<\/h3>\n

The abovementioned technicalities of machine learning pose significant legal challenges in public administration. Machine learning can produce great results statistically, but in certain cases, a lot can also go wrong.<\/p>\n

3.1. Administrative Risks: Digital Delegation and Privatization<\/h4>\n

The authority of the government can only be exercised by a competent institution. This institution may use automatic devises, such as a traffic light or computer, for this purpose. The more discretion given to the algorithm, the more acute becomes the question of whether the decision is actually subject to the control of the competent institution or whether it is running its own course.[35]<\/a> In our view, the decision is always formally attributed to the institution using the algorithm and they remain legally responsible for it. But with larger decisions to be made, a substantive problem actually arises: can the institution make the algorithm sufficiently consider all the important details of a decision?[36]<\/a><\/p>\n

We can assume that the state will have a practical need to delegate the development of its algorithms largely to privately held IT companies. That makes it important that we not lose democratic control over the companies directly managing the algorithm, such as making sure they don\u2019t gain full control over the content of administrative decisions or maximize their profits at the expense of the quality of administrative decisions. Therefore, as we develop our e-government, we have to analyse whether the current public procurement and administrative cooperation laws sufficiently address these risks.[37]<\/a><\/p>\n

3.2. Impartiality<\/h4>\n

For decades, people have been hoping that artificial intelligence can help create a bias-free, selfless, comfort zone-free decision maker that treats everyone equally. Unfortunately, the reality of machine learning has shown some serious difficulties with the problem of bias. Artificial intelligence tends to discriminate against some groups of people when the quality of input data or the algorithm itself is inadequate. For example, when some groups have been previously monitored more closely than others, it becomes reflected in the training data (e.g. blacks in predictive policing in the U.S. or in recidivism assessment systems).[38]<\/a><\/p>\n

3.3. Legal Basis<\/h4>\n

The Constitution \u00a7 3 (1) 1 states in Estonia that governmental authority can solely be exercised pursuant to the law. The question of when and how the legislature should authorize institutions to implement machine learning technology cannot be answered simply or unilaterally. If machine learning is used only in the preparation of administrative decisions (e.g. to forecast a pollutant before issuing an environmental permit) but the final administrative decision is made by a human official following normal procedural rules, then machine learning can be considered one detail of the administrative procedure and control over the decision making remains at the discretion of the administrative institution (Administrative Procedure Act \u00a7\u00a05 (1)) thereby not requiring any special provisions.[39]<\/a><\/p>\n

If the role of the human in decision making is limited to that of a rubber stamp or disappears altogether, then it may be a matter requiring parliamentary approval. In each area (licenses, social benefits, environmental protection, law enforcement, immigration, etc.), the widespread implementation of intelligent systems raises specific issues that need to be resolved separately and balanced with appropriate substantive and procedural guarantees.[40]<\/a> Aside from the legal issues, it would be wise to consider the risks to public finance: does it make the legislature a slave to the robot? An expensive and complicated implementation system may start to obstruct legislative changes and political will.[41]<\/a><\/p>\n

The Taxation Act \u00a7\u00a0462<\/sup> (1) grants an implementing institution broad powers to make automatic administrative decisions in the field of taxation without intervention by an official. A more detailed list must be established by the Ministry of Finance.[42]<\/a> The law does not impose restrictions on the type or manner of decisions that can be automated. Due to its rather precise legal definitions, taxation is considered rather suited for automation. Here, a well-founded reliance on a broad mandate shouldn\u2019t produce unacceptable results. However, granting total power of authority[43]<\/a> to fully automate any administrative decision may result in violations of \u00a7\u00a03 (1) and \u00a7\u00a014 of the Constitution.<\/p>\n

3.4. Supremacy of the Law<\/h4>\n

Pursuant to \u00a7\u00a03 (1) 1 of the Constitution, the exercise of governmental authority may be guided by an algorithm only if the word of the law is followed at all times during its application.[44]<\/a> But this requires the human or self-learning system to convert the law into an algorithm. In some cases this may be possible in principle, though it would be a substantial task, but that would require the developer to have very in-depth knowledge of information technology, mathematics and the law.[45]<\/a> However, many legal provisions cannot be described in the unambiguous variables specific to an algorithm.[46]<\/a> This is due both to the inevitable vagueness of the instrument of law \u2013 the human language \u2013 as well as the intentional slack that ensures flexibility in legislation.[47]<\/a><\/p>\n

Instead of step-by-step instructions (conditional programs), the law often uses outcome-orientedfinite<\/span> programs:[48]<\/a> general objectives<\/strong> such as improving living environments, public involvement and informing the public, balancing and integrating interests, sufficiency of information, expedient, reasonable and economic land use (Planning Act \u00a7\u00a7 8\u201312); discretionary powers<\/strong>, such as the right of a law enforcement agency to issue a precept to a person liable for public order to counter a threat or eliminate a disturbance (Law Enforcement Act \u00a7\u00a028); undefined legal terms<\/strong>, such as overriding public interest (Water Act \u00a7\u00a0192 (2)) or danger (Law Enforcement Act \u00a7\u00a05 (2)); general <\/strong>principles, such as human dignity, proportionality, equal treatment (Constitution \u00a7\u00a010, \u00a7\u00a011 sentence 2 and \u00a7\u00a012).<\/p>\n

Due to the uncertainty of the law, legal subsumptions[49]<\/a> (such as the decisions necessary to implement a law \u2013 is an object a building in the sense of the Building Act, is a person a contracting entity in the sense of the Public Procurement Act, whether the recipient of rural support is sustainable, or how to define a goods market in competition supervision) are not mere formal logical acts but require judgement. Before a situation is resolved, the decision maker must interpret the norm to explain whether the legislator wanted to subject the situation to the norm or not. What\u2019s more, the decision must be made in situations that didn\u2019t occur to the legislature, such as a new border tax avoidance scheme. Here, the implementer needs to assess whether he or she is dealing with permissible optimization or abuse (Taxation Act \u00a7\u00a084). Those implementing the law \u2013 the ministers, officials, judges and contracting parties \u2013 continue to interpret it and fill in the gaps in the regulatory process started by parliament. It is up to them to make the law concrete.[50]<\/a><\/p>\n

We must note that there is some similarity to machine learning here: a learning algorithm is not yet complete in the form that humans created it. It keeps developing itself and is able to create new models to classify situations. So, couldn\u2019t the legislator\u2019s real will not be modelled in this way a well? Is it not a standard classification task for a smart system almost like finding cat pictures? Unfortunately, the source material for machine learning \u2013 data from the past \u2013 cannot in principle be sufficient to further develop the law as code.[51]<\/a> A law\u2019s enforcer must also account for existing judicial and administrative practice[52]<\/a> and the generalizations that crystallize out of it but his or her sources must not be limited only to this.[53]<\/a><\/p>\n

An official or a judge must be able to perceive, understand and apply a much broader context: the history of the law, the systematics of norms, the objective of this law and the general meaning of justice, but especially the direct and indirect effects of the decision. It is not possible in all fields to produce sufficient quantitative or qualitative data to describe all the layers of law and its operating environment. And it is far from possible for (current) smart systems to follow all of this material in real time. Therefore, many situations require a rational being who understands the peculiarities of the specific situations being regulated and, when necessary, creates a new law appropriate for that situation instead of searching for it in previously tested patterns.[54]<\/a><\/p>\n

The vagueness of legal concepts expressed in natural language is not a flaw in the law. It must remain possible to argue over the law in order to reach fair decisions in specific situations. But this requires open and honest discussion over different interpretations and ways of assessing the facts. Even if you translate the law into zeros and ones, you don\u2019t escape the need to interpret it. This need would simply move from the decision making stage to (1) the expert system creation and calibration stage or (2) the intelligent system learning stage.[55]<\/a> In both cases, at least the persons concerned and presumably also the competent administrative institution (considering the complexity of machine learning) lack an effective opportunity to have a say in the interpretation. Because of their complexity, the decisions made by a self-learning algorithm are not just difficult to predict, they are structurally unpredictable.[56]<\/a><\/p>\n

But how can you ensure that the algorithm won\u2019t deviate from the law as it learns? Periodic testing and auditing is not a sufficient solution because tests are also unable to anticipate or run through all of life\u2019s possible scenarios. The costs of such extensive calibration and testing would eventually outweigh the benefits.[57]<\/a> Also, the machine language translation of a law that an administrative robot could supposedly follow is anything but static. It is corrected not only by new laws, interpretations in case law[58]<\/a> and decisions made in constitutional review, but also by the development of the context of the law \u2013 society. Weak artificial intelligence is not capable of perceiving or applying these changes itself.<\/p>\n

The main question here is not whether and to what extent a machine makes mistakes. A machine doesn\u2019t perform any legal thought operations. In the best case and only with sufficient amounts of data, machine learning (in its current capacity) can merely mimic legal decisions through statistical operations but not comprehend the content of the law or make rational decisions based on it.[59]<\/a> But that is precisely the demand set by\u00a0\u00a7 3 (1) 1 of the Constitution. We are not claiming that an expert or smart system isn\u2019t able to replace any legal assessment, just that solutions to the above described problems must be found when using such systems.<\/p>\n

3.5. Discretionary Power<\/h4>\n

These problems are exacerbated by discretionary decisions where the law does not prescribe clear instructions, such as whether to require the demolition of a building, what requirements to set for service providers in a procurement, whether and under what conditions to allow extraction in an area with groundwater problems or where to build a landfill. Ostensibly, discretionary power does not give authorities the right to make arbitrary decisions. Discretionary decisions must also obey the general principles of justice, the purpose of the law and all of the relevant facts specific to each individual case (Administrative Procedure Act \u00a7\u00a04 (2)).[60]<\/a><\/p>\n

An algorithm that has been completely defined by humans is not suited to make discretionary decisions because circumstances are unpredictable. True, there is some measure of standardization and generalization in making judgements, such as with internal administrative rules, but officials must retain the right and the duty to deviate from such standards when it comes to atypical cases.[61]<\/a> However, optimists believe that even though the capability is lacking at the moment, it is not rigid algorithms but machine learning that will be able to take advantage of its dynamic discretionary parameters to soon work within value principles and discretionary bounds.[62]<\/a><\/p>\n

This does not seem realistic in the near future.[63]<\/a> First of all, decisions of this kind are too unique to be able to create large enough amounts of data for machine learning to be capable of modelling them. Secondly, discretionary rules and the general principles of justice may seem like simple maxims at first glance. They may even be represented as mathematical formulas, but this does not yet guarantee their practical applicability to machine learning. Let us illustrate with R. Alexy\u2019s proportionality formula by trying to explain its application through the example of an injunction to shut down a fish factory infected with a dangerous bacterium.<\/p>\n

\"\"<\/a><\/p>\n

Here, i <\/em>and j<\/em> are the principles considered in making the decision (in this case, fundamental rights: consumer health versus the freedom to conduct business). Wi,j<\/sub> <\/em>is the specific value of principle i<\/em> in relation to principle j<\/em>. In order for the Veterinary and Food Board to issue an injunction, i<\/em>, or health, must outweigh j<\/em>, or freedom to conduct business. In other words, Wi,j<\/sub><\/em> must be >1. I<\/em>\u00a0is the intensity of interference of the give principle, which expresses the extent of potential damage if one or the other principle recedes (consumer illness or death, factory bankruptcy and unemployment for its many workers). W<\/em> is the principle\u2019s abstract value and illustrates the general importance we attach to public health and freedom to conduct business. R <\/em>is the probability of damage that could result from violation of this principle (for example, if the factory stays open, their product will not necessarily be contaminated but may be, but if the factory is shut down, bankruptcy is certain).[64]<\/a><\/p>\n

Even if we are to disregard other criticism of this equation,[65]<\/a> the real difficulty does not lie in calculations but in ascribing correct values to the variables in the equation and in arguments over if and to what extent one or another principle (fundamental right) is infringed, what are the proven facts and are they even relevant to the judgement to be made. The likelihood of one or another outcome (R<\/em>) may depend on very special circumstances that did not occur to those inputting the algorithm\u2019s learning and working data. However, judgements made on the importance of principles (W<\/em>) and the intensity of interference (I<\/em>) are value-based and can only be made with an acute perception of the sizeable context accumulated over a long period of evolution in law and society. If this information is not easily accessible to the human official, the deficiency can be overcome by communication between the decision maker and the parties to the proceeding in an inclusive administrative proceeding (3.6 below).<\/p>\n

The weak machine learning technologies available today and in the near future are characterized by their limitation in understanding context and the content of communication.[66]<\/a> This applies to both undefined legal concepts (public interest, material harm, etc.).[67]<\/a> If, for example, the Law Enforcement Act \u00a7\u00a05 (2) defines a threat as a sufficiently probable offense (e.g. food poisoning), it still does not quantify the level of sufficient probability. This is a legal judgement that is based on value judgements on the significance of one or another amenity, not just a statistical prognosis of the occurrence of damage.<\/p>\n

To fully delegate a complex discretionary or judgement-based decision to an algorithm would, in our opinion, constitute a gross error of discretion against the Code of Administrative Court Procedure \u00a7\u00a0158 (3) 1 (failure of an administrative institution to exercise discretionary power). An algorithm can, however, be implemented as an aid.<\/p>\n

3.6. Fair Proceedings and the Principle of Investigation<\/h4>\n

Fair proceedings \u2013 especially the right to a hearing (Administrative Procedure \u00a7\u00a040 (1))\u00a0\u2013 play an important role in guaranteeing the substance of a decision as well as the dignity of the persons concerned.[68]<\/a> The establishment and further development of law in a state based on the rule of law must take place in the framework of honest and open (at least to the persons concerned) dialogue. In decisions affecting large numbers of people, such as spatial planning and environmental permits, the right to express an opinion must also be open to the public. Such discussions can merely be mimicked by contemporary algorithms (i.e. debate robots), not actually (meaningfully) held. In the case of machine learning, listening to the discussion would be all the more necessary as the algorithm may not be programmed or have learned to account for unpredictable circumstances. A rare event may turn out to be decisive to the right prediction, like a broken leg means a person won\u2019t do his weekly work-out even if his years of behavioural patterns would indicate otherwise.[69]<\/a><\/p>\n

Coglianes and D. Lehr point out that the right to a hearing is fairly flexible under U.S. law and that machine learning without a hearing could, in some situations, make more accurate decisions on average than humans through the hearing process.[70]<\/a> This is not adequate justification. A citizen or business that falls into the margin of error does not have to be allayed by pretty statistics and retains the right to demand a lawful decision on his case. The Administrative Procedure Act \u00a7\u00a040 (3) in Estonia provides several exceptions to the right to a hearing. These exceptions can be augmented with special laws if the real effectiveness of a hearing is low. But no general exception to any administrative acts on algorithms can be granted. The larger the discretion of the authority, the more necessary communication becomes to the proceedings and, therefore, the less possibility there is to use fully automated decisions, i.e. when applying artificial intelligence, the person concerned must retain the opportunity to interact with an official.[71]<\/a><\/p>\n

The effective protection of rights and public interest is guided by the principle of investigation in the Administrative Procedure Act (\u00a7\u00a06)\u00a0\u2013 an administrative institution is obligated to take initiative in investigating all relevant facts. This is also a challenge for algorithms because they cannot deal with circumstances that haven\u2019t been entered into their systems. The reality around us is not yet completely digitized or machine-readable with sensors. Therefore, a machine can only consider fragments of the actual situation in its analysis.[72]<\/a> But a human is able to take initiative in searching for additional data from sources that have not been provided or are not in their manual.<\/p>\n

Not knowing important information does not exonerate the decision maker from erring against a prohibitive norm.[73]<\/a> This is why German law requires the intervention of a human officer, obliging him or her to manually correct an automated decision in the light of additional circumstances.[74]<\/a> However, with intelligent implementation, machine learning can be applied to the principle of investigation, such as to select tax returns that need more extensive, manual control.[75]<\/a><\/p>\n

3.7. Reasoning<\/h4>\n

The reasoning behind decisions made by governmental authorities is a core element of a fair procedure. According to \u00a7\u00a056 of the Administrative Procedure Act, an administrative act must state its legal and factual basis (the provision delegating authority and the circumstances justifying its application) and, if the act is based on discretion, at least the primary motives for choosing between different options (e.g. why the pulp mill should be in Narva and not Tartu or why the construction of a wind farm should be prohibited).<\/p>\n

This is not a mere ethical recommendation but a fundamental, constitutional obligation.[76]<\/a> A law enforcement mandate can only be granted to an entity that can demonstrate that their decision is lawful \u2013 in accordance with external limits as well as internal rules of discretion. This brings up an important difference between the private and public sectors: the use of freedoms does not need to be justified but the use of authority does. A person receiving a notice of tax assessment or demolition injunction does not have to accept an official\u2019s claim that \u201cI don\u2019t know why, but the machine made this decision about you.\u201d<\/p>\n

Neither the Administrative Procedure Act \u00a7 56 nor the Taxation Act \u00a7 462<\/sup> makes exemptions for automatic, including algorithmic, administrative decisions. Such exceptions would violate the Constitution as well as generally accepted standards in democratic, rule of law states.[77]<\/a> As we have seen, creators of algorithms are often unable to explain the decisions made by a robot due to machine learning opacity. In 2011, Houston in the U.S. used an algorithmic decision making process to terminate employment contracts with teachers. During the ensuing litigation, the school administrator was unable to explain the functioning of the algorithm, claiming that he had no ownership or control over the technology.[78]<\/a> The U.S. Government has also cited the issue of opacity as a matter of concern.[79]<\/a><\/p>\n

Some experts see the possibility of solving the problem of opacity by using artificial intelligence to develop language processing programs enough so that the computer can analyse numerous prior justifications to synthesize a machine argument that is seemingly similar legal arguments.[80]<\/a> This method would still use only statistics, not reasoning based on the methods of jurisprudence, meaning it could only offer an inadmissible semblance. But reasoning must be genuine.[81]<\/a><\/p>\n

There is a growing search for ways to increase transparency in machine learning based on the principles of accountability and explainability. Among other things, this requires greater access to learning and source data, data processing, algorithms and their learning processes.[82]<\/a> These challenges collide with business secrets, internal information, personal data protection and, above all, a human\u2019s ability to analyse the work of an algorithm. Moreover, the most important aspect of the reasoning for an administrative act is not its technical description of how<\/strong> the decision was made, but the motives behind why<\/strong> the decision made was this particular one. Even in the case of decisions made by humans, we are not interested in the biochemical details of the decision maker\u2019s brain but in his or her explanations. There is little benefit to expanding the overall transparency of machine learning to the reasoning of individual cases.[83]<\/a><\/p>\n

As an alternative, development has started on so-called explainable artificial intelligence (xAI<\/em>). Since the actual mathematical processes of machine learning are too complex and sizeable for humans to productively investigate them directly, developers are trying to employ artificial intelligence for this task as well, such as trying to model complex implementation using a simpler and more comprehensible algorithm.[84]<\/a> Also, there are methods that try to construct similar fictitious situations where the algorithm gives a different answer. To do this, some variables are ignored or changed (e.g. gender or age) while others are kept constant. This technique can be used to parse out the criteria instrumental to a decision.[85]<\/a> But this still only takes us half way: it is an explanation of the background of a statistical judgement, not a legal judgement itself. [86]<\/a> That may suffice if the administrative act is solving a complex but mathematically solvable problem, such as predicting development or the likelihood of an event (e.g. the increase or decrease in the population of a protected species when a railway is built).[87]<\/a><\/p>\n

Such judgements and predictions can be necessary, but the obligation of justification has a wider berth. In general, an administrative act may need an explanation of why legal provision x<\/em> is applied and not y<\/em>; why a statutory provision is interpreted as a<\/em> and not b<\/em>; what facts have been ascertained and why; why these facts are pertinent according to the relevant law;[88]<\/a> why it is necessary to implement a certain measure (e.g. why should a dangerous structure be demolished instead of rebuilt). All of these issues need counter-arguments to the positions held by the parties to the proceeding that were not addressed in the decision. A robot cannot give adequate explanations for these thought operations because it does not perform such operations.[89]<\/a> If an administrative act requires a substantive legal justification, then the current level of information technology means we need to input a human into the \u201ccircuit\u201d.[90]<\/a><\/p>\n

3.8. Judicial Review<\/h4>\n

To ensure legality, it is recommendable to subject both private and public sector artificial intelligence applications to multifaceted monitoring (documentation, auditing, certification, standardization).[91]<\/a> This is necessary but cannot replace the judicial protection of persons who find that their rights may have been violated (Constitution \u00a715 (1), European Convention on Human Rights Art.\u00a06 and 13, Charter of Fundamental Rights of the European Union Art.\u00a047). If sufficient substantive and factual argumentation is given for administrative decisions made using an algorithm, there is no fundamental problem with judicial control. But difficulties arise in the absence of such argumentation. [<\/a>91a<\/a>]<\/a><\/a><\/p>\n

Coglianes and D. Lehr point out that courts tend to give deference to agencies when it comes to technically complex issues.[92]<\/a> But it\u2019s the algorithm that makes the decision complex! Implementing algorithms, however, cannot become a universal magic wand that frees the executive institution from judicial review for any decision. We can only talk of loosening control in situations where judges would defer even without the use of artificial intelligence for other reasons, such as the economic, technical or medical complexity of the content or if the infringement on the rights of affected individuals is not excessively intense. Here, a complex administrative decision may include elements with different control intensity.[93]<\/a> E. Berman debates this matter. She sees the opportunities for use of algorithms to be proportional to the leeway in discretion. This debate is somewhat confusing because it does not account for the breadth of discretion afforded by the law, or the significant influence of general principles and basic rights. Her ultimate conclusion is that control may be allowed to weaken where infringements are not very grievous and regulation is sparse and it can disappear altogether in situations where no one\u2019s rights are affected (e.g. deciding where to locate police patrols).[94]<\/a> We can agree with this conclusion. As a general rule, administrative court proceedings retain control of rationality (including proportionality), which requires substantive, administrative decisions that are at least monitored by humans as well as legal justification with the exception of routine, mass decisions.<\/p>\n

The problem of control cannot be solved simply by the administrative institution disclosing the content and raw data of the algorithm to the court.[95]<\/a> To analyse this material, the court would need its own IT knowledge or expert assistance. This is not realistic considering reasonable procedural resources nor in accordance with the constitutional roles given to the branches of power (Constitution\u00a0\u00a7\u00a04). It would mean placing the primary responsibility for the compliance of an algorithm in the hands of the court where \u00a7\u00a03 and\u00a014 of the Constitution place it in the hands of the executive power.<\/p>\n

There is no need to turn algorithms into direct subjects of judicial control. It is not necessarily important whether the data is distorted or has calibration errors or bias \u2013 these deficiencies may not affect the end result. It is also not the responsibility of the appellant to prove such deficiencies when challenging an algorithmic decision. And it is not a reasonable solution to compensate for the complexity of the algorithm with longer appeal times. Algorithmic administrative decisions must also obtain final, conclusive force within a reasonable amount of time.[96]<\/a> In court, it is important for administrative decisions taken by an algorithm to be legally justifiable.<\/p>\n

The executive institution must be convinced of the legality of its decision and the process of forming such a conviction must be traceable without any special knowledge of computer science. This means using a so-called administrative Turing test, meaning that citizens and businesses must not detect any difference in whether a law enforcement decision by the executive institution is made with the help of artificial intelligence or not.[97]<\/a> An institution using algorithms can implement the help of the algorithm for making a decision if it is able. If not, a representative of reasonable thought \u2013 an official \u2013 must step in.<\/p>\n

4. Conclusion: The Division of Labour Between Kratt<\/em> and Master<\/h3>\n

Administrative decisions vary widely in terms of content, legal and factual framework and decision making process. Depending on the field and situation, rigid, standard solutions, generalizations and simplifications may be allowed to a greater or lesser extent in administrative law.[98]<\/a> There are quite a few routine decisions that are subject to clear rules (e.g. in the areas of social benefits and taxes) and those can be trusted to computers working with non-learning or learning algorithms.[99]<\/a> It may also make sense to use self-learning algorithms in areas where governmental decisions have a wide breadth and decision making requires a more non-judicial analysis (e.g. determining the locations of police patrols or modelling protected populations).[100]<\/a> But the important and complex decisions in society (e.g. where to build a railway or whether to build a nuclear power plant) are not routine and cannot be automated, at least not fully, because of a lack of appropriate learning data. These decisions need human judgement.[101]<\/a><\/p>\n

In situations that fall between those two extremes, it is realistic to expect cooperation between the robot and the official where the scope of each role may vary greatly depending on the field and situation:[102]<\/a><\/p>\n