How to develop responsible AI

Innovative solutions based on artificial intelligence (AI) and machine learning (ML) offer great potential in various applications - especially in the fields of health and sustainability. However, a responsible approach to the new technological possibilities is crucial.

At Zühlke, we believe that artificial intelligence (AI) will have a positive impact on our society and environment. Innovative AI solutions in medicine can improve patient outcomes by improving diagnosis and liberating healthcare professional’s time by automatizing of administration, allowing them to spend more time with their patients. In other fields, such as in sustainability, AI can also play a pivotal role in reducing energy consumption and CO2 emissions by optimizing transport networks and monitoring deforestation, just to name a few examples. It can be used to automatize tedious tasks in all industries and thus improving working conditions for employees across all sectors.

The honeymoon is over

AI and machine learning has been seen like a magical cure for everything, but we have started to see a backlash of some of the first large-scale implementations. We have heard about how Instagram’s algorithms have started recommending dieting content to young teenagers, how AI based hiring tools discriminate against women, how hundreds of AI based applications were built to detect covid – but few of them helped. The list can be made infinitely long.

Regulation does not have to impede innovation

Such failures can be avoided with the right approaches. Developing Responsible and Trustworthy AI is not only a good idea to prevent reputational damages and costs due to reduced revenue and lost customers. It will also soon become a legal requirement - already in April 2021, EU published their intentions to regulate AI on their official website. Although many fears that regulating AI will slow down innovation, our experiences made in medical AI, a field that has long been subject to regulations, clearly shows that such hurdles can be overcome with the right frameworks. Rather than being an impediment for innovation, they promote high quality solutions by providing checklists ensuring that Good Machine Learning Practice is being followed along the full development process and preventing common pitfalls for unintended future consequences. Although the new regulations have not yet been set, the first communications from the European Union indicate that they will follow a similar pattern as the medical regulations and will be centered around risk evaluation and risk mitigation.

The ethical AI framework

Responsible AI covers ethical aspects, requirements on interpretability and sustainability along the whole development chain - from the initial decision to start the development of an application, to the user’s interaction with the final product. Based on a long experience of developing machine learning applications within the medically regulated field, Zühlke has transferred and expanded our learnings to develop frameworks for both ethical and interpretable AI. The frameworks cover multiple layers of the development process - the data, the model, and the human interaction. These are hands-on frameworks that clearly indicate which considerations are needed at every phase of the implementation and distributes roles and responsibilities.

Just like for medically regulated products, the ethical framework is centered around an assessment of the potential risks and harm that could be inflicted by the application. Based on the level of risk, it is then decided which mitigations are necessary to proceed with the development. The framework ensures that developed AI applications are compliant and aligned with corporate values and reduces the risk of unintended consequences. 

The layers of the framework

The Interpretable AI framework

Working with Interpretable AI is a prerequisite to ensure the compliance of the final application. This framework is therefore an integrated part of the Ethical framework but can also be used on its own.

The human centric layer of the framework covers clarity towards the user on the added value of the application and clarifies the need of explanations. It creates a human interpretable explanation of the model behavior and ensures transparency toward the end user.

In the data layer, the focus is on the understanding of the origin of the data. So does the plausibility of the selected feature and analysis of the confounders. It is also about creating clarity on what data is needed to be provided by the user and the consequences of sharing those data.

The model track covers the need of transparent model versus explainable models. It also analyses the fidelity and consistency of the explanation.

How to implement the frameworks

In order to create the desired impact, it is important that the framework becomes an integrated part of the work culture. This is reached by implementing comprehensible and easy to follow processes within the organization. Having an infrastructure and culture that favors transparency, a diverse working force that encourages an open communication and continuous training of its employees are other factors paving the way to a successful implementation. Last, but not least, a proper documentation of all design decisions made in the development process are necessary to ensure transparency and that all potential pitfalls have been considered.