1. Create repeatable training workflows to accelerate model development

One important aspect of MLOps is creating repeatable training workflows that can accelerate model development.

In SageMaker, you can use several features to create a repeatable training workflow, including:

• SageMaker Experiments: SageMaker Experiments lets you track and compare the results of your machine learning experiments, enabling you to quickly identify the most effective models and hyperparameters.
• SageMaker Processing: SageMaker Processing lets you run pre-processing, postprocessing, and other data processing tasks on large datasets in a distributed and scalable way. This helps ensure that your data is consistently processed and prepared for training.
• SageMaker Training: SageMaker Training lets you train machine learning models on large datasets using distributed computing resources. You can use built-in algorithms or bring your own custom algorithms to SageMaker.
• SageMaker Debugger: SageMaker Debugger lets you identify and debug training errors in real time. You can monitor the state of your training job and capture specific events, such as null values or weights that are too large.

2. Catalogue ML artifacts centrally for model reproducibility and governance

Cataloguing ML artifacts centrally is an essential step towards achieving model reproducibility and governance. In machine learning (ML), artifacts are produced during the model training process, such as code, datasets, models, and configurations. Cataloguing them centrally means storing them in a centralized location where they can be easily accessed, tracked, and managed.

There are several benefits to cataloguing ML artifacts centrally such as Reproducibility, Governance, Collaboration, etc.

Amazon SageMaker provides several tools for cataloguing ML artifacts centrally, including:

• SageMaker Model Registry: A managed service that provides a central location for storing, versioning, and sharing ML models. With the Model Registry, data scientists can easily track changes to their models, compare different versions, and promote models to production.
• SageMaker Pipelines: A workflow automation tool that helps data scientists build, deploy, and manage ML workflows. Pipelines allow data scientists to define a series of steps for training and deploying models, and then execute those steps as a single unit.
• SageMaker Experiments: A service that helps data scientists track, organize, and analyse their ML experiments. With Experiments, data scientists can easily capture metadata about their experiments, including code, data, hyperparameters, and metrics.

3. Integrate ML workflows with CI/CD pipelines for faster time to production

Amazon SageMaker provides a set of tools and features that make it easy to integrate machine learning (ML) workflows with Continuous Integration and Continuous Deployment (CI/CD) pipelines, enabling faster time to production for ML models.

CI/CD pipelines are a set of practices and tools that enable developers to quickly and reliably build, test, and deploy code changes to production environments. By integrating ML workflows with CI/CD pipelines, data scientists can automate the process of building, testing, and deploying ML models, reducing the time and effort involved and increasing the speed at which models are delivered to production.

Here are some of the ways in which SageMaker enables the integration of ML workflows with CI/CD pipelines:

• SageMaker Model Building Pipelines: SageMaker Model Building Pipelines is a feature of SageMaker Pipelines that enables data scientists to create automated end-to-end workflows for building, training, and deploying ML models. Data scientists can use Model Building Pipelines to define a series of steps for the ML workflow, including data preparation, model training, evaluation, and deployment.
• SageMaker SDK: The SageMaker SDK is a Python library that makes it easy to interact with SageMaker services, including Pipelines and Model Building Pipelines. Data scientists can use the SDK to create and manage pipelines programmatically, enabling integration with existing CI/CD pipelines.
• SageMaker JumpStart: SageMaker JumpStart provides a collection of pre-built ML models, algorithms, and workflows that can be easily integrated with CI/CD pipelines. Data scientists can use JumpStart to accelerate the process of building and deploying ML models, reducing the time and effort required.

4. Continuously monitor data and models in production to maintain quality

In the field of machine learning, it's essential to continuously monitor the data and models in production to ensure that they maintain quality and performance over time. Monitoring data and models can help identify and address issues such as data drift, model decay, and bias, ensuring that the model continues to perform well and generate accurate predictions.

Amazon SageMaker provides several tools and features to help data scientists and machine learning engineers continuously monitor data and models in production, including:

• SageMaker Model Monitor: A managed service that continuously monitors the data and predictions generated by your ML models in production. Model Monitor detects and alerts you to data drift, concept drift, and quality issues, enabling you to take corrective actions to ensure that the model continues to generate accurate predictions.
• SageMaker Debugger: Debugger also provides real-time metrics and visualizations of model performance, enabling you to identify and address issues quickly.
• SageMaker Autopilot: A feature of SageMaker that automates the process of building, training, and deploying ML models. Autopilot provides automatic monitoring and retraining of models, ensuring that they remain up-to-date and continue to perform well in production.
• SageMaker Clarify: A tool that helps identify and mitigate bias in your ML models. Clarify provides metrics and visualizations that enable you to understand the sources of bias in your models, and it provides recommendations for addressing those issues.

MLOps Tools and Technologies

MLflow

MLflow is an open-source platform for managing the end-to-end machine learning lifecycle. It was developed by Databricks, the company behind Apache Spark, and is now a part of the Linux Foundation's AI Foundation.

MLflow is designed to help data scientists and machine learning engineers manage their machine learning workflows, from data preparation to model deployment. It provides a centralized platform for tracking experiments, packaging code, and sharing models.

Here are some of the key features of MLflow:

• Experiment tracking: MLflow provides a tracking API and UI for logging and visualizing machine learning experiments. This enables data scientists to keep track of different versions of their models, compare results, and reproduce previous experiments.
• Packaging code: MLflow enables data scientists to package their code and dependencies into reproducible environments, making it easier to share and deploy models.
• Model registry: MLflow provides a centralized repository for storing and versioning machine learning models. This enables data scientists to share and collaborate on models, and to deploy them to production environments.
• Deployment: MLflow provides integrations with popular deployment tools such as Docker, Kubernetes, and Amazon SageMaker, making it easy to deploy models to production.

In the above example, MLflow is used to track the parameters and performance metrics of a logistic regression model trained on a synthetic dataset. The mlflow.start_run() function is used to create a new MLflow run, and the mlflow.log_param() and mlflow.log_metric() functions are used to log the model parameters and performance metrics to the MLflow tracking server. The mlflow.sklearn.log_model() function is used to save the trained model to a file and log it to the MLflow tracking server.

Amazon SageMaker

Amazon SageMaker MLOps is a set of tools and best practices to help developers and data scientists to build, train, deploy, and manage machine learning models at scale. It is built on top of Amazon SageMaker, which is a fully managed service that provides developers and data scientists with the ability to build, train, and deploy machine learning models.

SageMaker MLOps provides a suite of tools for automating and managing the machine learning development lifecycle. Some of the key features of SageMaker MLOps include:

• Model training: SageMaker MLOps provides a managed service for training machine learning models at scale. It can automatically scale training resources to meet the demands of large datasets and complex models.
• Model deployment: SageMaker MLOps provides a set of tools for deploying machine learning models to production. It supports a range of deployment options, including batch and real-time inference, and it provides automatic scaling and monitoring of deployed models.
• Model monitoring: SageMaker MLOps includes tools for monitoring the performance of deployed models in production. It can track key metrics such as accuracy and latency, and it can alert developers when performance issues arise.
• Model management: SageMaker MLOps provides a centralized repository for storing and versioning machine learning models. It can track changes to models over time, and it can provide a history of changes for auditing and compliance purposes.

This code example demonstrates how you can use SageMaker to train, deploy, and manage machine learning models while incorporating MLOps practices such as data versioning, hyperparameter tuning, model versioning, and automatic model deployment. You can customize this example to suit your specific use case and data requirement.

Overall, SageMaker MLOps can help organizations to streamline the machine learning development process, reduce time to market, and improve the quality and reliability of deployed models.

Deployment Strategies[i]

Choose a deployment strategy MLOps deployment strategies include blue/green, canary, shadow, and A/B testing.

Blue/green

Blue/green deployments are very common in software development. In this mode, two systems are kept running during development: blue is the old environment (in this case, the model that is being replaced) and green is the newly released model that is going to production. Changes can easily be rolled back with minimum downtime, because the old system is kept alive.

Canary

Canary deployments are like blue/green deployments in that both keep two models running together. However, in canary deployments, the new model is rolled out to users incrementally, until all traffic eventually shifts over to the new model.

Shadow

You can use shadow deployments to safely bring a model to production. In this mode, the new model works alongside an older model or business process and performs inferences without influencing any decisions. This mode can be useful as a final check or higher fidelity experiment before you promote the model to production. Shadow mode is useful when you don't need any user inference feedback. You can assess the quality of predictions by performing error analysis and comparing the new model with the old model, and you can monitor the output distribution to verify that it is as expected.

A/B testing

When ML practitioners develop models in their environments, the metrics that they optimize for are often proxies to the business metrics that really matter. This makes it difficult to tell for certain if a new model will improve business outcomes, such as revenue and clickthrough rate, and reduce the number of user complaints.

Conclusion

In conclusion, MLOps is a critical methodology for organizations looking to scale their machine learning workloads. By combining best practices from DevOps with machine learning, MLOps enables organizations to automate and streamline the entire machine learning lifecycle, from development to deployment to continuous improvement.

— Author: Yasir Ul Hadi

Reference

[1] https://docs.aws.amazon.com/pdfs/prescriptive-guidance/latest/ml-operations-planning/ml- operations-planning.pdf

[1] https://aws.amazon.com/sagemaker/mlops/?sagemaker-data-wrangler-whats-new.sort-by=item.additionalFields.postDateTime&sagemaker-data-wrangler-whats-new.sort-order=desc

Businesses are embracing app modernization on a vast scale. The reason can be to meet greenfield necessities, make business future-ready, or to upgrade monolithic legacy applications. On their journey to modernization, businesses are using containers and Kubernetes as primary technologies to modernize the design and distribution of their applications. The key business goal remains the same, which is to have an all-time-available work system in place. A system that is scalable, portable, flexible, and reliable. Architecture based on microservices and Kubernetes, and the 12 factor app methodology can help achieve such a system.

The 12-factor app style of development surfaced about 10 years ago, much before containers. And, since then the 12 principles of the 12 factor app have become a universal standard for cloud-native app development. The 12 factor app development stages offer a set of guidelines for a proper outline for developing modern microservices. And, Kubernetes is known for being an orchestration platform for containers used to deploy and control these microservices.

The 12-factor app principles:

• Has only one aim: to offer a course of action for cloud-native application development and deployment. They ensure that happens by making applications highly scalable and disposable.
• Help you and your team to embrace DevOps and microservices in the app development process.
• Simplify the process, which increases the development time and reduces the time to market.
• Were designed to build Software as a Service (SaaS) applications by alleviating the difficulties associated with long-term software development.

This article explains how organizations are leveraging the 12-factor app development method and Kubernetes to architect cloud-native apps. Understand how 12 factor app is helping businesses to modernize by establishing scalability, resiliency, robustness, mobility, and reliability across their applications. Let’s get started.

 Leveraging 12 Factor App Principles and Kubernetes

1. A single Codebase for Applications, Multiple Deployments

A 12-factor app methodology states that only one Codebase or a set of code repositories should exist. These are deployable multiple times but never have many codebases. If there are any shared codes, they should be factored into libraries and called through the dependency manager.

Multiple deployments of a codebase are possible by making it active across all instances. The only difference is the versions, which are also tracked in the version manager.

Once the code base is in place with the 12-factor app approach, it can be built, released, or run in separate phases in the Kubernetes environment. Kubernetes and containers have text-based representations. The predictable system states are managed by automation tools in separate files. It is better to manage such evolving artifacts with source control. Using a version control system such as Git can help eliminate the introduction of sudden changes and facilitate tracking the changes added to your system.

2. Declare and Isolate Application Dependencies

The 12-factor app methodology uses the declaration and isolation method for application dependencies. Declare any dependencies explicitly and also check them in the version manager. This approach makes it easier to get started and enhances repeatability. It also becomes easy to track any changes made to the dependencies.

Another approach is to package the app and all its dependencies into a container. This makes it possible to remove the app and all its dependencies from its environment. In addition, it ensures that the app functions as expected regardless of the differences in development and staging environments.

3. Archive Config as Environment Variable

As per 12 factor app principle configs should be archived as environment variables (env vars) but not constants. Env vars are easy to change as the need arises for new code deployments without changing the codes. This flexibility quickens the native-app development process.

Additionally, you can manage env vars independently every time you deploy them. It also becomes easy to scale up as the development process progresses towards completion and deployment.

The 12-factor app strictly separates the application configuration from the code. Kubernetes ConfigMap supports storing configuration by declaring it. This can be helpful for production and development environments that need different configurations to deploy the same code.

4. Backing Services: As Attached Resources, Easy to Swap

Backing services include support applications and systems that your application needs to connect and communicate with, such as databases. They are usually grouped as attached resources that should be accessible when needed.

Modern applications that are microservices-based use backing services. These backing services are handled as attached resources in the 12-factor app. Due to this, in case of any failure, you can simply change the attached resources and not the whole application codebase.

Backing services in the 12-factor app methodology are configurable and easy to change. You can change them from one state to the next as the need arises. The switching is possible by just slightly changing the configuration.

It is the best practice to separate the backing services (such as logging, messaging, databases, third-party services, caching, and others) from the system. And then interact with them through an API. Sticking the APIs to consistent contracts will let you change the basic implementations without highlighting them to clients. Kubernetes ConfigMaps can be used to store connection information to skip building the container image again, in case of any revision in the connection information.

5.  Split Build, Release and Run Phases

The 12-factor app methodology distinguishes all the stages of cloud-native app development.

• They changed the codebase to deploy it. And once the first stage is completed and the next starts, you cannot alter the code in the previous one.
• You should build deployable components independent of the environment in the first stage. The second stage involves taking the reusable components already developed and combining them with a specific configuration to match the target environment.
• The last phase is the run stage. It involves packing the entity created in the previous one in a container and running it in the target environment.

Organizations prefer to automate the development and testing tasks with CI/CD toolchains. Splitting your CI/CD pipeline into a series of sequential tasks can increase productivity. It helps to provide greater insight into failure and improve accountability. For example, dedicate a pipeline exclusively for building a container image at a time. After that, to run the container instance, you can perform the testing, promoting, and deploying of that image.

6. Stateless Processes

The 12-factor app methodology allows you to run cloud-native applications in the environment as one or more processes. The only restrictions are that they should be stateless and never share data. That enhances scalability and portability across cloud computing infrastructure. Data compilation is done during the build stage. Any other thing that requires persistence forms part of the backing services.

Containers are short-lived and when the container goes away, the data inside the container ceases to exist. The state of containerized workloads must be reduced. This helps to maintain a good user experience while remaining unaffected by application scaling.

7. Port Binding to Export Services

This stage of 12-factor app development involves binding your packaged application to a port. You can use the Kubernetes service object if the workload is exposed internally to the cluster. Otherwise, you need other methods such as node ports, Ingres Controllers, and OpenShift routes.

Packing your application inside containers makes networking and port collisions easier by reducing the workload on hosts. Software-defined networks in Kubernetes platforms take over many operations.

8. Concurrency and Scalability

Scalability is one of the primary features of any cloud-native application. That is usually done by deploying more app copies instead of enlarging them. To achieve this, the 12 factor app methodology uses a simple yet reliable operation.

The developer designs the app to take on different workloads by assigning processes varying tasks. An example is an application where a web process handles HTTP requests and a worker processes a long-running background activity.

The pod-based Kubernetes architecture supports the scaling of application components as per varying demands. With the 12-factor app’s stateless processes element, scalability becomes a consistent function that can help to gain an expected level of concurrency.

9. Disposability: Robust Cloud-native apps

According to the 12-factor app methodology, all processes are disposable. They should have minimal startup time, shutdown gracefully, and be immune to crashes and failures. All these capabilities make scaling easier, enhance faster app development, and make the deployments more robust.

The app should create new instances when it needs to and take them down as necessary. It is this 'disposability' property that makes cloud-native applications more robust. In microservices, processes are disposable. That means, in case any application stops working unexpectedly, users stay almost unaffected and the failures are managed gracefully. You can also use Kubernetes ReplicaSets to uphold the stage of availability for microservices by specifying the max-to-min bounds for the number of replicas.

10. Dev/prod parity: Carrying out Development and Production Similarly

The 12-factor app methodology bridges the gap between cloud-native app development and production. That makes it possible to continuously deploy or roll out new features. Also, developers can write code, deploy, and review the app’s features. This process is usually fast, and completed in minutes or hours.

For organizations that pack workloads into containers, initializing the container image in one environment, it shall run on any infrastructure or environment. But there are chances of environmental drift. Consider standardizing the same distribution of Kubernetes across all environments to eliminate this. It helps create a consistent experience for container platform users.

11. Logs are Event Streams

The 12-factor app methodology does not require routing and storage, writing, or managing of the application’s output or log files. Any running process writes its event stream to STDOUT without buffering. A developer views this stream in the foreground of their user interface. This helps to determine how the app behaves and draw conclusions. The event streams also make it easy to troubleshoot or debug an application.

12. Run Admin Processes as One-off Processes

With a 12-factor app methodology, admin and management tasks such as database migrations, scripts, and batch programs run as one-off processes. They are treated as long-running processes. They also utilize the same dependency isolation methods as the app’s usual processes.

It is best practice to isolate the application administrative tasks such as data restore and backup, caching, or migration from the application microservices and carry them out as separate processes. You can use Kubernetes jobs to execute these mundane administrative tasks that are part of the application lifecycle.

What are the Business Benefits of 12 Factor App Methodology

The 12-factor app methodology is the how-to guide for creating cloud-native applications. Many giant tech companies, such as Amazon, Heroku, Microsoft, and others, make use of these 12 principles as they technically help them to enhance business agility by expediting innovation and go-to-market capabilities. With these 12-factor principles, you can design and maintain a robust and modern app architecture required for cloud-based applications.

This methodology is the solution for developers developing the following:

With the 12-factor apps methodology, you can create cloud-native applications that are:

• Suitable for deployment on modern cloud platforms, minimizing the need for servers and server administration
• Enabled for continuous deployment with minimal differences between development and production
• Scalable without significant change or effort
• Capable of using declarative formats for setup automation.

Conclusion

Web applications, platforms, and frameworks using the 12-factor app methodology have generated measurable business outcomes with enhanced productivity in the past few years. This guidebook is suitable for DevOps and cloud app development, which should be a blueprint for developing resilient, scalable, portable, and maintainable applications. Considering these 12-factor app principles with Kubernetes ensures you build a robust solution for your business.

However, this methodology is not the ultimate solution for everyone. Whether or not it works for your business depends on your business model and needs. So, you should not worry if your software development process deviates from the principles of 12-factor app methodology. You are good to go if you understand the reason and the expected outcome.

Do you want to learn more about 12-factor app development principles and its real-time use cases? Contact us today to know how we can help your company.

Book 1-hour free consultation

These algorithms need to be trained using labelled data, which can be difficult to obtain due to the small number of fraudulent transactions that actually happen compared to legitimate ones. However, techniques such as oversampling and undersampling can be used to address this issue today by balancing the number of fraudulent and legitimate transactions in the training data.

For example, in case of credit card frauds, the fraudsters come together and creates multiple bank accounts (often spanning different time and geographies) to make them look like genuine accounts. Traditional ML approaches fail to uncover the network of fraudsters hiding among the genuine accounts. Often the data containing flagged transactions are not exhaustive(since the suspicious account makes a few genuine transactions before they start) and hence these models that are trained in those data are usually unsuccessful in discovering coordinated attacks as in case of credit card frauds which uses multiple accounts.

In the following section, let’s explore how graph database and graph neural networks help address some key issues pointed out in the above sections in the context of credit card frauds—or any case involving more than one perpetrator.

Graph Neural Networks (GNN)

In the classical ML approach where we train predictive algorithms like decision trees, random forest or XGBoost, we typically store the transactions data in tabular format with columns as features. However, in the financial realm, transactions can be efficiently stored as graph databases where each node represents accounts and each edge represents transactions. The node will contain features associated with that account (ie location, data, etc). This representation of the existing transactional data helps the stakeholder understand different properties linked to a fraudulent account.

Figure: Graphs makes it easy to understand different connection between the data (Source: neo4j.com)

This helps us with analysis and predictions at different levels:

• Node Classification: the task here is to determine the labelling of samples (represented as nodes) by looking at the labels of their neighbors. Account level predictions are not very popular in the traditional ML approach since in most of the cases, we predict if the transaction is fraud or not.
• Link Predictions: Link represents transactions or any activity between nodes. A simple use case could be detecting suspicious transactions from genuine accounts could indicate theft or illegal account take overs.
• Community classifications: Within the entire network of transactions and accounts, it is now possible to uncover clusters with strong similarities. These would help the model to predict and classify accounts vulnerable to attacks or find group of illegal accounts.
• Anomaly Detection: In a collection of nodes, we find outliers in the graph in an unsupervised manner (data with no labels).
•Graph classification: the task here is to classify the whole graph into different categories. The applications of graph classification are numerous and range from determining whether a protein is an enzyme or not in bioinformatics, to categorizing documents in NLP, or social network analysis.

Application of GNN in Industry Use Cases

GNN-based models, such as RGCN, can benefit from topological information by combining network structure and node and edge attributes to build a meaningful representation that separates fraudulent from legitimate transactions. By heterogeneous graph embedding, RGCN can efficiently learn to represent many kinds of nodes and edges (relations).

• Loan Default Risk: For commercial banks and financial regularity institutions, monitoring and assessing the default risk is at the heart of risk controlling process. As one of the credit risks, default risk is the probability that the borrower fails to pay the interest and principal on time. With a binary outcome, loan default prediction could be seen as a classification problem and is commonly addressed utilizing user-related features with classifiers including neural network and gradient boosted trees. Since the probability that a borrower defaults may be influenced by other related individuals, there is plenty of literature forming a graph to reflect the interactions between borrowers. With the rapid growth of GNN methods, GNN methods are widely applied on the graph structure for loan default predicting problems.

• Stock movement Prediction: Though there are still debates on whether stocks are predictable, stock prediction receives great attention and there are rich literature on predicting stock movements utilizing machine learning methods. However, the task of stock prediction is challenging due to the volatile and non-linear nature of the stock market. The limitation of these non-graph approaches is that they often have a hidden assumption that the stocks are independent. To take the dependence into account, there is an increasing trend to represent the stock relations in a graph where each stock is represented as a node and an edge would exist if there are relations between two stocks. Predicting multiple stocks’ movements could then be formed as a node classification task and graph neural network models could be utilized to make the prediction.

• Fraud Detection: Observing that fraudsters tend to have abnormal connectivity with other users, there is a trend to present users’ relations in a graph and thus, the fraud detection task could be formulated as a node classification task. Aiming to detect the malicious accounts, who may attack the online services to seek excessive profits, studies show that fraudsters have two patterns: device aggregation and activity aggregation. Due to economic constraints, attackers tend to use limited number of devices and perform activities in a limited time, which may be reflected in the local graph structure.

• Event Prediction: Financial events, including revenue growth, acquisition and bankruptcy, could provide valuable information on market trends and could be used to predict future stock movement. Therefore, it draws great attention on how to predict next financial event based on past events and currently GGNN model is often used to accomplish the task.

(ref: A Review on Graph Neural Network Methods in Financial Applications | DeepAI)

Figure: Image convolution and Graph convolution (Source: towardsdatascience)

The intuition of GNN is that nodes are naturally defined by their neighbors and connections. To understand this, we can simply imagine that if we remove the neighbors and connections around a node, then the node will lose all its information. Therefore, the neighbours of a node and connections to neighbours define the concept of the node.

An important aspect of the training while implementing graph neural network is a process called Graph Convolution. In many ways the idea behind this is similar to that of image convolution which is widely used in image processing. The idea of convolution on an image is to sum the neighboring pixels around a center pixel, specified by a filter with parameterized size and learnable weight. Spatial Convolutional Network adopts the same idea by aggregate the features of neighboring nodes into the center node.

Advantages of using GNN over classic ML algorithms

The reason why this approach is more effective is because each node is classified not just by looking into the node features, but also the neighboring nodes. The task of all GNN is to determine the “node embedding” of each node, by looking at the information on its neighboring nodes.

Figure: Each node prediction is arrived at by considering the node’s feature and its neighbors (Source: towardddatascience)

This allows the model to recognise the node’s connection with other nodes that are further away. Hence it is now possible to discover hidden pattern that would have not been captured by other traditional algorithms.

When multiple layers of graph convolution are performed, this results in a node’s state containing some information from nodes multiple layers away, effectively allowing the GNN to have a “receptive field” of nodes or edges multiple jumps away from the node or edge in question. This is different from the anomaly detection using random forest. In random forest algorithm, the model finds columns(or features) that can split the data into two parts resulting in a more pure subsets of each classes (fraud or not_fraud) and where the depth of traversal is indicative of anomaly. The model does not look into all the features of a user at once. However in case of graph neural network, with each convolutional layers, the model looks not only at every features of a user, but multiple users at a time.

In the context of the fraud detection problem, this large receptive field of GNNs can account for more complex or longer chains of transactions that fraudsters can use for obfuscation. Additionally, changing patterns can be accounted for by iterative retraining of the model.

Explainability is Necessary

Predicting whether a transaction is fraudulent or not is not sufficient for transparency expectations in the financial services industry. It is also necessary to understand why certain transactions are flagged as fraud. This explainability is important for understanding how fraud happens, how to implement policies to reduce fraud, and to make sure the process isn’t biased. Therefore, fraud detection models are required to be interpretable and explainable which limits the selection of models that analysts can use.

One of the reasons why the industry has been reluctant to use neural networks is because they have to be treated like a black box. It is not clear why such model classifies something or which features have been crucial in making a prediction. Classical ML approaches had an edge over neural networks in this case. For example, decision tree algorithms use a metric called Information Gain to split the features efficiently into separate classes. This allows us to see which features have been more useful for making predictions.

Researchers are now putting a lot of effort in making GNN more explainable. GNNExplainer, for example, is proposed to provide an interpretable explanation on trained GNN models such as GCN and GAT. Model explainability in financial tasks is of great importance, since understanding the model could benefit decision-making and reduce economic losses.

Implementing GNN on cloud: Scalability

Figure: Implementing real time prediction on AWS (Link to Github repo)

It's critical to predict frauds in real time. Nevertheless, creating such a solution is challenging. There are not many online resources on converting GNN models from batch serving to real-time serving because GNNs are still relatively novel to the industry. Building a streaming data pipeline that can send incoming events to a GNN real-time serving API is also difficult since the dimension is very high and nodes are densely grouped hence its computationally heavy.

Cloud service providers like AWS have launched services to help developers apply GNN to real-time fraud detection. Amazon Neptune is a fully managed database service built for the cloud that makes it easier to build and run graph applications. Neptune provides built-in security, continuous backups, serverless compute, and integrations with other AWS services like Sagemaker, Glue, S3 and many others.

Amazon Neptune ML is a new capability of Neptune that uses Graph Neural Networks (GNNs), a machine learning technique purpose-built for graphs, to make easy, fast, and more accurate predictions using graph data. With Neptune ML, you can improve the accuracy of most predictions for graphs by over 50% (study by Stanford) when compared to making predictions using non-graph methods.

Conclusion

This article makes a case for using graph neural networks for detecting fraud as compared to other available ML approaches which were originally designed for tabular data. GNN models can develop meaningful representations to separate fraudulent users and events from legitimate ones by combining graph structure with qualities of nodes or edges, such as users or transactions. This capability is essential for identifying frauds in which fraudsters cooperate to mask their odd features while yet leaving some indications of relations.

In conclusion, utilizing ML or Neural Networks for fraud detection is a viable approach for businesses to protect themselves from the increasing prevalence and cost of frauds and scams. It is also important to create a data culture within businesses to leverage the existing data to gain deep, actionable and rich insights on potential areas of fraud and to perform advance analytics on them. Combining the power of AI and cloud technologies like AWS, businesses can detect and prevent frauds in real-time, gain competitive advantages, mitigate fraud risks and protect their financial assets.

— Author: Blesson Davis

The distributions of the original data vs synthetic data

It is apparent that the distribution of the generated data is comparable to that of the original data, and therefore, it can be safely used for training the machine learning model.

Model Training and Evaluation

The results of data analysis and model evaluation are shown below.

Scatter plot of a few continuous variables v/s target

Pair-wise scatter plot of a few continuous variables

Heatmap of a few continuous variables for the target

Accuracy of XGBoost Classifier

Confusion Matrix Plot

The preceding procedure can be utilized to train a machine learning model to identify individuals who have CKD at an early stage. The tool helps to detect signs of disease quickly and easily by taking into account only a small number of factors, such as those found in routine lab tests, urine tests, and basic personal data.

Predictive modeling

Predictive modeling in CKD care can be used to identify individuals at high risk of developing CKD and predict outcomes through analyzing data such as electronic health records.

• Identifying high-risk patients: Machine learning algorithms can be trained on large amounts of data, such as patient EHRs, to identify patterns and predict outcomes.
• Predicting progression: Predictive modeling can be used to predict the progression of CKD such as diabetes by analyzing data such as blood glucose levels and medication

To develop a predictive machine learning model for CKD progression, it is necessary to have longitudinal patient data capturing disease progression through the five CKD stages. Mathematical models have been employed for the initial classification of patients into these stages, followed by machine learning modeling to develop the predictive model.

Let’s build a predictive model using the XGBoost classifier to predict the stages of CKD using a synthetic dataset.

Limitation: This method has certain limitations. Specifically, due to the unavailability of labeled data for each stage, a mathematical model formula was utilized to create a new column in the dataset representing the CKD stage. However, it must be acknowledged that obtaining labeled data from healthcare professionals would be a more robust and reliable approach. Thus, the current limitation presents an opportunity to further improve the methodology through enhanced data collection methods.

Accuracy of XGBoost Multi-Class Classifier

Confusion Matrix Plot

Chronic Kidney Disease (CKD) is a global public health issue that demands immediate attention. Treating End-Stage Renal Disease (ESRD) is financially prohibitive, burdening an already stretched healthcare system. But hope lies in data science and machine learning, offering a proactive strategy to combat CKD and prevent its worst-case scenarios.

Early detection is crucial in managing and preventing CKD. Predictive machine learning models, using advanced mathematical and statistical concepts, can help doctors intervene early, change patients’ lifestyles, and stop CKD progression. Leveraging these cutting-edge technologies can lead to improved patient outcomes, reduced healthcare costs, and a healthier society. Proactive intervention, tailored treatments, and early detection are the keys to success in chronic disease management. Let’s work together to harness the power of data science to make this vision a reality.

LIFESPAN – The Healthcare Management System

As mentioned earlier, forecasting the progression of Chronic Kidney Disease poses a challenge due to the limited availability of longitudinal data. Nonetheless, this obstacle can be overcome with appropriate techniques and sufficient resources. It is imperative to acknowledge the complexity of the task, but with careful planning, comprehensive data collection, and the utilization of advanced modeling techniques, we can successfully predict CKD advancement and provide valuable insights for both medical professionals and patients. The main question, therefore, is how we can obtain the required longitudinal data.

Introducing the revolutionary new product from Minfy — LifeSpan!

LifeSpan can collect vital health data on numerous attributes, not just limited to CKD but any existing disease. With LifeSpan, you can streamline the way health data is collected and managed — from hospital administration to digital records, eliminating the tedious paper trail and saving valuable time. Say goodbye to cumbersome paperwork and hello to a future where data is easily accessible, and compliance is effortlessly tracked. Be ready to experience the future of health data collection with LifeSpan.

— Author: Gaurav Lohkna

References

[1] https://indianexpress.com/article/cities/chandigarh/over-7-8-million-in-india-living-with-chronic-kidney-diseases-6313251/

[2] https://journals.lww.com/kidney360/pages/articleviewer.aspx?year=2020&issue=10000&article=00016&type=Fulltext

[3] https://www.worldkidneyday.org/2020-campaign/2020-wkd-theme/#:~:text=Kidney%20disease%20is%20a%20non,life%20lost%20globally%20by%202040.

[4] Image: https://www.siemens-healthineers.com/en-in/news/chronic-kidney-disease.html

[5] Image: https://www.dneph.com/wp-content/uploads/2020/01/Stages-of-CKD-Arrow-Diagram-Only-2.jpg

[6] https://medium.com/nerd-for-tech/how-ai-is-changing-the-game-in-chronic-disease-care-d4ca145a7f49

[7] Data: https://archive.ics.uci.edu/ml/datasets/chronic_kidney_disease

The world's life expectancy has increased as a result of various advancements in healthcare facilities. However, as the world's population ages, healthcare systems must contend with a rise in demand for their services, growing costs, and a staff that is under pressure to meet patient needs.

With the growing population of senior citizens and the adverse effects of climate change, there will be a huge rise in health-related issues. It will lead to a higher influx of patients with various needs to several healthcare institutions. To tackle this meteorically rising need and the economy behind it, healthcare systems need to transform their service deliveries from episodic to management-focused long-term care.

But how can healthcare systems achieve it effectively?

This is where healthcare AI solutions come into play. Artificial intelligence (AI) solutions have the potential to handle these challenges and empower the healthcare industry to be future-ready.  By introducing new innovative ways of tackling operational challenges, AI solutions are and will be transforming the healthcare industry in 2022 and beyond.

However, before moving on to discuss various AI solutions in healthcare, let's first understand how AI solutions sync with the definition of healthcare systems

What are AI Solutions in Healthcare?

It is quite known that artificial intelligence (AI) is the capacity of computers and other machines to think, learn, and behave very similarly to humans. The employment of cognitive approaches like AI algorithms in medical situations is referred to as artificial intelligence (AI) solutions in healthcare. Very often to predict certain medical outcomes, AI solutions in healthcare are used to build quick response and effective analysis platforms against huge chunks of medical data.

Industry Application of AI Solutions in Healthcare

The healthcare industry is already using AI solutions for better clinical decisions and faster operations. In healthcare and life-science companies, AI solutions offer several advantages over traditional analytical methods. AI solutions can make healthcare systems more precise with the ability to understand training data, which further helps humans get unprecedented insights into treatment variability, care processes, diagnostics, and patient results.

Over the last few years, AI solutions have significantly contributed to the healthcare industry. From providing better service to patients to performing advanced analysis and treatment AI solutions are fueling healthcare institutions to achieve better growth and revenue. Now, let’s take a closer look at 5 healthcare AI solutions that are transforming the healthcare industry as a whole.

 1. AI & Machine Learning (ML) Solutions for Medical Imaging & Diagnosis

"Medical imaging" refers to a variety of methods that are used to examine the human body to discover, monitor, or treat several disorders. Leveraging AI solutions, medical imaging and diagnosis can be done with better precision and accuracy. Unlike humans, AI solutions can help in better recognition of intricate patterns of various human body ailments. Few real-world applications of this can be:

• The algorithms at the core of the AI solutions can expedite the process of reading complex images from CT scans or MRIs.
• AI solutions-based Machine Learning (ML) platforms in healthcare are helping to analyze MRI a shorter timeframe right from the symptom onset. It finally helps in detecting many life-threatening diseases at an early stage. For example, a brain MRI analysis with AI solutions based Machine Learning (ML) platforms helps identify tissue changes reflective of early ischemic stroke.
• Doctors can perform better and more accurate disease diagnoses with the help of automated image diagnosis systems.

Using these AI solutions in your healthcare institutions, you can directly boost healthcare expert’s productivity. Additionally, using these AI solutions in healthcare, you can ensure that your highly skilled staff spends time treating more patients rather than evaluating medical pictures.

2.     AI Solutions for Next-Gen Patient Screening

AI can be helpful in pre-screening patients long before they arrive at a hospital.

• Instead of sticking to conventional pre-screening questionnaires, healthcare institutions need to enable AI solutions-based speech and text-based interactions. It will lead to faster screening and better management of a higher volume of patients.
• Reinforcing ML with AI solutions will further help to identify novel trends and enhance patient complaint diagnosis. In comparison to a preset or static survey, AI solutions like chat-bots can easily replicate the expertise of a real-world healthcare practitioner.
• Adopting AI solutions-based Natural language processing (NLP) applications allows your potential patient to speak or write to a doctor about their symptoms. The AI agent and patients will quiz one another to mimic a conversation between a doctor and a patient.
• AI solutions can provide virtual health assistance to help advise a patient on how to proceed with their medical care. It includes timely reminders about check-ups, whether to visit the emergency room, schedule a consultation, or take an over-the-counter medication. If the patient needs medical attention, the AI solution can recommend a doctor based on the patient's location, availability, and insurance information.

In cases of health emergencies or growing accessibility issues in urban areas, AI solutions in healthcare can be a revolutionary innovation.

3.      Predictive AI Solutions for Preventative Healthcare

Preventative care, as the name itself suggests, is any procedure or method that defends against any probable health issues. Today, the term "preventive care" solely refers to regular medical checks, immunizations, dental cleanings, etc. Leveraging predictive technology of AI solutions can significantly enhance ongoing preventative treatments and boost overall healthcare infrastructure.

• Smart wearables like the Apple Watch, Fitbit, Garmin, etc. work on the basis of AI solutions coupled with sensors. These smart fitness wearables can support preventative care in the healthcare sector.
• Information on heart rate, activity, nutrition, VO2, and sleep is automatically supported by these devices. With add-ons, these devices can now measure blood pressure, blood sugar, and weight as well.
• Preventative healthcare is possible by using the data from these wearables to identify potential health issues even before they occur. This information can be used to build analytical models that will help in making accurate predictions regarding certain illnesses.
• These models are able to gather data from wearable sensors in real-time, alerting users when they have reached a threshold calling for preventative medical care.

The gadget-based healthcare AI solutions help share data with the physician for a detailed medical understanding. Additionally, it can help offer better care by spending less time on the patient's symptoms and background.

4.     AI Solutions for Healthcare Data Management

The healthcare industry has to deal with a huge pile of data. This increases the chances of straying imperative data. Big medical data comes with bigger data management challenges. It can take years to process this data, connect significant data points, develop an appropriate diagnosis, or support new medicines’ discovery out of this huge data.

• Data is an asset to every business, and hence, it should be of top priority to preserve it. AI is helping many healthcare organizations to realize important data points from health records’, analyze  and then present it for easier examination and maintenance of patient records.
• AI solutions when deployed in healthcare, can lead to easier data processing and bring quick visibility into patterns, which can be very daunting for any individual.

5.     AI Solutions for advanced Healthcare Drug Discovery

COVID-19 has shed light on the creation of vaccines and technologies like AI, ML, or DL have proved to be of greater help in expediting the vaccine discovery process. It has taken a year and more for the world to discover the vaccine. It has only shown that the processes for identifying vaccines and pharmaceuticals are labor-intensive, expensive, time-consuming, and usually unsuccessful.

• Pharmaceutical companies are increasingly using AI tools like DL to develop and test new drugs.
• It needs a lot of data analysis to develop new pharmaceuticals since there are so many possible chemical combinations. AI solutions being exceptionally faster and quicker at handling enormous data volumes can accelerate the entire process.
• AI solutions can help create more medicines that are authorized in less time and at cheaper costs.

AI solutions can help bolster clinical efforts toward faster drug discovery. This will lead to faster market release and subsequent cost & time savings.

Summary

• AI solutions in the healthcare sector can help every healthcare worker to view patient data and other information to provide accurate diagnoses and treatment suggestions.
• AI solutions let healthcare institutions gather huge data sets like clinical trial data, claims data, demographic data, etc. Deploying AI solutions can lead to faster data processing which will probably take years if done through the conventional methodology.
• Using Ai solutions in the healthcare sector can lead to the discovery of necessary data patterns and insights, thus, potential for further scalability.

The above, plus various other benefits of AI solutions, have made it a turning point in the healthcare field. Whether in the form of machine learning that aids in the development of  medicine, patient prescreening, in-treatment communications through NLP or AI solutions like imaging analysis, AI solutions are already outgrowing the conventional methods and are bound to grow more in coming years. Explore how our AI solutions are modernizing our clients’ healthcare operations and helping them achieve big in their business. Do you want to know how AI solutions can benefit your healthcare company? Contact us today to learn more.

Book a 1-hour free consultation with pricing details.

Minfy declared a Growth Champion 2023 by Economic Times and Statista!

Minfy is amongst India's Fastest Growing Companies (32nd) as per a study run by The Economic Times and Statista and is listed amongst India's Growth Champions 2023.

Economic Times and Statista pick and rank a select group of businesses that have perfected the art of gaining ground year after year. Aside from growth, key considerations are that businesses have real economic substance, sustainability, and legitimacy.

Thank you The Economic Times and Statista.

#Indiagrowthchampion2023 #betterfocusbettergrowth #bionicforabetterworld #economictimes

Click here to see the detailed rankings.
https://lnkd.in/gvGq9_5q

Demand forecasting is a common use case of AI-ML. It can be used to identify areas of improvement and best practices that can help businesses improve its forecasting accuracy. It helps in production planning and determine incremental impacts of new initiatives, plan resources in response to expected demand, and project future budgets.

The following image depicts a simple forecasting cycle. It is all about using historical data, train forecasting model on it, generate forecast to make informed decisions. It’s a cyclic process — when new data arrives, we feed it back into the forecasting model and retrain it so that the model remains up to date.

Demand forecasting cycle

A frequent question we get from customers, consultants and other business experts is “What would you consider a good level of forecast accuracy?”

However, it may not be the most relevant question to raise, though. First of all, forecasting is seldom the objective in any retail or supply chain planning scenario. It is always a means to an end. A forecast is only useful if it helps us accomplish other objectives, like increased shelf availability, accurate production planning, less wastage.

While forecasting is a crucial component of all planning activities, it is only one cog in the machine, which means that other factors like changes in consumer behaviour, economic turndowns, historical data quality can have a significant impact on the forecasting accuracy.

What degree of forecast accuracy can actually be obtained depends on a number of factors. This is one of the reasons why comparing forecast accuracy between businesses, or even between products within the same business, is so challenging.

When integrating demand forecasting systems, it’s essential to understand that they are vulnerable to unpredictable situations. So, the demand forecasting machine learning models should be re-trained according to current reality.

With large forecast horizon, the probability that future demand will be impacted by developments that we are not currently aware of grows noticeably. Weather-dependent demand is a straightforward illustration. If we need to make decisions on what quantities of summer clothes to buy or produce half a year even longer in advance, there is currently no way of knowing what the weather in the summer is going to be.

On the other hand, if we were in charge of overseeing the replenishment of ice cream in grocery shops, we could utilize short-term weather forecast to estimate how much ice cream to transport to each location.

Important features to consider while training ML model for forecasting:

 Seasonality

Seasonality refers to the regular and predictable pattern of demand for goods or services that occurs at certain times of the year, such as holidays, weather changes, or cultural events. What’s more, understanding of seasonality for each of your products can give you a head start in demand planning. With a good grasp of how much consumer demand varies, you can select the right demand forecasting models, focus on making plan adjustments and spot true outliers that require further attention.

For example, a business that sells winter clothing may experience a significant increase in demand during the winter months and a decrease in demand during the summer months. Similarly, a business that sells ice cream may experience higher demand during the summer months and lower demand during the winter months.

Weather data feature effect

Demand will typically vary from region to region, depending on local calendars and weather. You may need to segment your data geographically to identify seasonality patterns. At the same time, aggregating forecasts to a less granular level — product category instead of product, for example — may make it easier to distinguish seasonal patterns from random noise.

Rather than including only historical time series and demand, incorporate weather data into a machine learning model for demand forecasting as the model can account for the impact of weather on consumer behaviour and give better predictions accordingly. This can lead to help businesses make better decisions about inventory management, marketing strategies, and other aspects of their operations.

Furthermore, weather data can be used to create more granular and accurate models by breaking down the weather into different variables such as temperature, precipitation, humidity, etc. This allows for a more nuanced analysis of how weather affects demand for different products or services.

The following code snippet reads weather data from a csv file and pre-processes it by resampling to daily frequency and imputes the missing values by linear interpolation imputation technique, so that it can used with the historical time-series data to train a forecasting ML model.

Code snippet to pre-process weather feature

Location

For certain use cases, location is an important feature when training a demand forecasting ML model.

Competition & Vendors: The location of a business and its associated vendors location where a particular product is being sold can play a vital role. Also, if there are many similar businesses in the area, the demand for a particular product or service may be lower than in an area with few or no competitors.

Demographics: The demographics of an area may play a crucial role in determining the demand for a product or service. Different age groups, income levels, and cultural backgrounds have different preferences, and this can be reflected in demand patterns.

Infrastructure: The availability and quality of infrastructure such as roads, transportation, and logistics can also affect demand. Areas with poor infrastructure may have lower demand due to difficulties in accessing or transporting products.

Holidays

Sales patterns can vary significantly during holiday seasons due to changes in consumer behaviour, such as increased shopping activities, changes in preferences, and higher demands for specific products or services.

Including holiday data into the machine learning model helps to improve the accuracy of sales forecasts during these periods, which is essential for businesses to plan their operations and inventory management. Additionally, taking holidays into account in the model can also reveal important insights into customer behaviour and preferences, which can help businesses tailor their marketing strategies to better engage with their target audience.

Trends

One of the benefits of including trends in demand forecasting algorithms is the ability to anticipate market changes and respond accordingly. For example, if a trend analysis indicates a shift in consumer preference for a particular product, the company can adjust their production and marketing strategies to meet the new demand. Additionally, trend features can help identify whether changes in demand are due to short-term fluctuations or long-term changes in the market.

Moreover, trends can also help identify potential bottlenecks in the supply chain, which can help companies optimize their inventory levels and reduce waste. By accurately forecasting future demand, companies can avoid stock outs or overstocking, which can lead to significant losses.

Let’s say you have a dataset that contains daily temperature readings for a specific location over a period of four years. You want to include a feature that represents the trend in temperature over time. One way to do this is to calculate a moving average of the temperature readings for each day, using a window size of, say, 30 days. This will give you a new feature that represents the average temperature over the past 30 days, which can be a good indicator of whether the temperature is trending up or down.

Code snippet to pre-process Temperature trend feature

Above code snippet, reads the ‘temperature_data.csv’ file which contains a column called ‘temperature’ that has the daily temperature readings. The rolling () function is used to calculate a rolling average of the temperature column with a window size of 30. The resulting rolling average is then added as a new column to the data frame, with the name ‘temp_trend’.

Machine learning models/tools that can be used for demand forecasting

• Amazon Forecast: Amazon Forecast is a time-series forecasting service based on machine learning (ML) and built for business metrics analysis. Amazon Forecast uses machine learning (ML) to generate forecasts with just a few clicks, without requiring any prior ML experience. If you have a time-series forecasting use case and you don’t have much experience in machine learning, Amazon Forecast is likely a good choice.

• Amazon SageMaker:It is a fully managed machine learning service. With SageMaker, data scientists and developers can quickly and easily train and test machine learning models, and then directly deploy them into a production-ready hosted environment. If you require more flexibility and control over your machine learning models, add and create features, built relationship between the features then Amazon SageMaker can be a better fit.

• Amazon Deep AR: The Amazon SageMaker Deep AR forecasting algorithm is a supervised learning algorithm for forecasting scalar (one-dimensional) time series using recurrent neural networks (RNN).

• ARIMA: An autoregressive integrated moving average model is a form of regression analysis that gauges the strength of one dependent variable relative to other changing variables. The model’s goal is to predict future data points by examining the differences between values in the series instead of through actual values.

• SARIMA: Seasonal Autoregressive Integrated Moving Average, SARIMA or Seasonal ARIMA, is an extension of ARIMA that explicitly supports univariate time series data with a seasonal component. It adds three new hyperparameters to specify the autoregression (AR), differencing (I) and moving average (MA) for the seasonal component of the series.

• LSTM’s: Long Short — Term Memory is a type of neural network architecture that is commonly used for time series forecasting. LSTM networks are particularly useful for dealing with time series data that have long-term dependencies or complex patterns. The basic idea behind LSTM is to use a series of memory cells to remember and store important information about the input sequence, and then use this information to make predictions about future values in the sequence.

Conclusion

Demand forecasting is a crucial aspect for any business operation, and the use of machine learning algorithms can significantly improve the accuracy and efficiency of the forecasting process. By considering historical data, current trends, and other relevant factors, businesses can better understand their customers’ needs and optimize their production and inventory levels to meet demand while minimizing the loss.

You should ensure that the process of re-training your ML model with new data happens at continuous intervals. The goal of model retraining is to adapt the model to changing patterns in the data, or to improve its accuracy as new data becomes available.

Retraining a model is important because machine learning models are only as good as the data they are trained on. It can also help to address issues such as model drift, where the performance of a model deteriorates over time due to changes in the data distribution.

In conclusion, demand forecasting using machine learning is not a one-time process, but a continuous effort that requires ongoing monitoring and retraining of models on latest available data. By leveraging the latest technologies and insights, companies can stay ahead of the curve and thrive in today’s dynamic and rapidly evolving marketplace.

 - Author Yasir Ul Hadi

References:

1. https://towardsdatascience.com/sales-forecasting-from-time-series-to-deep-learning-5d115514bfac
2. https://www.relexsolutions.com/resources/machine-learning-in-retail-demand-forecasting/
3. https://www.relexsolutions.com/resources/measuring-forecast-accuracy/
4. https://alloy.ai/choosing-the-right-demand-forecasting-model/
5. https://mobidev.biz/blog/machine-learning-methods-demand-forecasting-retail

— Author: Gaurav Lohkna

P.S.: For a more detailed and technical analysis of AI-based solutions in Chronic Kidney Disease management (one of 13 major chronic diseases), kindly await the publication of my upcoming study/article. It provides an in-depth examination of cutting-edge technologies, algorithms, and methodologies in chronic care.

References
[1] World Health Organization (WHO). Noncommunicable diseases: Mortality https://www.who.int/data/gho/data/themes/topics/topic-details/GHO/ncd-mortality
[2] World Health Organization (WHO). Global health observatory data. https://www.who.int/gho/en/