Relevance Score Assignment for Artificial Neural Network

Simple SummaryContent extracted from patent full text and abstract with AI.

This patent describes a method and system for assigning a 'relevance score' to each item (such as a pixel, word, or data point) analyzed by an artificial neural network. By reverse-propagating the output (the network's decision) through the network using a specific mathematical procedure, the approach determines how much each individual part of the input contributed to the overall decision. This makes the inner workings of AI, especially deep neural networks, more interpretable by highlighting which parts of the input were most influential for the network's prediction, often presented via heatmaps or similar visualization tools.

Use CasesContent extracted from patent full text and abstract with AI.

• Explaining image classifications in computer vision applications (e.g., clarifying which image regions influenced a diagnosis in medical imaging).
• Highlighting regions of interest (ROI) in images or videos for medical, security, or remote sensing purposes.
• Summarizing or extracting key features from text documents in NLP applications (e.g., highlighting important words in legal or financial documents).
• Assisting auditors and regulators in understanding AI decisions for compliance and transparency (e.g., in finance or lending decisions).
• Optimizing neural networks by identifying and pruning less relevant connections, leading to more efficient models.
• Adaptive data processing, such as performing less lossy compression on the most relevant parts of media files (images, video, audio).
• Supporting human decision-making in applications where AI predictions need to be justified or explained, such as healthcare, legal, or insurance.
• Investigating causes for AI classifications in anomaly or fraud detection systems.

BenefitsContent extracted from patent full text and abstract with AI.

• Enables transparency and interpretability for decisions made by complex neural networks ('opening the black box').
• Supports trust and compliance in AI systems by providing understandable explanations for automated decisions.
• Can be implemented on many types of data (images, text, audio, graphs).
• Helps human experts quickly identify key factors or regions influencing a network's decisions, saving time and effort.
• Improves the efficiency of neural networks by enabling targeted optimization or pruning based on relevance scores.
• Allows dynamic allocation of resources in data processing (such as compression or measurement) based on the importance of different input regions.
• Does not require modification of the existing trained neural network—applies to pre-trained models.
• Useful for enhancing AI usability in safety-critical and regulated industries (e.g., healthcare, finance, legal).

Technical Classifications (CPCs)

Inventors & Applicants

Patent Abstract

The task of relevance score assignment to a set of items onto which an artificial neural network is applied is obtained by redistributing an initial relevance score derived from the network output, onto the set of items by reversely propagating the initial relevance score through the artificial neural network so as to obtain a relevance score for each item. In particular, this reverse propagation is applicable to a broader set of artificial neural networks and/or at lower computational efforts by performing same in a manner so that for each neuron, preliminarily redistributed relevance scores of a set of downstream neighbor neurons of the respective neuron are distributed on a set of upstream neighbor neurons of the respective neuron according to a distribution function.