‍

This article shows how to process utility bills automatically. Specifically, how to extract and obtain key information from bills related to supply and consumption useful for numerous processes. The main aspects covered in the article are summarized in the following list:

• Application scenarios
• Use case description and issues
• The information of interest to be extracted
• Alternatives for processing
• myBiros: a modern IDP approach.
• Conclusions

Let's see right away in what scenarios it can benefit to process bills automatically.

Application scenarios

Having the information (in structured format) contained in utility bills proves useful in many processes and application scenarios, for example:

• Processing and monitoring of all energy invoices received and availability of data to support supply tenders;
• Production of analyses, reports and KPIs preparatory to energy efficiency activities;
• Automatic accounting and reporting of all invoices;
• Supporting the establishment of forecast budgets and monthly expenditure allocations.

Use case description

Bills fall into the category of documents semi-structured, in fact each provider defines the format to be used at will. The various formats typically contain a very similar if not equal set of information. The same provider, depending on the type of supply, may change bill formats over time. Given the complexity of the documents involved and the large number of different formats, numerous complications arise that penalize traditional solutions, limiting their accuracy and thus the degree of automation of the entire system. The following are some examples:

• Document capture is not standardized. The bills to be processed may be acquired from different sources, resulting in heterogeneous formats in which the documents are received. The most common formats are digital pdfs, scans, and photographs. The heterogeneity of formats complicates their processing. One example is document captures using smartphones, which often produce blurred or rotated images that are therefore difficult to read. Such issues, reduce the applicability of traditional approaches and penalize the efficiency of traditional data entry. To resolve these issues, image normalization steps are required that increase the complexity of the pipeline and decrease the generality of the approach.
• The bill formats are many (at least one per provider and per type of supply), the information of interest is numerous, and their location in the documents varies. Reasoning in an international context, in addition to format change, it is necessary to nimbly manage language change and units of measurement used. The number of useful formats and field positions to be considered can therefore also become very large. An additional complication is the common change in bill format over time.
• Several interesting pieces of information are contained in tables or pictures, this further complicates the process.

These are just some of the problems associated with automated bill processing. The article follows by describing methodological alternatives for solving the problem. For ease of discussion, the article will deal with Italian energy bills without loss of generality.

Relevant informationto be extracted

The most important information to extract involves consumption, supply details, and contract holder information. These typically can be found in different formats and units. Another aspect that complicates the use case is the amount of relevant information, in fact we are talking about more than 25 different fields. The main information of interest is reported below: tariff, type of consumption, total consumption, cost of energy commodity, reporting period, POD, total payable, provider data, consumption bands (such as f0,f1,f2,f3), recipient and holder data, supply data (voltage, committed power, etc. )

To properly process such a document, therefore, several features are required to be used in synergy: extraction of key-value information, interpretation of tabular data, and classification of bill type (gas, electricity, etc. ).

Alternatives for processing

Manual approach

Manual extraction of data from energy bills (but the same is true for any bill) is costly, time-consuming, and error-prone. The processing steps require skilled people who can identify relevant information in the document and extract it consistently from sometimes complex layouts. Some challenges and issues related to manual processing include:

1. Cost Issues
   Although manual extraction of data from documents may work for small companies with limited processing piers, it becomes expensive as the company expands. In many cases, manual data extraction involves hidden costs in addition to those associated with hiring more employees to perform data extraction. Adding more employees to the workflow often leads to coordination problems, which in turn can lead to errors, especially in identifying and entering data. Data validation itself is a critical step that adds to the cost component of processing. In fact, information extraction, without verification steps, can have error rates as high as 4 percent. The 1-10-100 data entry rule is well known in data entry back offices: verifying the accuracy of the data at the point of entry costs about $1, cleaning up errors by rechecking the entire batch of data costs $10, and escaped errors cost the company $100 or more.
2. Time Issues
   Manual data extraction is time-consuming, especially for global supply chains, as multiple checks and approvals are required and teams in different countries often need to be coordinated. Integrating the various processing and verification steps can also become complicated because of the various people involved in the hierarchy and the approval levels required.
3. Human fallibility
   The data entry process is repetitive and tedious and can be demoralizing. In addition, energy bills do not have a standard format. Although much of the information listed in the previous sections is present in all documents, each vendor uses a different format with significant spatial variability in the data. Manual data extraction is laborious given these factors. Language variations between place of issue and place of delivery can also pose a significant challenge in understanding the data. All of these variables contribute to an increased likelihood of introducing errors into the process itself.

Traditional OCR solutions

The processing of bills using traditional OCR techniques and template matching/regex is a decidedly ill-advised and wasteful approach. This is because it is necessary to have an ad hoc set of rules and templates for each document type. The formats are many and the vendors are potentially undefined in number a priori. The languages to be considered are often numerous for a solution that must work in processes with global reach. This makes the number of rules or templates needed decidedly numerous and constantly changing as new formats and countries are considered. All this results in a high setup and maintenance cost for the solution and often poor performance. In addition, maintenance and configuration of the solution must be done by trained resources with technical training.

In general, all the problems presented in the use case description plague both manual and traditional approaches. This has led to the need for higher performance solutions that solve the complications recounted so far. Thanks to recent developments in the field of AI and particularly Deep Learning, higher quality results can be achieved. In addition, time and cost are lowered in each step of the pipeline. Starting from OCR capable of learning, improving over time and transcribing even handwritten documents to semantic analysis and interpretation of tabular data (and much more). The set of techniques based on artificial neural networks for comprehensive document processing is commonly called Intelligent Document Processing.

Intelligent Document Processing (IDP)

A modern approach based on Deep Learning techniques is the best choice for solving such problems. In fact, the ability to use the best techniques of Computer Vision for document analysis and reading, and NLP for natural language understanding, makes it possible to solve previous problems. It is not necessary to adapt the solution each time (writing new rules or configuring new templates). It is sufficient to have a sufficient amount of data belonging to the process to instruct the system.

Another advantage is the ability to apply the same approach to solving different tasks, such as: key-value data extraction, tabular data extraction, and document classification. Such an approach can also benefit strongly from the human validation step. The latter consists not only of correcting errors made by the system, but also enables continuous learning of the algorithm. In doing so, the algorithm improves over time and calibrates itself on the specific process.

Compared to traditional solutions, the maintenance and evolution of the system is also simplified. In fact, adding a new field that you want to extract, a document category to classify, or wanting to add a new language among those supported does not involve writing code. The collection of new documents will be sufficient, and subsequent retraining of the system can be easily followed even by non-technical resources. Finally, the most effective IDP solutions allow for unprecedented accuracy of results far surpassing traditional approaches.

myBiros and benefits

myBiros is a performant, easy-to-use and versatile Intelligent Document Processing solution that enables automatic document processing. Core functionalities are information extraction and automatic document classification. All this is offered through a prebuilt set of ready-to-use APIs with pre-trained templates for common use cases and the ability to retrain the entire pipeline (both the OCR engine and the document interpretation system) for custom cases.

By leveraging advanced deep learning techniques that analyze multimodal features, it is possible to process all document types with a single solution. The system uses pre-trained models, data-augmentation techniques, and for that reason can be trained with a small volume of data allowing even processes involving a small volume of documents to be automated.

This solution includes a scoring mechanism: in fact, the system reduces false positives by enabling the ability to review low confidence data while minimizing errors. Interaction with a human user enables the system to correct errors while continuing to train the system so that past mistakes are not repeated(human in the loop and continuous learning). Finally, the high scalability of the cloud-based architecture makes it possible to process highly variable masses of documents without having to allocate expensive resources in advance.
Additional features include the ability to process tabular data, identify artifacts present in the image, and the ability to process heterogeneous and multi-language documents with a single pipeline.

The features mentioned so far enable myBiros to perform optimally in bill processing. By effectively and quickly managing to identify all relevant information. If you are curious about how myBiros works in order to simplify bill processing, please contact us and try our demo. We are ready to help you!

‍