29-12-2017

Most people still consider Machine Learning to be far removed from their own personal lives, but that’s not necessary. But what if someone told you that it is really easy to implement machine learning technology and make intangible situations tangible? This blog provides a step by step explanation of how to use machine learning in a very useful way. If you are a bit tech-savvy, you are easily able to create a lot of features to improve day-to-day processes and functionalities by yourself.

Employees are company’s most valuable assets. Satisfied, engaged and motivated colleagues are more productive and will leave the company less quickly. But, it is sometimes quite hard to see whether someone is happy or motivated. Besides that, most companies only measure employee satisfaction once a year (mostly with traditional and/or biased surveys).

Here we won’t talk about the ability to increase employee satisfaction but only describes an objective and reliable way to measure the current state of mind of all employees. Our goal is to create a setup environment where we use low-budget hardware and machine learning technology to measure employee satisfaction near real-time. The experiment can only be a success if we are able to capture clear pictures of the faces of a person. That is possible if the hardware is set up correctly.

What do you need for the experiment:

  1. Raspberry Pi with camera module
  2. A free accessible machine learning API like Google Vision or Microsoft Azure API
  3. SAP Cloud Platform

For our setup, the Raspberry Pi was installed and stationed in a strategic location in our coffee corner (figure 1). The coffee corner is a strategic location because almost everyone visit this this place more than once on a daily base. The right camera angle is crucial for a reliable analysis. Ideally, everyone within a 3 meter range should be detected correctly.

Figure 1: The Raspberry Pi has to be positioned correctly to detected faces at the right angle

The Raspberry Pi took a picture every time the camera detected movement. These images were then sent to an Machine Learning API which analyzes the image and returns the results in JSON code. We did multiple tests in this setup with different angles and distances. Despite some blurred, shaked and defocused photo’s, most photo’s do contain recognizable faces and were able to be successful analyzed afterwards. Figure 2 shows a picture we took and the results that we retrieved out of the API.

Figure 2: Photo taken by the Raspberry Pi and the received results in JSON code

The retrieved results are probability scores per emotion. In this example, the emotion ‘neutral’ has the highest probability (94,71%) and that seems to correspond with the real emotion. We would like to model this raw JSON data and therefore the API results are stored in the SAP HANA database, send via the SAP Cloud Platform Web Service. Subsequently, the logged emotions are consumed in a visualization tool like SAP Analytics Cloud or SAP BusinessObjects.

So, imagine the possibilities when this test setup is refined and expanded. An HR-manager (or in our situation a Chief Human Resources Officer) is then able to see when and where employees are happy, full of disgust, in fear or disappointed and can act on that.

There are some privacy issues when using such a setup. People should not be recognizable at a later moment and it should not be possible to trace the data of a specific person back to them. What we did to mitigate these are for example:

  1. the taken picture will never leave the local directory of the Raspberry Pi and will be deleted after analysing the expression of the person.
  2. The emotion recognition API only uses facial landmarks to analyse the image.

Despite these precautions, we also warn people about the experiment and the use of the camera and we shut down the camera when the situation was not appropriate.

So as you can see, it is not hard to deploy Machine Learning in real-life situations. Be creative and try it yourself. The example here is quite elementary and set up in an experimental environment. With limited resources and minimal investment it is possible to quantify intangible processes or situations. However before you start exploring, make sure what you want to accomplish and measure. In this case, the test setup works and our goal (objective emotion measurement) is achieved.

SAP Cloud Platform and SAP HANA are ideal to support these kinds of experiments, because it combines an application server with an database server. With the real-time connection to different visualization tools, all the results can be processed rapidly.

At Nextview we combine Operational Excellence and Innovation to create “Smart Business”. The easy usage of Machine Learning is one example of powering business with disruptive new possibilities. Interested in what Machine Learning or Smart Business can mean for your organisation? Please contact me directly via jip.koning@nextview.nl or read this blog from Ton van Dolder about “Smart Business”. Do you want another example of a SAP Leonardo project? In that case, Pieter Hendrikx’ blog about “The Smart Garbage Bin” is a nice one.