The results are satisfactory despite of some minor flaws. The finished shopping cart looks like the following. The weight sensors are between two black planes, and are the only contact points of these two planes. The RFID scanner and camera are attached at the front of the shopping cart.
The following picture shows, the mobile phone application can correctly show everything in the shopping cart. The recommendation is also provided (the item with red price tag is recommendation):
For the packed item, the recognition accuracy is almost 100 percent, which is natural because the property of RFID. As for the image recognition, the accuracy was not in a satisfactory range even when the designated item is in the picture captured by camera. This was because AWS Rekognition analyzes every object appearing in the image, and we set the maximum number of returned tag to be 20. Thus, when the background is in a mess, there is considerably high probability that we cannot receive the tag we want. We made improvement by wrapping our shopping cart by white papers. After this adjustment, the accuracy reached over 90 percent. Errors still occur because the range of possible returned tags is too large, so at some specific angles, AWS Rekognition identify an orange as a grapefruit. Our solution is to restrict the possible tags, which also makes sense for practical use, because a supermarket can only provide limited choice of products. After adding the restriction, the accuracy of image recognition also reached nearly 100 percent.
The main flaw that may cause errors is that the weight sensors are not accurate and stable enough. The measurement of the whole weight scaling device is accurate to 1 gram, however, its fluctuation can be as large as over 50 grams when there is nothing on it. Thus, we have to make the threshold that triggers camera and RFID scanner pretty large. The consequence is that sometimes a relatively light product like a small banana may fail to trigger the object identifying mechanism.
The main flaw that may cause errors is that the weight sensors are not accurate and stable enough. The measurement of the whole weight scaling device is accurate to 1 gram, however, its fluctuation can be as large as over 50 grams when there is nothing on it. Thus, we have to make the threshold that triggers camera and RFID scanner pretty large. The consequence is that sometimes a relatively light product like a small banana may fail to trigger the object identifying mechanism.