Template matching NCC

视频教程8 - NCC模板匹配：https://singtown.com/learn/49598/

NCC algorithm :

# Template Matching Example - Normalized Cross Correlation (NCC)
#
# This example shows off how to use the NCC feature of your OpenMV Cam to match
# image patches to parts of an image... expect for extremely controlled enviorments
# NCC is not all to useful.
#
# WARNING: NCC supports needs to be reworked! As of right now this feature needs
# a lot of work to be made into somethin useful. This script will reamin to show
# that the functionality exists, but, in its current state is inadequate.

import time, sensor, image
from image import SEARCH_EX, SEARCH_DS
# Introduce SEARCH_EX and SEARCH_DS from image module and only SEARCH_EX if you choose to use from import. 
# SEARCH_DS There are only two parts needed instead of the whole module of image. 

# Reset sensor
sensor.reset()

# Set sensor settings
sensor.set_contrast(1)
sensor.set_gainceiling(16)
# Max resolution for template matching with SEARCH_EX is QQVGA
sensor.set_framesize(sensor.QQVGA)
# You can set windowing to reduce the search image.
#sensor.set_windowing(((640-80)//2, (480-60)//2, 80, 60))
sensor.set_pixformat(sensor.GRAYSCALE)

# Load template.
# Template should be a small (eg. 32x32 pixels) grayscale image.
template = image.Image("/template.pgm")


clock = time.clock()

# Run template matching
while (True):
    clock.tick()
    img = sensor.snapshot()

    # find_template(template, threshold, [roi, step, search])
    # ROI: The region of interest tuple (x, y, w, h).
    # Step: The loop step used (y+=step, x+=step) use a bigger step to make it faster.
    # Search is either image.SEARCH_EX for exhaustive search or image.SEARCH_DS for diamond search
    #
    # Note1: ROI has to be smaller than the image and bigger than the template.
    # Note2: In diamond search, step and ROI are both ignored.
    r = img.find_template(template, 0.70, step=4, search=SEARCH_EX) #, roi=(10, 0, 60, 60))
    #find_template(template, threshold, [roi, step, search]) The 0.7 I the threshold is the similarity threshold and  ROI is the region to be matched(its upper left vertex is （10，0）), a rectangular of 60 by 80. 
    # Note: The size of the ROI should be larger than the template images and smaller than frame butter. 
    # Mark the matched image. 

    if r:
        img.draw_rectangle(r)

    print(clock.fps())

Note：The size of template image should be larger than the flash built-in OpenMV, so we need to plug the SD card in before we go on to the following steps(Card first, then power on.). And the template matching only apply to firmware version 1.5 and above or you, when operating it, would be warned that cannot find SEARCH_EX.

First, we need to create or import a template which have to be in pgm and its size cannot be larger than the pixels of OpenMV. It is a good choice to subtract a template image from OpenMV. Run helloworld.py routine first and cut the image when the frame butter displays it.

Choose save image selection to pc. Here the image we cut and store directly from OpenMV is in bmp and we need to transfer it into pgm, which can be finished in the following website.https://convertio.co/zh/bmp-pgm/

After that, we store the template in pgm to SD card, which has 8 templates. The template above is stored as ball0.pgm.

Then open the routine of template matching.

Change the filename of the template of 28th line, template.pgm, to ball0.pgm.

Then run it!

This is the directions for you to use the function find_template:

r = img.find_template(template, 0,7, roi=(10,0,80,60), step=4, search=SEARCH_EX) The 0.7 is the threshold is the similarity threshold and ROI is the region to be matched(its upper left vertex is （10，0）), a rectangular of 60 by 80. Note: The size of the ROI should be larger than the template images and smaller than frame butter.

The mechanism of template matching is ncc algorithm so it can only match areas similar in size. If you want to match images with different sizes, the templates has to be in different sizes.

If there emerges following problems:

1.The template image could be too large so we recommend your image smaller than 80*60.

2.The internal storage of OpenMV2 is not enough so we have to change QQVGA to QQCIF.

模板匹配和特征点检测的比较：

• 模板匹配（find_temolate）采用的是ncc算法，只能匹配与模板图片大小和角度基本一致的图案。局限性相对来说比较大，视野中的目标图案稍微比模板图片大一些或者小一些就可能匹配不成功。

• 模板匹配适应于摄像头与目标物体之间距离确定，不需要动态移动的情况。比如适应于流水线上特定物体的检测，而不适应于小车追踪一个运动的排球（因为运动的排球与摄像头的距离是动态的，摄像头看到的排球大小会变化，不会与模板图片完全一样）。

• 多角度多大小匹配可以尝试保存多个模板，采用多模板匹配。

• 特征点检测（find_keypoint）: 如果是刚开始运行程序，例程提取最开始的图像作为目标物体特征，kpts1保存目标物体的特征。默认会匹配目标特征的多种比例大小和角度，而不仅仅是保存目标特征时的大小角度，比模版匹配灵活，也不需要像多模板匹配一样保存多个模板图像。

• 特征点检测，也可以提前保存目标特征，之前是不推荐这么做的，因为环境光线等原因的干扰，可能导致每次运行程序光线不同特征不同，匹配度会降低。但是最新版本的固件中，增加了对曝光度、白平衡、自动增益值的调节，可以人为的定义曝光值和白平衡值，相对来说会减弱光线的干扰。也可以尝试提前保存目标特征。