[PYTHON] Detect mosaic points in the image

(Added on May 24, 2017) I registered the code on github. It is the same as what is described on this page. https://github.com/summer4an/mosaic_detector

Premise

https://twitter.com/stsiizk/status/858517661889646592 that? Maybe this is ... So I decided to prepare (?) For the coming day.

There are multiple types of mosaics, but this time we will focus on a large number of boxes painted in a single color.

I tried google, but it was all about mosaic processing, and I couldn't find a way to detect the mosaic part, so I did it myself.

The goal is to detect the mosaic part of the image and paint it white.

I used python3.5.2 and OpenCV3.2.0.

policy

After trial and error by building code with image processing software and python, we obtained satisfactory results with the following procedure.

  1. Convert to grayscale
  2. Edge detection with Canny
  3. Black and white inversion
  4. Blur a little
  5. Pattern matching for grid images of various sizes
  6. Fill the matched area

The reason why I blur a little with 4 is that the result of 3 is that the border line of the box does not become a straight line with a width of 1 pixel, but it goes back and forth between the widths of 2 pixels, so pattern matching is difficult It's because of it. The reason for using grid images of various sizes in 5 is that pattern matching is vulnerable to scaling. I used a grid image of 11 to 20 pixels square, which seems to be common.

code

First, create a grid image for pattern matching of each size below.

python


#!/usr/bin/env python
# -*- coding: utf-8 -*-

#Generate a pattern image file for when the mosaic size is from 11 to 20.
#The following reference materials.
#  http://qiita.com/suto3/items/5181b4a3b9ebc206f579

from PIL import Image

def make_image(masksize, filename):
    picturesize = 2+masksize+masksize-1+2
    screen = (picturesize, picturesize)

    img = Image.new('RGB', screen, (0xff,0xff,0xff))

    pix = img.load()

    for i in range(2,picturesize,masksize-1):
        for j in range(2,picturesize,masksize-1):
            for k in range(0,picturesize):
                pix[i, k] = (0,0,0)
                pix[k, j] = (0,0,0)

    img.save(filename)
    return

for i in range(11, 20+1):
    make_image(i, "pattern"+str(i)+"x"+str(i)+".png ")

If you execute it, you can get 10 images like this. pattern11x11.png pattern20x20.png

The following is detected using this grid image.

python


#!/usr/bin/env python
# -*- coding: utf-8 -*-

#Detect the mosaic part and paint it white.
#The following reference materials.
#・ Template Matching
#    http://docs.opencv.org/3.2.0/d4/dc6/tutorial_py_template_matching.html
#    http://opencv.jp/cookbook/opencv_img.html#id32

import cv2
import numpy as np

import sys
args = sys.argv

if len(args) != 2:
    print("too few argument.")
    sys.exit(1)

img_rgb = cv2.imread(args[1])

img_gray = cv2.cvtColor(img_rgb, cv2.COLOR_BGR2GRAY) #To grayscale
img_gray = cv2.Canny(img_gray,10,20) #Edge detection
img_gray = 255-img_gray #Black and white inversion
img_gray = cv2.GaussianBlur(img_gray,(3,3),0) #Blur a little

cv2.imwrite('output_gray.png', img_gray)

for i in range(11,20+1):
    pattern_filename = "pattern"+str(i)+"x"+str(i)+".png "
    template = cv2.imread(pattern_filename, 0)
    w, h = template.shape[::-1]

    img_kensyutu_kekka = cv2.matchTemplate(img_gray,template,cv2.TM_CCOEFF_NORMED)
    threshold = 0.3
    loc = np.where(img_kensyutu_kekka >= threshold)
    for pt in zip(*loc[::-1]):
        #cv2.rectangle(img_rgb, pt, (pt[0] + w, pt[1] + h), (255,255,255), 1)
        cv2.rectangle(img_rgb, pt, (pt[0] + w, pt[1] + h), (255,255,255), -1)
    cv2.imwrite('output_progress_'+str(i)+'.png', img_rgb)

cv2.imwrite('output_result.png', img_rgb)

cv2.imshow('window1', img_rgb)
cv2.imshow('window2', img_gray)
cv2.waitKey(0)

cv2.destroyAllWindows()

result

Process images with mosaics in various sizes below. 000_pic8_many_mozaic.jpg The image is from http://gahag.net/011032-cat-sunflower-smell/.

The result is below. output_result.png

I was able to fill in all the areas where the mosaic was applied. The extra parts are also filled in, but it will be complemented anyway, so it will be OK.

I am satisfied with the results of the other images as well.

Reference material

--How to use the Image library http://qiita.com/suto3/items/5181b4a3b9ebc206f579

Recommended Posts

Detect mosaic points in the image
Detect folders with the same image in ImageHash
In the python command python points to python3.8
Cut out A4 print in the image
What is wheezy in the Docker Python image?
Determining if there are birds in the image
Determine the numbers in the image taken with the webcam
I tried to detect the iris from the camera image
[Python] Get the numbers in the graph image with OCR
Convert the image in .zip to PDF with Python
Image format in Python
Try loading the image in a separate thread (OpenCV-Python)
Image uploader in Flask
Image normalization in TensorFlow
Python OpenCV tried to display the image in text.
Find the average / standard deviation of the brightness values in the image
I wrote a script that splits the image in two
Download the file in Python
Find the difference in Python
Dot according to the image
Remove the frame from the image
Pipenv Simply the main points
Tweet with image in Python
Methods available in the list
Image Processing Collection in Python
Image addition memo in reportlab
Detect keystrokes in python (tty)
The image is a slug
pyenv Simply the main points
[Python] Solving the import problem due to the difference in entry points
I tried to process the image in "sketch style" with OpenCV
I tried to process the image in "pencil style" with OpenCV