Your browser (Internet Explorer 6) is out of date. It has known security flaws and may not display all features of this and other websites. Learn how to update your browser.
X

Milestone 3 and Milestone 4

Milestone 3 contains the Milestone 3 binaries.

Milestone 4

I am an idiot for not thinking of this before!

It’s 2 days before the 3rd milestone. I am making my program prettier and making it so I can show off more things easier. I can now show what the picture looks like during different stages of the program and very easily showing off exactly what I am doing.

The thing is… this would’ve be highly useful to me quite a bit earlier, if I had focused on getting this kind of things like first two weeks, I would’ve been able to much easier get my stuff rolling.

invariantsscreenshot

 

I can see the picture at ANY stage, even during last stage, where it’s invariant numbers. It took a bit work to get text onto a picture, but with this I can easily show things off! It’s very useful for debugging though, to see whatever happens at any stage. I wish I had done this earlier.

I can even show off my super awesome fouriers that doesn’t do anything at all! That is irrelevant though, because they looks just impressive.  Now I just need to make a power point for the presentation and make sure I can run the program on the computer in the room where I am gonna do the third milestone presentation.

PUUUUUUSH!

It’s the final push before time is up. I have not managed to do nearly as much as I wanted to. I haven’t touched searching at all, my comparison is unsatisfying, and it’s only 3 days left. I have come across one of the core problems, the contour detection.

In simple pictures it work fine, in more complex it does not.

screenshotlist

 

First picture it looks great. I get a good contour of the picture and it’s EXACTLY what I want. Second picture is more tricky because of its weird colours. The contours get a bit messy, but the outer contour is still good so it’s still usable for comparison.

Third picture show the real problem. While it might look decent, 4th and 5th picture shows what really is going on in the picture. It’s not one big contour, but several small that looks connected, but if you look contour for contour you see they are not one big. This makes it completely unusable for shape comparison. To fix this I would need to use better parameters for contour detection OR make my own. If I was to improve my parameters I would have to do it smart someway so it tests several options and goes for the best one. This would be very expensive since my code is everything but perfectly efficient.

I have also done some look into values of the invariants(Thanks Dr Johannes for helping me out with that) and they seem to be bigger the more complex the figure is. The numbers vary quite a bit and noise/imperfect contours generally messes up any chance of getting a proper comparison done. Because the level of noise/bad contours the invariants generally just work so so.

 

I have yet started with colour comparison considering it’s only 3 days to third milestone I am not sure if I start with that before it. I might do, but I also might just improve my GUI for easier doing debugging and showing off the program, and adding different kinds of visual things, so it’s possible to show what happens during different steps of the program.

Fourier gave me tons of trouble, OpenCV gives me no debug info at all if something goes wrong. If the picture is in the wrong format, is too big, too small, have weird name, transparency the wrong way, uses wrong type of colours or anything else like that, it crashes. It gives an assertion and a memory location, but nothing else. Nothing to see in calls tack, nothing to step through to. Just a crash with no info at all. It’s extremely frustrating, and since I know so very little about the fouriers, I get that quite a bit, I have to guess what I am doing wrong. I have managed to get out a fourier picture, but I don’t know if it’s in the right format to work with or anything like that. I am unsure if I will do anything fancy with my fourier, or if I just focus on get something fast and easy done with colours and then work more on showing off what I have.

So many problems, so little time.

oooh look a fourier

Close to the end of the project! Look finally some in-program fouriers!

fourierscreen

 

I will be using this to preferably accomplish some cool stuff and improve my shape recognition.

Same is similar

I can now take two pictures that is exactly the same, and tell that they are similar. The comparison is currently a mess I will have to make it somewhat smarter.

Currently I have a big pile of issues.

#1. Comparing shapes in pictures instead of comparing pictures.

#2. Pictures with varying number of shapes, how do you say they are similar?

#3. Different pictures have different ‘perfect parameters’.

 

#1 and #2 probably partially will be solved when I added fourier descriptors to the picture. #3 will be increased the more methods I add. Right now if I take out the contour of the picture with wrong parameters I will obtain tons of noise. I will probably have to improve the method for getting a countour, and/or I will have to dynamically find the best parameters. I have an idea for how to get the best parameters.

If I take out the contour with like 10 different parameters, then I just take the one with least amount of contours, more contours generally mean more noise.

 

The bigger problem is the Hu Invariants. Comparison done with them is a bit interesting. The seven hu invariants is seven float number somewhere from like 0.1 to like 1^-70. The numbers vary quite a bit with the shape, and the magnitudes are so vastly different. When I compare shapes I compare them using the numbers, the magnitudes are so extremely different though, so it’s a bit difficult.

Close but no cigar

Right now I am fairly close to having a simple Hu invariant comparison in. I have slight problems doing the comparison of Hu invariants done. Right now I take out central moments, get Hu invariants from that, but I haven’t done any comparing using it. I am not entirely sure about the number spreads and how they scale.

Probably it’s like a day of work off doing the comparison. Maybe on monday I can show off some simple stuff.

 

In less bright news I have reached a problem. Right now I take out contours of every shape in the picture, and it doesn’t let me compare a full picture but instead I just compare the shapes in the picture. Right now I am just gonna compare each shape in both picture to each other. But it’s still a very clumsy solution, and I don’t have a good one yet. Partially it will be fixed with fourier descriptors of the picture, we’ll see later on.

Hell! It’s about time!

Finally it’s time to show something off. No more boring talk about things happening too slow and that I work behind the scenes with nothing that can be showed off. Now finally something to show off beginning of the work and how the program will do its comparison. I haven’t done the transforms myself, I am using OpenCV for this, but now I can practically show it. I expect by end of week have a “simple” image comparison to show off, using Hu Invariants.

BeforeAnything

First this is how the picture looks like before anything is done with it. It’s my pretty debug picture I talked about before.

greyscale

This is how it looks like after I have transformed it into grey scale instead.

invertedThreshhold

The grey scale picture is then turned into a binary image, by using adaptive thresholding. The thresholding adapts pixel for pixel, and compare it to it’s neighbours for a thresholding. It gives off some sort of contours, and removes noise in the picture. Probably lots of work can be put here to ensure the thresholding have good parameters or even tests a few different for best result.

contours

The binary image is then used to take out the contours of the image. The contours can then be used to create Central moments and with that Hu invariants. Using Hu invariants two shapes can then be compared to each other, independant of roation. This steps shown off right now is just the basic first steps, required for more complex things, and does not really show off any work I have done, it’s just graphical and thus is easy to show. I hope that by end of week have something more exciting to show off that shows off something I have actually done and not just showing that I can make a library run.

Screenshot

currently

 

This might not look too exciting, what this is, is that the picture is loaded in through a QT gui, sent to OpenCV, handled by OpenCV, then converted to QTs format and printed out on the screen.

The picture on screen is 512×512 and is a test picture to see if anything weird happens , and if something weird happens it’s easy to identify what is happening.

Base is done.

Basics of the program is up and running. There isn’t really much to show, since it’s pretty boring behind the scene stuff.

 

I have stumbled upon some problems here and there, and realized my lack of experience slapping me in the face a few times.

Slow week

Monday was planned milestone, but my presentation got delayed to Tuesday. From Wednesday on I have been starting to lay down the ground work. I have decided to use OpenCV, since it gives me so much help manipulating pictures.  The boredom of making a library work, slowed down the pace slightly. I am not very good at doing things I don’t find fun, but with some slow but steady struggling I now have OpenCV up and running in my program.

Next week I will finally get to implement my algorithms, and I can see how well they will work in action. That will be the make or break of it. I will see how well my theoretical idea works out in the end. I expect some things will pose problems, and I will notice what my algorithm will handle and what it wont. By end of next week I will probably have something up.

Milestone 2

Milestone two is done now. It’s supposed to be proof of concept, but since my project was more research centric, I didn’t really have a proof of concept but more like a concluded research.

Milstone 2

Conclusion of the week.

Like Rebecca Black says; “It’s Friday”. The work week is done, and on Monday there is the second milestone. I will discuss and tell about the techniques I have decided to work with now.

First off I will be using a combination of Fourier Descriptors and Hu Moments to describe my pictures. To use them I will  find contours of my picture and then I will use HuM/FD to describe the contours. Then using the HuM/FD I will compare it to another picture. Comparisons done that way will ignore rotation, translation, skewing, and scaling. Doing comparisons using HuM’s is very easy and can be done fast on already pre processed pictures.

Step by step:

  • Take the picture turn it into greyscale since colour is irrelevant.
  • Find the contours of all shapes in the picture
  • Turn the contours into fourier descriptions and Central Moments, which then is turned into Hu Moments.
  • Compare fourier descriptors and Hu Moments between two pictures.

Compare colour by taking a histogram of picture and by mixing together pixels into bigger groups and compare those. Colour is secondary to shape and will just be a nice extra feature, for slightly better results.

For searching the Hu moments will be used as a low cost comparison, and a general colour of picture as well. I will probably discuss more details about searching the future since right now main focus is to get single picture comparison to work. I still have to figure out how to compare pictures when there is very many shapes in the picture in which order to compare them etc.

Third and last week of research

Being in the middle of my third week of research. I finally found the grand father of papers on the topic of image comparison and pattern recognition; “Visual  Pattern  Recognition  by  Moment  Invariants” by Hu Ming-Kuei from 1962.

In this paper he makes mathematical proofs for and explains how his 7 invariants works. They are usually used for pattern recognition independent of rotation, translation, mirroring and skewing. Pretty much every paper on the topic refers back to his paper. It seems to be a cornerstone, and while I haven’t fully grasped how to use it and how to combine it with other things, what seems to be a strong solution for my problem is to combine his invariants with  fourier transformations.

If things works out as planned I have a working solution both for comparison and searching. End of week I will probably write some more details about everything and some sort of conclusion of my research. I will write about my solution and explain how it works a bit then.

Tons of research

After a initial week or writing documents, and doing some research, my research was a bit lacking, and what I had so far was a bit lacking. The nature of the project also means it is in need of a longer period of research than just one week. The week have been spent reading papers about image comparison, Fourier transformation and uses of it, and looking at Google image search. Also I will most probably be using OpenCV to not have to ease my workload a little bit and focus more on the algorithm.

Fourier Transfrom

Showing off a pretty Fourier transform. For now I just leave this in the air, I will most probably be using Fourier transformation, and I will talk more about them when more details are fleshed out.

Making an algorithm that can take any image and compare it to any other without any restriction probably is too difficult to accomplish during this 2 month project, so I will need to put down some restrictions. Focus will be on comparison of shapes in pictures IE a grey scale picture(or conversion to that). And then a simple colour comparison will be run by using a histogram of the colours. Comparison will also be restricted to pictures of same size, but hopefully it will be able to detect rotations. This is the initial plan, next week will focus on how to do the shape comparing.

Kudos to Dr Johannes Hirche for input and feedback on project, and for suggestions for changes.

Milestone 1

Milestone 1 and the presentation about the research for the project is now done. Link to milestone 1 contents.

Milestone links also appears on the right side.

Start of project

This is the start of the specialization project! I will post here weekly about the progress, show some screenshots and post documents so they will be accessible.

I will be creating a program to compare two pictures to see how similar they are and eventually make it possible to search in a db using the comparison algorithm.