Friday, September 9, 2016

Sunset or Sunrise - Machine Learning Project

Project Status: Still Dreaming


It is my hypothesis that given enough photos of sunrises and sunsets and applying machine learning techniques a neural network could be generated that could accurately predict if a given unlabeled photo was of the sun rising or setting.

The Method

Grab one photo of the sun which we know to be either of it rising or setting but we don't actually know which state at the moment. This photo will serve as our input. Once we finish completely with this photo we will keep repeating our method with as many new photos of sunrises and sunsets as we can find.

Break this photo down into feature sets. Feature sets are anything notable we can algorithmically find in the photo. Actually features don't even need to be that notable, the more distinctions we can find, no matter how seemingly unrelated, the better. These aren't distinctions such as this photo has a tree in it or not, but rather small distinctions such as here are tiny monochrome versions of the edges found throughout the photo that make up the larger objects.

Repeat this feature finding process on each feature we found above in the same way we did with the original photo. Then do this again for each of those until we have several layers of features to look at.

Based on the probability of each feature being seen within an image of a sunset or a sunrise, make a guess as to whether this given photo is a rising or setting view. We assume here that the photo given will always exist in one of these two groups but not both.

Wait, what? You can't just bundle up all the decision making into one throw-off statement like, make a guess. That's cheating.

Ok, you busted me. Honestly, it is cheating a little bit. So far we have only used one image and we have no prior knowledge of sunrises or sunsets. When I say "based on the probability of each feature" we actually don't know what those probabilities are yet. Ultimately we just had to take a shot in the dark on that one and we have a 0.5 chance (a 50% chance because probabilities are often represented with numbers from 0 to 1) of being correct or wrong.

Now that we made a guess we need to know if it was correct or not so that we can learn from our mistakes and reinforce what we find to be true.

To find if we were correct, let's pull the metadata from the image. Images have hidden text fields which store extra information about various things. If we pull the time and date (hopefully the timezone will often even be included) we should be able to known if this photo was taken early in the morning or late in the evening. To help strengthen our trust in this metadata we should also pull other fields when they are available such as the location. Even the filename could stand to help, as the owner may have named it "sunrise.jpg" or something that can help us lean one way or the other.

Now we have a pretty good idea if the photo was legitimately of the sun rising or setting and we can see how our guess held up. If we were correct let's retain that knowledge by bumping up our probability value stored for each feature found in the photo that lead us to believe this was whatever the correct answer was. Similarly, if we were wrong let's learn from that also by slightly lowering the probability value stored for any features that lead us to the wrong answer.

If these are still newly found features, then we have not set a probability to tweak in either direction. What we have to do with no prior knowledge is randomly set a value to start with and then as we learn from being either correct or wrong over time the value will settle into place and we will begin making more accurate guesses and less shots in the dark.

In Summary

I honestly do not know if it is possible or not to accurately label a photo as a sunset or a sunrise by applying machine learning and neural networks in this method. It is my prediction that giving a machine enough photos it could eventually find patterns which are not readily noticeable to a person looking at the photo.

Maybe as the sunsets its colors, glow and the way it untimely interacts with other objects including the camera lens differ in some discernible ways from when it rises in the morning. I do not know, but I would love to setup this project to find out.