Yes This time it is about Shazam: We all love shazam and we wonder how it works right. Let me give you some clue. It just gets the finger prints of songs and they are so small and efficient any part of the finger print would reveal which song you are listening to. The finger print of the songs are really very very small and in an in-memory database you can query 27 million songs (Probably it is much larger as of today). Without in-memory technology it would be impossible for these 300 million users to search for what they are looking for. Every week they are adding an other 2 million users into their user base. The question is not just a fast search but during any massive event half of their users might query the database right in the same moment. Them it is also about scaling with number of users. 10 million searches a day. On the other side of the equation you might have Cortexica. They want to be the shazam of fashion or Shazam of any other visual search. They have some benchmarks at their web site. You might wonder the difference between google image search versus contexica. See this one.
Now the question is why are we talking about those web sites? All of us have heard (at least if you are reading my blog you are probably) about “BIG DATA” problem. Everybody talks about it but very few have a real idea what the problem is. Let me put it in simple context. Without some disruptive approaches neither Shazam nor Cortexica can solve their business model problems. With the available technologies can Salesforce.com track 500 million tweets and follow 1 million key words and give a meaningful social sentiment analysis? How are we going to solve this problem if we are doubling the size of the problem twice every 5 to 9 months. By moores law we should be there in terms of CPU power right ? NOOOOOO we need to grow our processing power at least a factor of 4 to catch up with the requirement. Imagine we have not started the image and video search in all dimentions. SO the big data problem is about what we can not DO, rather that we can DO. Today most of the “BIG DATA”guys are into the game of “what we can do” today? The real problem is “what we can not do?” today and the worst thing is “what we could never achive, if we keep the moore’s law”
Think about that once more? “What we could never achieve if we can not beat the moore’s law?”
What a silly question “moore’s law is moore’s law and it proved to be correct over a significant time. It is physics. Right? If we could have , we would have ????? ARE YOU SURE? Think about what drives moore’s law. Is it really physics or is it the investments and ROI on investments, CPU demand, Over all growth of CPU power requirement ! Aren’t you puzzled with different growth rates? Why does data growth mimic Moore’s law. Or do we invest and build CPU’s just to balance as required?
Moore’s law will change for ever? Why ….. WE NEED MORE and MORE and MUCH MORE processing power ….. We need more computational power over the live data. What drives it? What makes the difference? Is it really big data or is it a side product?
The question is “ What gets hot while you are living?” ???? While walking, watching TV, Talking to your friend, during your daily life. What is HOT is hot!