Big Data and AI
April 22, 2015
Written for Canadian Community News by Mike Sterling
To Comment on this article Click Here
Although the lay person rarely hears the words 'Big Data', it is the Holy Grail of Cloud based applications.
There is a race. In the running are Microsoft, IBM, Google, Apple, Wolfram and more.
Google is trying to capture all the data that exists. This includes everything already indexed on the net, but also how humans interact.
The latter can be such trivial things as thermostat information gathered real time. Ah, you don't believe me? Google can't think that's important enough for their attention? But, oh yes, your home is where they want to be. On your wrist, what you see and how you feel.
Google just bought Nest for $3.2 Billion Dollars US. Take a look at the company Click Here Not much there for all that money
Google wants your home data and they will grow the Nest product line from thermostats to smoke alarms and how your frig is doing. Of course your car is important.
I know you want an inventory control system on your frig and the latest flyer information above the ice dispenser, don't you?
Of course the others are wanting your data too. They think they can sell you things and promote products based upon what they perceive you like and do.
There are some really serious uses, however. Not that the others are not serious, but listening to the latest iTune on to your watch pales in importance to Cancer research.
This is not just search engine stuff like Google and Bing from Microsoft. It's much, much more.
Rest assured that the big players want to know more and more about your personal life and habits. That's a heavily scripted play that is in more than three acts.
The real story is IBM and companies like Wolfram Research. IBM has Watson and Wolfram has Alpha and Mathematica.
Why does Warren Buffet now own 8.7% of the old giant, IBM?
He is a value buyer not a person who buys stock based upon the current market value or perceived value. He is and has been an investor who does not want to reorganize a company. He wants to grow with it. He does not buy into a company that has a bad product and bad management. He does not buy to break up or change management.
IBM has the most adventurous and interesting strategy and they have Watson, the super computer that destroyed Jeopardy Champions. (Click for Story) The whole company is being organized around Watson Technology and consulting. They go together.
What is Watson's goal? It is to be the first and ongoing base for artificial intelligence. Yes, Watson is what Hal (from 2001 A Space Odyssey) wished he could be.
A Brief Look Back. How did we get here?
From the first use of computers we marveled at their ability to store information that could be retrieved upon demand. Most of the early databases were 'flat' files. They were raw data and not organized. Big sorts had value, however.
Along came companies like Oracle. They made billions giving us a formal database.
Inside the database they forced us to tag information with things like 'First Name', 'Employers', 'Days lost on sick leave'...
Then along came search engines. They recognized that Big Data was really unreachable with old database technology. It was just too complex to label key words and reorganize the complexity of Big Data. Big Data just gets bigger.
You could try to shoe horn your company into Oracle-like databases, but there was always data outside what the originators foresaw. Design, redesign and do that again and again. The reason? Need grows and Big Data gets bigger.
Along came programs that left the data as is and just indexed it for fast searches. The data was flat, but indexed for speed of access.
So, if the original data had structure, search engines use it, but they do not depend on structure at all to get information out and into the hands of those that can use it.
Companies like Open Text, Alta Vista, Yahoo, Google and Microsoft's Bing came along to exploit the idea of indexing and fast retrieval. Some dropped away.
Can you imagine the Internet without Google and Bing? Strangely Apple has not entered the search game with a single product of their own.
The Future is ...
Artificial Intelligence (AI) combined with Big Data Search Engines equals a partner that beats Hal and leads to a tipping point, wherein a non-human becomes a true partner.
Let's take a look at how IBM's Watson works.
Watson uses natural language to 'talk' and 'listen' to us. Recently they added Japanese.
Watson attempts to be context sensitive so Watson hopes to know the play on words in jokes and news headlines and more. That's a difficult part of natural language.
Watson looks for patterns in Big Data. As Watson ages, it gains value, just like Google and Bing do as the information grows exponentially in particular areas that have gained Watson's attention. Watson looks further, deeper and is tireless. As of now Watson is told to investigate an area, but soon, it will range far and wide on its own.
Ever wonder how the search engines guess what you will type next?
Watson is designed to keep track of a gigantic corpus of knowledge. But, more importantly, it is built to 'burrow' into mountains of data for meaning, patterns and connections.
This knowledge gathering is constantly designed to be subject to human and other AI corrections. Human to AI, AI to human and AI to AI are designed into Watson. That is, Watson has a lot of curators, who help find the truth.
Everyone uses Wiki for quick information. If the subject is important enough, it is massively looked at for validity and style.
I once saw a Wiki subject animation showing all the times that the article had been changed by people who detected errors in the original and in the follow-ups. It was impressive.
Watson is designed to work with human curators and field experts to improve or get 'smarter'.
Many Silicon Valley experts think that once Watson-like technology is focused on a subject like health care, patients will be better served than they are by their local physician alone. I am sure this is true. Watson is tireless once unleashed.
We still have a problem, don't we? At our parents knee we were told that success was 99% sweat and 1% inspiration. Watson can out sweat us. But, oh how wonderful inspiration is! It is illusive, though. Can Watson have inspiration?
With Watson we are hoping for true inspiration. What is inspiration and is it a common thing for humans? How will Watson-like AI machines deal with inspiration?
A Personal Inspiration
Can Watson have nspiration?
Like most people, I don't get inspirations very often. Once I got a real inspiration.
I solved an unsolved problem in the American Mathematical Monthly. The results intrigued me. Not much inspiration not inspiration was required.
Over a 35 year time frame, I expanded the results to a modest degree. I felt somewhat trapped, however, as connections and patterns were coming very slowly. Sometimes none for years.
The results were beautiful, but I wanted more. You see mathematicians, scientists and engineers thrive on connections and patterns. That's the nature and essence of discovery.
Years later, I came across a footnote in an obscure 1938 book written by G. H. Hardy the famous Cambridge Don.
It allowed me to extend what I had discovered into a famous formula, the Rieman-Zeta Function. Not much inspiration required.
I was thrilled and wanted to see more. I did some calculations and the results were just numbers. I had hoped for more like something to do with famous constants like pi or e or the golden ratio found in some famous work. I had an intuition that the great Euler was hidden in the shadows.
I tried a single Internet search for the meaning of one decimal number. I got a single result and it showed that the number was beautifully related to something the savant Leonard Euler had done in 1734 or 1735! It was a glorious day for me.
This one discovery led to more and more connections and patterns. One flash of inspiration was simply to try to look up the meaning of a single number. Hardly earth shaking and again 99% sweat.
Could Watson have done what I did?
I think so. Watson would have found the obscure footnote in Hardy's book in micro-seconds and not decades. Yes, it took me decades and lots of luck.
Once found Watson would have added my mathematics on both sides of the equation in the famous Rieman-Zeta Function as suggested generally in the footnote. Watson would have computed the converging infinite sums and products. Watson would have found my obscure single hit.
Watson would and could find much more. Maybe too much for me to sort out, but maybe I could ask Watson to narrow the focus. I might have to sort through all the patterns of Watson's inspiration.
I wish I knew Watson better. Watson gets better with age. I don't.
Look for Watson in health care. Look for Warren Buffet to be right again.
Remember inspiration is not far from perspiration.
Click on the ads for more information
books, sports, movies ...
Thursday, April 23, 2015