The rapid application of machine learning is required for enterprise success; the pace of research application and in particular machine learning measured in years ten years ago now occurs in days in 2016.
A computer mastering the champion Go player — the abstract strategy board game — was to happen in 2026 and not the 4-1 match wins of DeepMind’s AlphaGo beating Lee Sedol in March, ten years ahead of predictions. And with the Internet of Everything, there are enterprises transforming and using the data deluge of more than 44 zetabytes (by 2020) or 44 billion terabytes and machine learning is the key.
Google Inc. recently released to the public TensorFlow, their open source library for machine learning in line with releases from Microsoft, Amazon, IBM, Facebook, Baidu. Google’s CEO Sundar Pichai is opening up their cloud to customers and see their early and continuing investments in AI and machine learning as their edge as they work to catch up to market leaders Amazon and Microsoft. Pichai has Diane Greene leading the charge as their new senior vice president enterprise business bringing her prior credentials as CEO for virtualization vendor VMware.
It seems mobile-first has been replaced with AI-first as the theme today and evidenced with Google’s cloud-based machine learning. Enterprises will benefit from the competition and the wide proliferation of machine learning tools in the cloud. There is a Digital Quake where more than 80 per cent of jobs and enterprises will change by 2030 or FAIL. To succeed enterprises must leverage cloud and machine learning implementations.
This brings us to the interview with Google’s own Peter Norvig, who helped usher in the world of machine learning and has a storied history in the field.
Who is Peter Norvig?
Heralded as an icon in his field, Peter is a Director of Research at Google Inc. Previously he was head of Google’s core search algorithms group, and of NASA Ames’s Computational Sciences Division — making him NASA’s senior computer scientist.
He received the NASA Exceptional Achievement Award in 2001. He has taught at the University of Southern California and the University of California at Berkeley, from which he received a Ph.D. in 1986 and the Distinguished Alumni Award in 2006. He was co-teacher of an Artificial Intelligence class that signed up 160,000 students, helping to kick off the current round of massive open online classes.
His publications include the books: Artificial Intelligence: A Modern Approach (the leading textbook in the field), Paradigms of AI Programming: Case Studies in Common Lisp, Verbmobil: A Translation System for Face-to-Face Dialog, andIntelligent Help Systems for UNIX. He is also the author of the Gettysburg Powerpoint Presentation and the world’s longest palindromic sentence. He is a fellow of the AAAI, ACM, California Academy of Science and American Academy of Arts & Sciences.
To listen to the interview, you can go to the non-profit ACM Learning Center podcasts or click on this MP3 file link in the learning centre
Here are extracts from the full interview:
Ibaraki:
Peter, please share some experiences and lessons from age four to your professional career in computing?
Norvig:
There weren’t a lot of computers around at that time, but I think I did have the right background to get me where I am today. My father was a Math professor and my mother was an English professor so if you combine the math and the words, that takes you to Google somehow.
Ibaraki:
Going on from there can you share some experiences and lessons from your roles prior to Google?
Norvig:
I started out as an academic and was a math major as an under-graduate and when college was over I couldn’t wait to get away, so I worked as a programmer for a couple of years. After two years I said it took me four years to get tired of college and only two years to get tired of work, maybe I’d like school twice as much and headed out to Berkley to grad school. I ended up being a professor for a number of years, but over time I got a bit frustrated in that the size of the project you could do as an academic (particularly as a young academic with not much of a track records) was very limited….I worked at big companies like Sun Microsystems and small companies like start-ups….even the .gov top level domain running the computer science division at NASA. I guess the idea was to find an interesting project and find a good group of people to work with and beyond that a lot of freedom to consider different types of possibilities.
Ibaraki:
You have this rich history – do you have any lessons that you can share from those experiences, for example at NASA (something that still carries you today)?
Norvig:
It’s definitely different working for the government, where the goal is to be a good steward of the taxpayers’ money more than it is to be effective and quickly solving your problems. At NASA in some ways it was similar to a role as a teacher in that a lot of it was communication….My role was mostly around trying to communicate what their work was to the rest of NASA (and to the rest of NASA computer science is a sideline).
Ibaraki:
Right now innovation and innovation colonies and agility are key issues. You’ve got this background in start-ups; do you see any lessons from your start-up background that you can pass on to enterprises that are trying to innovate in their environment?
Norvig:
The way to do innovation is to have good ideas and nurture them. It’s a process of putting together the teams who can get something done with the problem areas, and innovation is trying to do that in a unique way….I think the key has always been a close partnership. It should feel more like everybody working together as part of a team and who are all bringing what they know (some of that is in the new area and some of that is in the existing area), but everybody’s goal is to solve the problem rather than to show off the innovation.
Ibaraki:
Can you talk about how you got into Google, what were your past research interests there and how do they continue to shape Google today and into the future and to the world?
Norvig:
I joined Google in 2001. In 2001 there were 200 people. I was a bit worried that maybe I was too late and they had already done all the interesting stuff. Fortunately they left a few things and the company of course has been growing ever since. I got into it for a couple of reasons: one, it was a good time to make a move….I had known about Google for a while, had run into a couple of people in various roles and it just seemed like that would be the best place to be. I was interested in this idea of doing more with data and Google was the one who had it….So it was the right time both for me and for them to get together.
Ibaraki:
Do you interface with the DeepMind people or Jeff Dean?
Norvig:
These are two separate things. DeepMind is a company we acquired; they are in London now. They work pretty autonomously. I’m in contact and talk with them but on a day to day basis they are pretty much on their own and do a fantastic job….Within Google we have a team looking at these neural net technologies and Jeff Dean is a prominent member of that team. Geoff Hinton is another member (and many others as well) and those are the ones I work more closely with on a day to day basis.
Ibaraki:
Peter, what about your hybrid research interests at Google today and into the future, and what does this mean to enable better business, education and the broader community?
Norvig:
I have a couple of main interests right now. One is around education. We work on making tools for online education for people to be able to put together courses so I’m interested in that, both internally (how do we educate our Google engineers) and externally (how do we educate the Google users about how to use our products) and also once we have built these systems for those needs, we might as well open source them and allow the world to use them as well….My second area is around deep learning and in some ways it’s also an educational effort of making it easier for engineers within Google to apply these machine learning techniques that we have….and if we can do that within Google then maybe someday we can open source that as well and get that out to the rest of the world.
Ibaraki:
You see controversy around machine learning, deep learning statements from people like Elon Musk and others, what are your views on this?
Norvig:
I think machine learning and AI in general are tools; we build these systems and control them and they’ve proven to be very valuable….I think it’s great that people are looking at ways to use the technology more wisely and to think about some possible ways they can go wrong – if we think about that ahead of time then we’d probably end up with a wiser usage of the technology. I’m glad that people are focusing on making it safer and better for everybody all around.
Ibaraki:
Peter, what are the outlier ideas, research, kinds of start-ups that could disrupt three or more domains of your choosing 10-20 years into the future in areas such as FinTech, BioTech, AI, and so on?
Norvig:
I guess one of the areas that is most happening to me is around BioTech and health. It certainly seems like there is a lot going on there. Google or Alphabet’s involved in some of that. You see a lot of other start-ups in the area. It’s not my level of expertise. I never took biology in college, but I’m fascinated by what’s happening and looking for things to happen there….I think robotics is a great area and it feels that we are on the curve of adoption for robotics. We are now where we were with PCs in maybe the late ‘70s or early 1980s where if you remember back then you could buy a computer maybe in kit form or fully assembled and you could sort of run a little game or a spreadsheet on it but you couldn’t really do very much that was useful. I think that kind of timeframe is going to happen to robotics….There are a lot of areas that involve software and not having to build a hardware product and those are easier to get started so I think we are in for an area of a lot of innovation and integration of the technologies that go into our life.
Ibaraki:
Some are suggesting two lines of rapid development, one is in biology with CRISPR-Cas 9 and Genome Editing and the other is with regards to machines and machine learning and deep learning. Some even suggested perhaps thinking about it almost like a metaphor for evolution. What do you see here?
Norvig:
Maybe that’s a good way to think about things, but I just think of it as a tool that’s going to help us and we’ll be in for a change but it’s a very different type of change.
Ibaraki:
Do you have any additional views on CRISPR-Cas9 (that is, genetic gene editing)?
Norvig:
We’ll get a lot more understanding of how the biological world works so I know people are very excited about the applications, but it’s not my area to be able to predict exactly where it’s all going to go.
Ibaraki:
I have had discussions with Tom Mitchell, Andrew Ng, Judea Pearl and others in the AI space, what do you think about Judea Pearl’s work?
Norvig:
He’s a real hero for me. I think he is probably the one most responsible for getting AI to where it is today….What Pearl did was show if there is any structure to the domain then you don’t need all the numbers, you only need limited numbers; in fact we can handle that. Then the second thing that happened was to say now we’ve made this shift from logic to probability, Pearl provided the technology to make that efficient so that was a great change….If you had to pick one person who was most responsible I would go with Pearl as being the one that helped kick start us along that road.
Ibaraki:
Judea has this work on codifying or providing the mathematical modeling for causality, do you have views on this?
Norvig:
If you look at the field of statistics we often get stuck by saying we can easily come up with the correlation, but it’s harder to come up with causation. (If A and B are correlated, did A cause B, did B cause A or is there some other C that causes both of them?). In the raw statistics we say that’s not a question you can answer, but Pearl didn’t like that answer and he wanted to come up with ways in which you can answer it. I don’t think that it’s been 100 per cent successful yet but I think he’s made good progress and he’s certainly shown some ways in which you can make that distinction.
Ibaraki:
You have Pedro Domingos trying to unify the five schools of machine learning through a master algorithm, what are your views on this?
Norvig:
I definitely like his book. I think he’s done one of the better jobs of bringing machine learning to a wider audience (a non-expert audience), so his book covers the whole field….If it were me I probably wouldn’t put as much emphasis on trying to combine them all. He is saying he is looking for this master algorithm, the one algorithm that will do everything. I guess I feel more comfortable saying I’m willing to have the toolbox. I don’t need the one super-powerful Swiss Army knife that does everything, I would be willing to have a toolbox with five separate tools in it and maybe have to think a bit about which tool you want to use for which job. So I think that’s just a question of style….But he’s certainly done a good job in explaining the individual approaches and explaining the importance of machine learning throughout all sorts of different industries.
Ibaraki:
I guess the key exciting area is that Google is trying to produce these tools that can be readily used and this is of high interest and a priority in the enterprise space to have tools.
Norvig:
It’s interesting, it’s a little bit tricky to get these things out there in that it’s always easier to take what you know and apply it to one specific domain than it is to make a general purpose tool that’s good for everybody.
Ibaraki:
You’ve traveled the world, you’ve done a lot of speaking and so on, do you have any interesting stories to share (perhaps something amusing, surprising, unexpected or amazing)?
Norvig:
When I was fresh out of college I was working as a programmer and they would send me off to travel to visit customer sites and so on. It really struck me then that the world might be changing a little because I’d be in the airport and 90 pe cent of the people there were these business men with their briefcases. So it really did look like Man Men in those days and it was very stereotypical. Then on each plane there would be a couple of us (scruffy younger people with backpacks rather than briefcases), and as you looked at them it turned out they were always carrying a computer magazine or something like that. So it seemed like the world was starting to split back then….I thought then, I bet something is going to happen with this and I guess I was right.
Ibaraki:
How about your future?
Norvig:
At Google we haven’t got to the end of the internet yet so there are still plenty of things to do. I’m excited about the projects we are working on now. Also, I had this previous career as an academic back in ’95, wrote a textbook with my co-author Stuart Russell from Berkeley and every seven or so years it’s time to write a revision. It turns out, right now is the time to write a revision so that’s another project I’m involved with now.