Fix your AI-limited mindset in 12 mins

Caltech's Professor Anima Anandkumar states, "A lot of these AI tools are getting better, but you still need to provide AI what to do." So how can we discover the questions we want to ask AI in our lives? Professor Anandkumar, who was a Principal Scientist at Amazon Web Services and a Senior Director of AI at NVIDIA, explains how she draws out creative thinking from her students at Caltech. She also shares her personal journey of developing the mission to "solve real-world scientific problems through AI," providing hints on how we can find our own mission. Through this interview with Professor Anandkumar, who has researched AI for over 15 years, consider what attitudes and abilities we need in an era where AI is increasingly advancing!

I'm Anima Anandkumar. I'm Bren Professor at Caltech.

I've been a professor at Caltech for about eight years now, and during that time I also had stints in industry. I was Principal Scientist at Amazon Web Services. At Caltech, I lead the AI and Science Lab, which means working on some of the hardest challenges we see in science and engineering, and how we can not only use existing AI methods to solve them, but really develop new ones. I think one job that will not be replaced by AI is the ability to be curious and go after hard problems.

For a lot of students, there is a strong motivation to just conform and go ahead. The number one thing I would ask is to question everything and think critically. I always begin classes by asking questions, not writing down math equations. Not going into the details, just intuitively, based on everything you've seen and done: what do you think?

I ask them simple cases. For instance, if you were to design a fire alarm, when should it say there is fire or not? That's something you can change. You can put a threshold on what level of smoke it detects, or when it decides something is smoky. That's a very practical question. If you just had fire alarms going off every day, we'd all be out constantly, and it would be useless. We would not have a functional office space.

But on the other hand, if it never sets off for fire, that would be bad too. So how do we balance this, and how do we model how noisy this sensor could be? Sometimes I see students who may not have the mathematical training, but they're very intuitive and practical. They might say, "I would go and measure how noisy it is," or "I would show different levels of smoke, use candles of different sizes, and go test it."

Many times, people are already somewhat aware of this, so they have some intuitive ideas, and that's already a good starting point. Sometimes they're really wrong. They have an intuition, but it's a wrong intuition, which is still okay, because intuitions are not always the only answer. I think a lot of it comes from asking questions, and now you can use AI tools to get answers quickly and also verify them.

A lot of it comes from being curious or interested in one specific topic. If somebody's interested in music, they can delve deeper into that; if somebody's interested in art, that spark has to come from within. Giving students the freedom to pursue where they are passionate, where they have a spark, is going to be the future. That's the right thing, rather than forcing everybody to learn everything.

I'm always motivated by the hardest challenges. I want to know what is difficult, but also why it's difficult. Even though I may not be able to solve it today, how do we build up the foundations to get there?

Growing up in Mysore as a kid, I loved solving math problems. I'd go to my parents' factory and read their program manuals. I was learning how they could be programmed. Unlike in other computer programs, here, if something was wrong, that would lead to a physical failure, parts not being manufactured correctRly. I remember thinking, "But how does it go into the computer, and how does the computer tell the machine what to do?" There were always gaps, because as a kid you don't know everything. But to me it was about observing, then understanding what the gap was. Even if I didn't get an immediate answer, I would remember that there was a gap. Later, when I was introduced to those topics, in my mind I'd think, "Oh, that's what it relates to." Somehow I had built up a mental map, with places for things I knew and things I didn't know, and I kept growing that as I progressed through the years.

When I was growing up, AI was considered science fiction. Naturally, there were lots of science fiction movies that I saw and was fascinated by, but that wasn't something people thought was practical. Since I was in middle and high school, and now it's almost 30 years. That's a long time, but the amount of progress that has happened is also astounding in so many ways.

Since I joined Caltech in 2017, the timing felt right to use AI as a tool and a framework to solve some of the hardest problems, which until that point were not considered practical. After I joined Caltech, I wanted to explore problems at the intersection of AI and science. I was talking to everybody on campus. I'd ask, "Do you need compute? What do you need it for? Let me understand the problems you're tackling."

I can't possibly go and solve each of these problems myself. My question then was: are there general tools we can develop that could impact many different areas? That put me back to mathematical foundations, because a lot of these real-world phenomena are modeled by partial differential equations.

Could we design AI that can solve this, and do it much faster and much better than what is currently being done with traditional simulations? To do that, we developed neural operators, an AI technology we invented that's trained to understand physical behaviors, not just high-level reasoning with text. 

Think of a hurricane. If you just eyeball a hurricane, can you tell where it's going to go? Most humans cannot. It's a superhuman skill to predict where hurricanes are going to go. To do that, we need fine-scale information and fine-scale modeling. This can't just be a coarse-scale image, like an image of a cat, where even if it gets blurry, you know it's a cat.

The same techniques don't work for phenomena like hurricanes. Once we developed the tools, the next natural question was what the practical use cases were. Weather models were a natural one, because they're widely used and have huge implications on our lives, especially for extreme weather events like hurricanes. If we get them right, that has the potential to save human lives and also bring down economic costs.

I was motivated by how it can be helpful to people, but I was also motivated by it being considered a very hard technical challenge. In fact, a few months before we released our model, a group of well-respected weather scientists published in the Royal Society Journal, under the impression that AI would take more than a decade, or even longer, to replace traditional ways to forecast weather.

They felt AI was not ready, that this problem was way too difficult. We released our model, and it took everybody by surprise. It was not only accurate, it was tens of thousands of times faster. What would take a big supercomputer for traditional weather models can now be run on a local gaming PC with just a consumer GPU. That's the beauty of machine learning as a field.

My mission is to constantly be curious and learning, and not assume that any problem is easy. We're not always stopped by what others think is difficult. As long as we can get the data and design the methods, we can go and try it.

I can't imagine a world where scientists will be out of jobs, because the definition of a scientist is somebody who tackles open problems. There are harder and harder problems to tackle.

If you want to look at the deep secrets of our universe, go down to the smallest scale and understand, at the atomic and subatomic level, how matter is constructed, up to the scale of galaxies and beyond, and understand how the universe is put together. There are still lots of open challenges. Many other teams, such as Google DeepMind, focus on what's known as an "AI scientist."

Meaning AI that comes up with new ideas. But so much of scientific progress isn't limited by a lack of new ideas. Lots of people have lots of ideas. The bottleneck is going to the lab, or going to the real world, and testing them. That's slow, and that's expensive. My focus is how we can replace those lab experiments.

Can we come up with AI that inherently understands the physics better, so we can completely avoid lab experiments, or only use them for final testing? With that focus and that physical knowledge, we can come up with AI-designed answers we can take directly to the real world, minimizing the need for testing.

To me, human agency is driving AI to do what you want it to do. You have the agency, as a human, to decide what tasks AI does, and then you're evaluating and you're in charge. You go and verify whether what AI is saying is true, and then, over time, provide that feedback to AI and make it better.

For young people, my advice is not to be afraid of AI, or worry about what skills to learn that AI may replace them with, but really be on that path of curiosity. Use AI as a tool to drive that curiosity, learn new skills and new knowledge, and you can do that in a much more interactive way.

Even when it comes to writing computer programs, a lot of these AI tools are getting better, but that only means they can do a certain set of instructions which are seen in data. You still need to provide AI what to do. You still need to be able to describe it, and that ability, to describe what tasks AI should do and have a high-level understanding of what AI is doing when it writes these computer programs, is still important, because a bad programmer who is not better than AI will be replaced. But a great programmer, who can assess what AI is doing, make fixes, and ensure those programs are written well, will be in more demand than ever.