Why the Smartest People in AI Stopped Using Browsers

In this conversation, Abhishek shares why he left Meta to found Yutori, why he believes the industry has dangerously normalized broken AI products, and the precise philosophy behind building agents that actually work the first time.

Abhishek Das: My name is Abhishek Das. I'm the co-founder and co-CEO of Yutori. With Yutori, we're building agents that can take actions and complete tasks on users' behalf on the web so that you can focus on whatever is most meaningful to you. The three co-founders are all AI researchers by background. It is a bigger bet than just another agent company. So Fei-Fei Li and Jeff Dean were excited to support us.

I come from a family of doctors and medical practitioners. So it was a bit of an irony that I'm scared of blood. The choice was pretty clear that I'm not going to pursue medicine. That's when I ended up deciding to pursue engineering.

The scientific process really appeals to me. The whole lifecycle of coming up with a hypothesis, then designing experiments to validate or invalidate those hypotheses, drawing conclusions, and then coming up with the next set of hypotheses. I found that really inspiring.

Abhishek Das: I went to IIT Roorkee for my undergrad. That was an amazing learning experience. I met some of the smartest people I know. First year of college I was getting good grades but very quickly I realized that electrical engineering, especially geared towards power systems, was not where my interest was. At the end of first year that was sort of the first major rebellious streak in me where I decided that I don't see a future in electrical engineering and I'm going to stop paying as much attention to it.

Instead I ended up spending a lot of my time learning programming and how to build software. IIT Roorkee had, even at the time, this was almost 13 or 14 years back, a really strong programming club and culture. In particular there was this group called SDS Labs, a group of 10 to 15 coders from every year who were just tinkering and building a ton of applications for the rest of the campus.

Seeing how users use something you built from scratch and put out there — that was a dopamine hit that kept fueling this. I would stay up nights to add features and improve things, especially because I was surrounded by people who were as obsessed with this stuff as I was. That was extremely motivating.

Abhishek Das: To be honest, I wanted to start something of my own for as long as I can remember. I had strongly considered it at the end of my undergrad, at the end of my PhD, and for various reasons didn't end up doing it. So it was just a matter of time. The main reason is that there are lots of interesting problems in the world to go after, and I did want to push on a vision I care about as opposed to working on somebody else's vision.

Over the last two or three decades, web browsers by and large have stayed the same. We open a browser, we open a web page, we click around, scroll, type stuff. There is an opportunity now to reimagine what that experience looks and feels like. We're going to be talking to AI assistants that take actions and complete tasks on the web. And a lot of it is going to be agents that work in the background in a proactive manner for you. That's what the future looks like.

Abhishek Das: It felt like before physical agents become a reality, digital agents will become a reality. The timeline for digital agents is shorter than for physical agents. If you think about interacting with the web maybe 5 to 10 years in the future, it is going to be at a slightly higher level of abstraction. Instead of us having to do digital chores ourselves manually, it lets us focus on tasks that are more meaningful and more interesting to us.

If we can delegate all the mundane stuff to AI agents on our behalf, it lets us focus on stuff that's more interesting. So it's more like humans and agents working together to overall improve productivity rather than agents replacing humans entirely. Part of it is also just making it accessible to more people. My parents, for example, no longer have to learn every new website and how to operate it. If they can just tell an assistant what they want done on a particular website and it does it for them reliably, that's awesome. So it makes it more accessible for more people.

Abhishek Das: In this day and age, there are basically a hundred different agent products out there saying that this can do anything on the web, and you try it once and it doesn't really work. There's also this notion of "it usually works" — like if you try it 10 times maybe three or five times it does the right thing. I push back on that getting normalized.

Agents are making a sequence of decisions. If we think of a 10-step, 20-step or 50-step workflow, even if the accuracy at each step is like 90%, the 10% error rate compounds very quickly. The overall success rate of a workflow is quite low. That's one of the reasons why the technology is not there yet to do long-horizon workflows.

Abhishek Das: Being able to recognize when it makes mistakes and backtrack from that to go down a different branch is really really important. We put in a lot of effort into building evals and guardrails. Every single production query that a user runs goes through a fairly comprehensive set of evals that lets us quickly identify where these agents are doing well versus not, and which domains need more work.

Because we're in the space of web agents, it will never be the case that we will be able to train on every single website that exists. There are new websites coming up all the time. We will always be training on a finite set of websites. People make mistakes on new websites all the time — clicking the wrong buttons, etc. So it is very natural to expect models to also make mistakes. But when it makes a mistake, is it able to recognize that and then backtrack and correct itself? That is a fairly important ingredient in the recipe of how we train and build and ship these models.

It feels like we have started normalizing and developed a tolerance for non-determinism and low reliability in shipping products. I don't like the normalization of slop and poor reliability especially with agentic products. If it's not good enough to work on the first try, it's not good enough.

Abhishek Das: We take sort of an 80/20 approach to it. There is always the prioritization question of, okay, there are 100 features we could be building. What are the top 10 we need to focus on? Some of those are informed by users and what they're asking for. But very often there are ways to build product that users may not be asking for. But if you build it, and a lot of intuition goes into identifying what those features might be, then users feel seen and they feel like someone is listening to us. Even though that's not exactly what they asked for initially.

I'll give you an example. The feature on iOS or Android where anytime you get a two-factor authentication SMS, it auto-reads your SMS and fills it into whichever app asked for it. It is hard to imagine a user asking for that feature. But it saves a few seconds multiple times a day for people all across the world. It's a tiny thing that makes users feel seen, like someone is actually giving thought to how to reduce these tiny paper cuts in our day-to-day life. So it is a marriage of intuition with what users are actually asking for.

Abhishek Das: In a world where it's very easy to come up with first prototypes using coding LLMs, the true differentiator is in taste and craft in how intuitive and well-designed the product is. One thing we do in the team that helps with that is we take dogfooding our own product very seriously. Every single week we have an hour to an hour and a half blocked out for dogfooding new features. At any given point we're running like tens of experiments internally and maybe one of them will ship to the production version of the product that external users will see. Constantly dogfooding our own product is a way to refine our own taste for what is good versus bad, what awesome or magical feels like. Like anything else, a lot of reps to build that muscle is one way to go about it.

Abhishek Das: The Grad-CAM project was led by one of my labmates. I was a supporting author on that paper. I was 25 when we did it. It's been extremely well-received. I think 20,000 to 30,000 citations is quite non-trivial. Interpretability in deep learning models was a big area of focus and still is to this day. It was motivated from the idea that classification models which go from images to one of a thousand or 10,000 categories — what part of the image are they actually looking at to make those predictions?

With AI models, it is important for models to convey not just the final prediction or the final answer, but also the proof of work. What are the steps that went into arriving at this final answer? Grad-CAM is one manifestation of that. But even in how we build the Scouts product today, you can set up scouts and agents to monitor the web for something and they will generate these reports and notify you when they find something of value to you. But there is a button in the UI that lets you inspect the work that went on behind the scenes. Which websites were visited, what did the agent actually look at to pull out this piece of information. That gives you a glimpse into the work that went into putting this together. It is very very important for trust building. For users to trust that yes, this is a reliable product.

Abhishek Das: A lot of our time and attention in how we're building our product at Yutori goes into thinking about how we should build so that we don't make the same mistake twice. Whenever we ship something, we have to get it right. It has to really work. It has to be reliable. Users have to trust that it works well.

If we put attention to detail into parts of the product that users can see, then the user is more likely to trust the parts of the product that they cannot see. Everything awesome that we see around us was built by individuals or groups who put in a lot of hard work and attention to detail. We should approach everything we are building with that kind of philosophy. It takes time to build something meaningful, to build something right, to bring a vision of the future to life. Building delightful and reliable product experiences doesn't just appear out of nowhere.