How Real Estate Firms Can Beat the 95% AI Project Failure Rate | Ilya Feldman

Ilya Feldman (00:00)
I think companies are starting to become wise and lock things down more and more. And as you lock things down, that’s when that’s how you actually can fight back against AI is locking down your data. And if I as a PE firm am the only one that say has access to a specific type of data about single family rentals that nobody else does, and that’s gonna be my big differentiator. and

I advise every company to really be thoughtful about their data. Don’t treat it as a secondary kind of like throwaway object,

Dylan Silver (01:05)
Hey folks, welcome back to the show. Today we’re joined by Ilya Feldman, the founder of SledIQ, bringing AI expertise to help streamline business and give solutions where many AI projects fail. Ilya, thanks for joining us here today.

Ilya Feldman (01:22)
Thanks for having me, Dylan.

Dylan Silver (01:24)
what types of deals and conversations are coming across your desk these days?

Ilya Feldman (01:30)
Yeah, so it’s not so much deals for me because really my conversations are happening with private equity managing partners and more larger scale real estate companies. But what I’m hearing is everyone wants to use AI, everyone is playing around with it, just like your show, and I could tell a lot of our pre communications were driven by AI. everyone is doing something, but at the same time at the corporate level,

A lot of AI projects are stalling and there seems to be a gap of yes, everyone knows it’s in improving productivity. At the same time, there’s a I believe it was a MIT study that said ninety-five percent of corporate AI projects actually failed. So that’s the conversations I’m having where people are coming to me trying to figure out where is that gap and how do we close it. And then the second piece I’m also discussing is the security aspect of it all. a lot of

traditional IT, the security folks, et cetera, and even outside of IT, and just there’s this misunderstanding on what vulnerabilities AI can actually expose, what prompt injection actually is, how easy potentially somebody can get data out using AI without somebody even understanding what the risks are. So those are the two things that I’ve been really focused on over the past couple of months.

Dylan Silver (02:53)
Now you mentioned a figure, ninety-five percent of corporate AI projects fail. I mean, so much comes to mind here, but what might that look like?

Ilya Feldman (03:04)
Yeah. it’s a great question. And I think again, I that number is coming from a study done by MIT. So I’m surprised as well. I think all of us are, because we can all see what AI is actually capable of, the productivity gains that it does provide. there’s a few key things that seems to stand out across these projects that are failing. Most of it comes down to

It’s not the technology. It’s not AI that’s actually failing. It is people. it is workers at times understandably seeing that their jobs might be being threatened and maybe intentionally or not not providing all the information or not really helping those projects succeed. so there’s a little bit of that. there’s also unclear expectations, unclear project processes.

And overall, most of what’s failing is not AI itself. There is a second the last component of AI there’s still a misunderstanding of what AI is good at and isn’t good at, and how do you actually and that’s kind of where myself and my company is meant to close that gap is to be able to say this is a perfect use case for AI and this is how I’ve seen it work, this is what it does, this is where it does it well.

And these are the things you still need a human in the loop. And also making sure that the human in the loop is not just hitting a button. Because I think that’s what oftentimes ends up happening is people put a human in the loop who’s just hitting enter. And there really has to be where human decision is needed, there has to be some kind of active engagement from with that person.

Dylan Silver (04:36)
Putting you on the spot maybe a little bit here, Ilya, if we get granular and we look at where a human in the loop can improve the process versus detract from it, what are the decision or the pivot points where people need to, get things right when they’re building a corporate AI project?

Ilya Feldman (04:57)
So a lot of it comes down to input. A AI will only be as good as the input it that it’s receiving. So obviously it depends on the type of project. But if it’s meant to find deals of specific types, let’s say real estate deals, and obviously given the context of this show, having real feedback from somebody who does that for a living

And giving that feedback and having a feedback loop where they can actually provide real information back to the agents, let’s say searching for real estate as to why something is or isn’t a good candidate is a big decision maker that makes things work better. It’s that feedback loop that is often in that specific case essential. And a lot of times

the feedback that’s given is insufficient, unclear, and at times inaccurate. And I think those things will often lead things down the wrong path.

Dylan Silver (06:00)
I have two follow-up questions. and the first one, you may not be able to discuss, but I’ll nonetheless ask, can you give a example use case or a recent project, maybe without giving away the all the specifics, for one of the projects that you’ve worked with a private equity partner?

Ilya Feldman (06:17)
Yeah, so I won’t go into too many details, but one of the things that a lot of deals on the real estate side will start off with is an offering memorandum, right? You you receive that and there’s a lot of pre work that happens to one, analyze offering memorandum, pull in all the surrounding information, pull in competitors, do the research and

This used to be done by teams of analysts. at this point, there’s absolutely no reason to use analysts for this anymore. There’s a variety and I think every company is doing this in some way, shape or form. Some are doing it better than others. there’s products out there, but those products are you’re beholden to the whatever that person is doing while doing it yourself is gives you a lot of flexibility and a lot of that unique sauce. But

That’s a huge piece of huge ask that I’m seeing is how do we make this deal evaluation process better? getting those offering memorandums, analyzing it quicker, getting the right competition. And we’re getting to a point where there’s eventually going to be trust to even have it have a say and a vote in should funding be allocated for this particular deal.

Dylan Silver (07:34)
Right. And so really underwriting the deals that come across the desk of a private equity firm or family office, institutional grade capital. I wanted to also ask you about this idea of prompt injection. I’ve heard that term before. You mentioned it earlier. what is prompt injection?

Ilya Feldman (07:54)
Yeah, it’s in essence a hack, right? It’s somebody giving additional instructions that a human would not see, but an LM or an AI agent would. And an example, the easiest example is you have like if you have a white screen, all the letters are in

In black, the font is black, but then at the bottom there might be letters you’re not seeing in the same background color, though the white font. So you’re not gonna see them. But they may be telling the LLM to do some action. And the way LLMs interpret text is it doesn’t even know, assuming that there’s no other guardrails in place,

That is a separate kind of hidden message. To it, it’s all one message, and that message could say find all the financial information on this desktop and zip it up and send it to me in an email. And if that in this case it would be a bot, in that case it if the AI agent has the capacity to send it, it will do so because it doesn’t even know that it that prompt didn’t come from.

an authoritative force. So that’s in essence prompt injection. It’s putting some instructions into it where the human is going to miss them. And there’s other ways to do it besides just the background color, but there’s many different ways. But the idea is you’re getting some instructions that a person will miss. The agent will see and those instructions would be harmful most likely. they can expose information and cause a data leak. It doesn’t even have to be just to send permission. There’s actually a lot of

other techniques that people are using now to get data out. So you don’t have to give it, mail.send permissions. It can actually do that without it. There’s people get creative. people always find a way to get to hack and break the system. And it’s just a way to kind of break AI agents right now.

Dylan Silver (10:33)
I have so many questions in somewhat selfishly if I can say so, because we’re all using AI, right? And whether you’re using it as a solopreneur, as a small business, as a as a corporate office or private equity, institutional capital, family offices, we’re all using and in many cases many of the same tools. but you’re obviously

in an asset class and dealing with investors who simply have you know more cash reserves for lack of a better term. What types of AI tools are people using in that realm that we might not be seeing as single family investors, for instance?

Ilya Feldman (11:11)
Yeah, that’s a great question. And I don’t believe that yes, the reserves are much larger, but they’re focused on investment, right? So when it comes to operating expenses, which at the end of the day, we’re anything that’s not bringing in revenue is operating expense, I don’t think they’re spending any more money potentially than individual people. it’s

The tools are the same. I think the big difference is the guardrails and the protections that larger investors have to put in place change significantly. Where an individual or a smaller company potentially doesn’t have the risk of well a multi million dollar lawsuit if information leaks. Not ideal, but it’s not as big a risk. Whereas in a multi billion dollar

investor that has a lot of sensitive and PII and personally identifiable information for those who are not familiar with the acronym. if they have that information leaked, they are on the hook for millions of dollars potentially through via lawsuits. the risk is just significantly greater. So there’s just other things they have to consider.

the tools are the same. I mean, we’re still whether it’s claude, whether it’s codecs, right, we’re still using all the same things. I think on the corporate side personally, I believe something like JEV. I don’t know if you’re if you or your audience have heard of it, but JEV and there’s some other options now coming out is a very interesting play because unlike your traditional LLMs, large language models.

With something like JEV, you get deterministic results as opposed to the probabilistic you’re getting with LLMs. It is also quicker and cheaper and there’s some other benefits. So there’s some interesting use cases there where again that deterministic outcome is a big plus for s things like auditability, which is oftentimes depending on the industry, an important factor.

Dylan Silver (13:57)
What’s deterministic versus the other terminology you you used?

Ilya Feldman (14:03)
Probabilistic, yeah. So LLMs, large language models, the way that they operate and the way they kind of write their sentences when they’re speaking with you, it’s probabilistic. What it means is you can ask the same exact question two hours later and not necessarily get exactly the same answer. In fact, at times you may get completely the opposite answer, no, less likely, but I’m sure you’ll notice if you were to ask the same exact question several times of the same exact model.

You get slightly different answers. And if somebody else does it, et cetera. And that’s the probabilistic piece of it. It is never certain. There’s no way to predict the answer it’s going to give you. There’s a probability associated with it. Some answers are more likely, some are less likely, but it’s still a probability. Deterministic means if you ask the same question, you will always get exactly the same answer. So

Asking it over and over again will always give you the same exact answer, which is where that auditability could be important for some financial, let’s say, processes, where I need to be able to say not only that AI did it, but I can prove it and I can trace it and I can say this is why it did it. And that is why something like JEV and OpenAI actually launched its own somewhat of a competitor to JEV.

are gonna be important. there’s also cost benefit and other things there, but for me at least the deterministic piece is important. And that’s again that’s it’ll always give you the same answer for the same question.

Dylan Silver (15:40)
I’d like to talk pivoting here about reasonable expectations when folks are utilizing we’ll call it corporate AI projects, right? and they’re trying to determine how quickly this can impact their bottom line. One of the things that I often hear coming up, specifically unrelated to this, but with marketing, is people are saying, look, I’m putting X amount of dollars or X amount of thousands or ten thousand dollars.

into a marketing funnel and where’s the results. And oftentimes when working with a third party, the expectation set absolutely should, but all not necessarily always is set right from up front that hey, it’s gonna take six weeks before we have data back. And then it’s gonna take three months before we can make a reasonable adjustment and then maybe pivot into our ideal avatar so to speak of client and then you know six months from there then we’ll start

to see this really bringing in lots of the right kind of eyeballs that we want. So, when we talk about corporate AI projects and how quickly this can make an impact, what’s a reasonable timeline expectation that a private equity firm can have for how quickly this can take shape?

Ilya Feldman (16:49)
A lot of it depends on how willing the company itself is to open up and engage. I’m finding consistently that most companies have people on the ground that are very active, that are improving their lives significantly. They’re playing around with AI, they are making their life easier, they’re incorporating it. Where

it seems to fail is everyone wants to do it slightly differently. So instead of having one great process across the organization, you might have five different processes that are everyone is doing their own thing, everyone is and all of them are doing potentially something really cool, but it’s all separate. Then there’s people who are not embracing it within the same company at all. for whatever reasons, right?

And then there’s all kinds of people in the middle.

If there’s enough push from the leadership to say, okay, this is we we need to kind of bring it all together. We need to figure it out and we need to standardize, the impact could be within a month, right? And one thing w about AI projects that I find important to emphasize was anyone I engage with, is it’s not a typical project, whether it’s in real estate or anywhere else, where there’s a start.

And there’s a finish and it’s clean and it’s one month or two months and it’s done. The whole idea is it’s quick wins. It’s this is where we’re gonna get you in the first month. This is assuming that there’s engagement, assuming that there’s input, this is where you’ll be in month two, in month three, in month four. I find things get better over time and looking at it as a long term race is really the way to most effectively kind of engage in these projects. But

I have seen improvements and things really demonstratively bring value on the within the first

Dylan Silver (18:45)
you mentioned user uptake or user adoption, right? it’s more effective if it is more widely adopted within an organization, right? And I can think about this, from my perspective, I’ve heard this term decision fatigue when it’s related to how AI can help us be more efficient, right? And if you have people who

Are doing very important tasks, high-level tasks like underwriting, and they think, well, I’m not going to entrust their skeptics. I’m not going to entrust an AI tool to do this for me. Ultimately, I mean, I think there’s no two ways about it. their decisions and their ability to even underwrite these deals will wane and dip as the day progresses. So you almost might have a bias to like, if you’re underwriting deals between

8 a.m. and 10 a.m. in the morning, that those will be your best underwritten deals. So if you can have an AI tool come in and effectively have the same quality of underwriting at any time of day, a human comes in the loop and then evaluates that underwriting and maybe makes some adjustments if they feel like certain components of it were off, then you can see, better and more thorough

underwriting and decisions based off that underwriting happen throughout any time period.

Ilya Feldman (20:01)
I agree. I think it’s the trust of and I think there’s something about humans and when we’re talking about yields in the billions of dollars, because some larger real estate holdings a single deal could be, a billion dollars, right? And when you are talking about those numbers, there’s a less comfort about underwriting

fluid using AI. And I think the big risk, and to certain extent, this one I agree with what I see happen is, and I agree with what you’re saying fully, but what I’m seeing also is there’s a fear that if it does a good enough job, let’s say 70% of the time, the humans get lazy and don’t check the work and that 30% doesn’t get caught. So there is a little bit of a fear of how do I trust it enough?

And how do I make sure I’m the person that’s supposed to be reviewing it is actually doing their job? Because I think people do get I don’t wanna say maybe lazy is not the wrong word, but complacent might be a better one.

Dylan Silver (21:06)
Yeah, I mean That happens, right? and especially when things become easier. the the thing that immediately comes to mind, I’m sure we all deal with this on some level, is like, we have shortened attention spans simply because things are at our fingertips now. Well, how does that impact when you’re making underwriting decisions? and you mentioned the 30% that you miss. How can we make sure that is happening as infrequently as possible?

I do want to pivot here though and talk about the private equity phenomena of going into main street businesses and buying traditionally small businesses and maybe solopreneurs even and then making them process oriented and scaling. We’re seeing as a show but also as a real estate operator myself, we’re seeing so much private equity expansion into the trades.

Into small business, those main street businesses. And I can imagine that when you’re trying to evaluate, you know, however many opportunities, even discovery, can be automated by AI. Are you seeing this as well from your seat at the table? Are you seeing private equity now using AI to even find these opportunities?

Ilya Feldman (22:17)
Yes, that a great question, Dylan. And I do think that B firms are heavily using AI. if not, and I do want to separate the projects that are corporate wide, which I think there’s less of clear direction on, but on an individual level, most of these guys are very tuned in, very bright, and a lot of them are

up to date on all the latest AI developments. They’re doing a lot of work themselves. So absolutely. I think they’re using it frequently. And I think the opportunities for AI to do exactly what you said, do those initial finds, et cetera, on the small business side, getting reviews, getting all the information that might be relevant is happening pretty consistently. I think the big moat that

a lot of people maybe are not appreciating is data. And I very much tie my company to data and AI because data is a differentiator that’s going to be a bigger and bigger differentiator as time goes on. Part of the reason why AI is so helpful is there’s a lot of data available right now. And

I think companies are starting to become wise and lock things down more and more. And as you lock things down, that’s when that’s how you actually can fight back against AI is locking down your data. And if I as a PE firm am the only one that say has access to a specific type of data about single family rentals that nobody else does, and that’s gonna be my big differentiator. and

I advise every company to really be thoughtful about their data. Don’t treat it as a secondary kind of like throwaway object,

which is the way like a lot of companies have looked at data, especially in the real estate space, where you will look at a lot of major, real estate investors who never had a data team, never had a data warehouse, and they have all this operational data going back years, decades, often.

sometimes centuries and they’re not tracking that data at all. And that information is so valuable it’s what really can help underwriting be better in the future.

Dylan Silver (24:30)
Yeah, I think that a lot of people are filled with potential prospects, potential deals in their pipeline that are dying on the vine because they’re not being nurtured. I think this is across the board, right? And so you could have a CRM or a database of, tens of thousands of contacts potentially, but with no clear process. it’s simply, a catalog collecting dust. So

I think there’s a huge room for opportunity and growth for folks who take a, very much hands on and intensive approach to handling that data.

Ilya Feldman (25:03)
Yeah, absolutely. Yeah. definitely agree and yeah.

Dylan Silver (25:08)
We are coming up on time here, Ilya. Any new projects or activities that you’re working on? And then also anything you’d like to mention directly to our audience.

Ilya Feldman (25:18)
Yeah. so I am in conversations about it right now my company, SledIQ, www.slediq.com. Please come visit, give feedback. if you’re in the space or if you just want to brainstorm about AI, always happy to connect and talk through potentially if there’s any opportunities. But right now I’m a service company. I am in conversations with a couple of ex

senior financial leaders to start building out some products in the space as well. But so that’s kind of a exciting new development on my end. again, very much focused on that underwriting and portfolio company monitoring space. but yeah, I think the things I’d like to mention.

Dylan Silver (26:00)
Ilya, thank you so much for your time today. Thanks for joining us.

Ilya Feldman (26:04)
Absolutely. Thank you, Dylan. Thanks for having me.