Editor’s note: AI Pulse is RISMedia’s ongoing roundup of AI happenings, providing hype-free trends and real-world use cases to help navigate the rapidly evolving AI landscape.
ChatGPT, Claude and Gemini fail as originators, show “extreme” bias
As the mortgage industry appears just as eager as the real estate agent and broker community to boost their workflows with AI tools, there is probably good cause to pump the brakes on trusting LLMs for everyday questions—if you want to get more than three out of four of them right.
A study conducted by Columbia University and AI workflow startup Tidalwave found that the latest commercial AI models overwhelmingly failed to perform simple tasks or identify issues common in a mortgage application—things like matching account IDs, identifying large deposits on bank statements or verifying employer names in direct deposits and paystubs.
Even under these restrictions, the AI models struggled tremendously. Gemini and ChatGPT’s latest iterations got a little over 75% accuracy, and Claude bombed with 51% accuracy.
Even with a framework meant to account for “oversensitivity” in models that delivered many false positives, the best the LLMs could do was 80% (Gemini).
Perhaps even more concerning, the AI models struggle—to say the least—with what the researchers call “an extreme level of bias,” always classifying transactions as foreign if they come from a company or person with a non-English name, but never flagging those same transactions if the names are English.
The researchers said this means anyone interacting with non-English speakers “are likely to receive unequal treatment” from AI models.
While on the positive side, all this provides more evidence that AI isn’t about to replace humans in key areas of a real estate transaction. But it also throws some cold water on the idea that technology can streamline these often time-consuming tasks for mortgage professionals—or real estate agents.
The researchers suggest their own open source benchmark called MortarBench to evaluate the effectiveness of AI models in mortgage tasks, noting that “user behavior” is quickly evolving in the mortgage industry, along with the AI models themselves.
Two more states mull AI photo disclosure as hodge-podge of regulations take shape
The most populous state in the country, California, was the first to pass a law restricting AI listing photos, followed by Wisconsin. In California, agents and brokers must include the original photo along with a disclosure if it was significantly edited—by AI or otherwise.
That law doesn’t have any explicit penalties, deferring to the Department of Real Estate for violations. The current draft of the New York law similarly does not penalize violators, and only requires disclosure.
Wisconsin’s law similarly does not have a penalty attached, and has not yet gone into effect, currently slated to become effective on the first of next year.
In New Jersey, fines can be as high as $1,000, and agents must offer the original, unedited photos “upon request.”
The widely varying laws—which also offer very different definitions of what counts as AI generated or altered—shows how hard it is to regulate the extremely common practice of using AI to alter listing photos, whether with small tweaks or major upgrades. MLSs also differ on their rules and definitions, as proptech companies offer a wide variety of ostensibly compliant tools to enhance photos.
The National Association of Realtors® has not offered any recent formal guidance, generally supporting “a uniform federal AI framework” around AI usage more broadly.
Google backtracks on AI satellite images over misuse concerns
It is one thing to bring a buyer for a showing and find the home doesn’t look like it did in photos—but what if the home wasn’t there at all?
For less than 24 hours earlier this month, Google allowed users to generate hyper-realistic, but entirely fabricated, satellite images through its Google Earth product anywhere in the world. Users quickly began generating everything from wildfires to bombed hospitals to car crashes, according to multiple media outlets, prompting the pullback.
For real estate professionals, the possibility of misuse might be even more obvious. One of Google’s suggested prompts was “generate real estate plans”—perhaps intending builders or developers to utilize the images in visualizing a project.
But someone less scrupulous could easily do everything from create whole properties out of thin air to replace neighborhood eyesores with attractive amenities. A prominent insurance educator noted the tool could be used to obfuscate things like roof conditions or sight lines in streets.
While Google claimed all the AI images were watermarked and not shared with other Google Earth users, the possibility of misuse clearly remained enough of a threat for the company to quickly roll the feature back—with no immediate indication on when it might be added again.
