I gave up Google for a month and put my every question to Bing with ChatGPT instead because I wanted to know what living in the future feels like. And honestly, after weeks of trying to talk a chatbot into giving me what I want I’m so glad to be going back to Google.
I learned that Google is still a more effective, useful tool for finding information than Bing with ChatGPT, at least for me. In fact, even regular Bing search proved itself more useful than Bing with ChatGPT when it came to looking up basic information.
This is important because Bing with ChatGPT feels like a huge swing by Microsoft, one that hopes to launch the company’s also-ran search engine to the forefront of a new wave of innovation characterized by AI-like chatbots worming their way into every service we use.
When I flew up to Seattle in February 2023 to attend Microsoft’s ChatGPT event I saw company chief Satya Nadella claim “this technology is going to reshape pretty much every software category”, and it’s clear Microsoft is leaning into that vision hard by integrating ChatGPT tech into not just Bing but Edge, Teams and the company’s suite of productivity apps via Microsoft 365 Copilot. Google is playing into this by rolling out its own Google Bard AI search chatbot to compete.
Of course, backlash has already begun as Elon Musk and other leaders call on ChatGPT and Bard to halt AI training beyond the latest iteration, ChatGPT-4. This is the same engine that powers Bing, but after a month with this chatbot I can see why Microsoft and OpenAI are keen to make further improvements.
Rather than walk you through my whole month searching with ChatGPT I’ll focus on some specific examples of how well it worked (or didn’t) so you have a better idea of what this tech can do — and where it still comes up short.
Bing with ChatGPT still can’t beat Google for usefulness
If you haven’t yet tried Bing with ChatGPT for yourself, I suggest you do now that Microsoft has made Bing with ChatGPT available to everyone. Once you do you’ll see the core gimmick: Instead of typing a search term in and getting a list of websites back, you type a query and get a conversational response generated by a chatbot trained on LLM (large language models) datasets.
Though Bing with ChatGPT appears to be responding to your queries like a human would, in actuality it’s relying on algorithms to string together plausible-sounding sentences around data found by Bing.
Most of the time it generates text that reads like a human wrote it, but not always, and even when it does there’s no guarantee the facts it reports will be accurate. Competitors like Google Bard face similar problems, as you can see in our Bing with ChatGPT vs. Google Bard face-off.
But when I gave up Google for the month and switched to Bing with ChatGPT for every search request I had to make, the winner of Bing with ChatGPT vs. plain ol’ Google quickly became apparent to me, and it was Google by a mile.
Here’s a particularly recent and relevant example: Some friends and I decided to see Paramount’s “Dungeons & Dragons: Honor Among Thieves” this weekend to support Chris Pine’s burgeoning acting career, so I hit the web to find the best showing.
I knew we wanted to see the film at an Alamo Drafthouse if possible, so I asked Bing with ChatGPT “is the D&D movie playing at any Alamo Drafthouse in the Bay Area?” Bing came back with a negative response, and fair enough, so I asked a simple follow-up: “What is playing at the Alamo Drafthouse New Mission [our local Alamo]?”
“What really makes this feel like a waste of time is knowing that if I had just used Google I could have gotten the answer in seconds.”
This is where my search journey stopped before it really got going. Bing simply came back and said “I’m not sure what movies are playing at the Alamo Drafthouse today. However, you can check their website for more information.”
Perfectly clear and logical, but not exactly helpful. Should I have just checked the Alamo Drafthouse website first, instead of bothering with Bing with ChatGPT? It’s kind of a bummer that I wasted time having a two-question conversation with a chatbot and have nothing to show for it.
But what really makes this feel like a waste of time is knowing that if I had just used Google.com (or the Google Omnibox that is my Chrome address bar) I could have gotten the answer in seconds.
To prove it, I fired up Google and typed the same search string Bing took from my last question (“Alamo Drafthouse Cinema New Mission movies”) in, and BOOM — in a second I’m staring at showtimes crowning the top of my Google search results, beautifully arranged alongside a link to the theater website, the phone number, photos of the theater and the like.
What’s even more damning is that if I navigate to the old Bing search (the one without a ChatGPT-powered chatbot to talk to) and search the same string, I get a Google-like page of search results from Bing replete with showtimes. So while Bing itself remains at least competitive with Google in terms of search, the addition of ChatGPT (at least so far) appears to make Bing less effective at answering certain types of queries. Common, reasonable queries too, like “is X movie playing at my local theater?”
Turning to Bing for car trouble
A similar story played out earlier this week when I unexpectedly needed to find a local, open auto window repair shop on short notice. My car’s passenger-side front window was smashed during the night, and I needed to find a shop that could replace it the same day.
I first turned to Bing with ChatGPT and asked “What is the best auto glass repair shop in Oakland?” Bing came back with a list of businesses it pulled from Yelp, and that’s it. Fair enough Bing, that’s the bare minimum I asked for.
Since I needed the car repaired same-day I asked Bing “Which can take my car today?” Bing couldn’t find the answer, so it told me so and asked me if I’d like it to look up the shops’ phone numbers instead so I could call myself and ask. Sure, thanks Bing. That’s what I would have done anyway and I probably could have done it faster, but it was nice of the chatbot to ask.
I ended up calling around, got a quote and got my car in early the next day. But I couldn’t help but feel like using Bing with ChatGPT actually caused my search to eat up more time, not less. To check, I went ahead and today and typed the same query (“What is the best auto glass repair shop in Oakland?”) into Google.
Almost instantaneously I received a list of links (to mostly different shops) alongside a map showing where they’re all located, which was instantly more helpful than what Bing with ChatGPT had given me.
However, the first half of Google’s search results were big attention-grabbing ads—clearly some shops are spending big with Google to promote their businesses. I was still able to get what I needed without scrolling too far, but I can’t pretend it wasn’t a hassle to have to look past all these sponsored Google ads to try and find the information I wanted.
So really I can’t say very nice things about either Google or Bing on this front. While Microsoft’s AI-powered search engine can’t match Google for ad density yet, that’s likely to change soon as Bing’s chatbot is getting ads that look awfully annoying.
What I can say is that as annoying as I found the ads cluttering up Google’s search results, I was still able to get the information I wanted faster and with less effort than when I was relying on Bing with ChatGPT.
Outlook
After a month without Google I have to say, Bing with ChatGPT can’t compare — at least, not yet.
After a month without Google I have to say, Bing with ChatGPT can’t comparec— at least, not yet.
Sometimes the AI-like chatbot gives surprisingly useful or insightful responses, but more often than not (at least in my experience) it adds nothing of value to Bing. Occasionally it even detracts in significant ways, providing false or misleading answers in a way that a more straightforward search engine will not.
But the field of AI research is advancing faster than I can comprehend, and ChatGPT parent company OpenAI seems to be pushing hard to iterate on its tech and get it into more products. That may mean that by the time you read this Bing with ChatGPT will be smarter and more effective than I found it, perhaps even improving on your day-to-day life in ways I can’t predict.
That’s a nice hope to have, because despite the call for caution it’s clear 2023 will be something of an AI arms race in tech.