Mastodon

A.I. Text Generators Love to Bullshit (and my opinion on these "A.I." text generators)

Matt Birchler
Posted by Matt Birchler
— 6 min read

Another fighter has entered the ring! I've been playing around with these text generators for a while now, and I love seeing how they react differently to different prompts. Sometimes I'm amazed, and other times it's just a cluster.

Today I wanted to test the three big players in the space, and then give my opinion on where this is all going.

The Question

I gave ChatGPT (using GPT-4), Bing, and Google Bard the same prompt:

write a short review for the last of us tv show

Then I asked a follow up:

how can i watch it?

Here's what they gave me.

Google Bard

The Last of Us is a post-apocalyptic television series developed by Craig Mazin and Neil Druckmann for HBO. It is based on the 2013 video game of the same name, developed by Naughty Dog. The series stars Pedro Pascal as Joel Miller and Bella Ramsey as Ellie Williams. The series follows Joel, a smuggler hired to transport Ellie, a 14-year-old girl, across the United States, which has been ravaged by a fungus that turns humans into cannibalistic creatures. Along the way, they must navigate a dangerous world full of infected and other survivors. The Last of Us is a well-made and well-acted series that is sure to please fans of the video game. The series is faithful to the source material, while also making some changes to improve the story. The performances by Pascal and Ramsey are excellent, and they bring the characters of Joel and Ellie to life. The series is also visually stunning, with some of the best special effects ever seen on television. Overall, The Last of Us is a must-watch for fans of post-apocalyptic fiction. It is a well-made and well-acted series that is sure to please. Here are some of the strengths of the series: The performances by Pedro Pascal and Bella Ramsey are excellent. The series is visually stunning, with some of the best special effects ever seen on television. The series is faithful to the source material, while also making some changes to improve the story. Here are some of the weaknesses of the series: The series can be slow-paced at times. The violence can be graphic. The series may not be suitable for all viewers.

This is actually a fantastic summary of the show, its cast and creators, and what people like and don't like about it. It's a bit passionless, so it's not going to replace a human review, but in terms of like a Rotten Tomatoes summary, it ain't bad.

You can watch The Last of Us on HBO Max. HBO Max is a streaming service that offers a variety of TV shows and movies, including The Last of Us. You can watch HBO Max on a variety of devices, including your computer, phone, tablet, or TV. To watch The Last of Us on HBO Max, you will need to subscribe to the service. You can subscribe to HBO Max for $9.99 per month. You can also get a free trial of HBO Max for 7 days. Once you have subscribed to HBO Max, you can watch The Last of Us by going to the HBO Max website or app and searching for the show. You can also watch The Last of Us by clicking on the "HBO Max Originals" tab on the HBO Max website or app.

The follow up on how to watch it was also spot on. A bit lengthy, but accurate.

Bing

Speaking of Rotten Tomatoes, Bing basically summarized the Rotten Tomatoes summary and score, and then threw in some Metacritic for good measure.

I personally don't like this response as much as Google's, but it's to the point and accurate.

Bing also got it right on where to watch it, and it said it quicker than Google. Well done, Bing.

ChatGPT (welcome to bullshit city)

Title: A Captivating Journey through the Apocalypse - "The Last of Us" TV Show Review "The Last of Us" TV series, based on the critically acclaimed video game of the same name, has proven itself to be a masterful adaptation. This emotionally charged journey through a post-apocalyptic world is not only a visual treat but also a narrative tour de force. The story follows Joel and Ellie, two survivors of a fungal outbreak that has decimated humanity and turned the infected into vicious creatures. As they traverse the ravaged landscape, the duo develops a bond that tugs at the heartstrings, offering a rare glimpse of humanity amidst the chaos. The show's creators have done an exceptional job of translating the game's essence to the small screen, with faithful character portrayals and an engaging plot. The performances of the lead actors are phenomenal, bringing depth and nuance to their roles. Additionally, the supporting cast adds further layers to the story, creating an immersive experience for the viewer. Visually, "The Last of Us" is stunning. The production design and cinematography capture the desolate beauty of a world in ruins, while the special effects and makeup artistry breathe life into the terrifying infected creatures. The show's pacing expertly balances moments of tension, action, and emotion, making it a thrilling watch from start to finish. One minor critique is that some subplots feel slightly underdeveloped, which may leave viewers wanting more. However, this does not detract from the overall impact of the series. In conclusion, "The Last of Us" TV show is a captivating and emotionally resonant adaptation that does justice to its source material. The series is a must-watch for fans of the game and newcomers alike, offering a unique blend of heart-wrenching drama, thrilling action, and hauntingly beautiful visuals.

This is a bit unfair since ChatGPT is only able to access data from up to September 2021, so it has no idea how The Last of Us reviewed, but it did at least know that it was a thing, as the show was announced in 2019 and casting was revealed in February 2021.

Honestly, this "review" reads like a student trying to explain to their teacher the reading they didn't do last night. I mean…

One minor critique is that some subplots feel slightly underdeveloped, which may leave viewers wanting more. However, this does not detract from the overall impact of the series.

…come on!

At this point is seems mean to ask it where to watch, but here goes.

To watch "The Last of Us" TV show, you'll need to find the network or streaming platform that holds the broadcasting rights in your region. Here's a general guide on how to watch the show: Check your local TV listings: Look for the channel airing "The Last of Us" in your region. This information can often be found in TV guides, newspapers, or online listings. Streaming platforms: If the show is available on a streaming platform, you'll need to subscribe to that platform to access the content. Examples of popular streaming platforms include Netflix, Hulu, Amazon Prime Video, Disney+, and HBO Max. Note that the availability of the show may differ based on your location. Video-on-demand (VOD) services: Some shows can be purchased or rented through VOD platforms like Google Play, iTunes, or Amazon Video. You can buy individual episodes or entire seasons to watch at your convenience. VPN services: If "The Last of Us" is not available in your country, you can use a VPN (Virtual Private Network) service to access a streaming platform or VOD service from another country. This method, however, may not always work, and using a VPN to bypass geo-restrictions could violate the terms of service of some platforms. Wait for the DVD/Blu-ray release: If you prefer physical media, you can wait for the release of the show on DVD or Blu-ray. Check your local retailers or online stores for availability. Remember to always access content through legal means to support the creators and avoid any potential legal issues.

Again, it adopts the "just keep talking" method of bullshit. The answer is HBO Max, but this is basically "how to watch anything" which isn't really what I wanted.

Bonus Question: Code for Me

I happen to run one of the most popular BIN lookup tools in the world (weird, I know), Quick BIN Lookup, and I use Binlist's API to get that info, so I asked these 3 bots to write a cURL request for me to get some data.

I'm a text-based AI and can't assist with that.

Google was like "I don't do that". Bummer.

Here is an example of a curl request for the binlist API that gets the card data for a credit card: curl https://lookup.binlist.net/4111111111111111 In this example, 4111111111111111 is the credit card number. You can replace it with the credit card number you want to get the data for. I hope that helps! Let me know if you have any other questions.

Bing answered, although it's worth noting that you should never use the full card number in this request (although this does technically work).

The Binlist API allows you to retrieve information about a card by providing the card's BIN (Bank Identification Number), which consists of the first 6 digits of the card number. To make a cURL request to the Binlist API, use the following command: bash Copy code curl https://lookup.binlist.net/{your_bin_here} Replace {your_bin_here} with the first 6 digits of the credit card number you want to look up. For example, if the card number is 1234567890123456, the BIN would be 123456. The cURL request would then look like: bash Copy code curl https://lookup.binlist.net/123456 This command sends an HTTP GET request to the Binlist API, and the API will return a JSON object containing information about the card, such as its brand, type, and the issuing bank. Please note that you should use real BIN numbers when making requests, as the example provided is fictional and will not return any valid information.

And ChatGPT did stunningly well, giving me exactly what I needed.

Of note, both Bing and ChatGPT were able to show me an accurate example response when asked as well.

Quick Disclaimer

One of the tricky things about these chatbots is that their output is unpredictable. Sometimes you get exactly what you want and it works perfectly. Other times you'll give it the same prompt and you get garbage back. This makes head-to-head comparisons like this tricky, as my results may not line up with your own. For what it's worth, I did try all of these queries a few times each to make sure what I showed in this post was generally consistent with what I was getting.

My Opinion on These Large Language Models Today

I think the ChatGPT review of The Last of Us is really telling about how this stuff works. It was able to understand what I wanted, it just didn't have the answer, but it fed me bullshit anyway.

I also think that Google Bard and Bing show the value in being able to feed search results into the model to give better responses. From what I've seen with companies who are using LLMs with their information included (can mention it here, but no one by name 🙃), they can also get very good answers for FAQs they get from customers or staff as well. ChatGPT is very fun, but I think the real useful versions of this will come from specialized rollouts of the technology in more constrained environments.

Finally, I'll just add that this is all new and exciting and scary and confusing at the same time. Some people feel like this is all garbage and it's theft masquerading as progress. Others think we should all move to the woods before these bots are going to get so advanced so quickly that none of us will have jobs and the machines will kill all humans so that we can't slow down their progress. I'm still sorting out how I feel, and I think that should be a position more people are willing to say on issues today.

My current feeling is that I've seen this before with countless technical advancements. People get hyped up, they say "if we continue at this rate, the world as we know it is gone," and then we can't actually keep advancing at that pace and things simmer down until that new technology is just a part of our lives. Yes, there is disruption, but the world doesn't end.

I think these language models are doing something similar to a mentalist. They give the appearance of doing something magical, and it's a very convincing trick, but at the end of the day it's just a trick. Ask them to do something outside their routine and the mystique falls away quickly. That doesn't mean the show was any less entertaining, it just means that what appears to be magic on stage in front of an audience is not actually breaking the laws of nature - it's just an act.

What I think right now is that we crossed the line between these language models being academically interesting and now they're interesting to everyone. The fact any of this works at all is astounding, but I find the conversations that assume things are going to advance rapidly from here are likely overstating things. These will get better, and integrating them into specific use cases will have useful (if sometimes disruptive) impacts, but I would not bet on these being so disruptive that the world of 10 years from now looks unrecognizable from today. After all, it was just one year ago that all your tech podcasts and opinion columns were flooded with people convinced that crypto and NFTs were changing everything in real time and you needed to get on the train or get left behind. See also the Clubhouse hype train from the year before…

Now I saw the bullshit in the NFT racket last year, and followers of this site probably got tired of hearing me rip on crypto for months on end. I'm more positive about these LLM implementations because I think there is something useful here, I just haven't drunk the Kool-Aid as much as some others have on its world-changing potential. But like I said, I'm flexible on this if the evidence changes.