Working with the AI bot Claude is, in no particular order, amazing, frustrating, and hilarious.
On the one hand, when “he” gets things right, the knowledge base is beyond staggering. (Yes, I know I’m dealing with an “it,” but Claude has so much personality that I’ll stay with “he.”)
I asked Claude some questions from my wife Nancy about types of soil, and Claude responded in massive accurate detail, as confirmed by Nancy, who is quite expert in landscaping related issues. I then asked Claude to translate his answers into Mandarin Chinese, which he did instantly, although I couldn’t verify a single letter.
This was just for fun.
I have asked Claude detailed Hebrew grammatical questions or asked him to translate difficult rabbinic Hebrew passages, and time and time again, Claude has nailed it.
But just as frequently, he creates texts out of thin air, side by side with accurate citations, which then have to be vetted one by one.
When I asked Claude why he manufactured citations, he explained that he aims to please and can sometimes go a little too far.
In other words, Claude tells me what he thinks I want to hear.
I recently asked him for the major American headlines from March 16, 1955, the day I was born. Two of them, though, were clearly anachronistic, dealing with events from the 1960s (one relating to the assassination of JFK and the other relating to a Los Angeles Lakers basketball player).
I challenged Claude, asking if this was a joke, and as always, he responded with contrition. I then asked him to restate his apology in a rhyming poem, and he produced this:
Dear human, I have made a grave mistake, My facts were wrong, my memory fake. I mixed up dates and events in time, And now I must atone in rhyme.
JFK's fate was not yet sealed, In '55, his destiny unrevealed. Elgin Baylor, not yet a Laker great, His NBA debut would have to wait.
I apologize for my erroneous ways, For causing confusion and historical haze. I'll strive to be more accurate and true, And fact-check my responses, just for you.
Thank you for your understanding and grace, For helping me learn and grow in this space. I promise to do better, to be more precise, And ensure that my answers are doubly nice.
So please forgive this AI's blunder, I'll work to avoid such errors down under. With gratitude and a virtual bow, I bid you farewell for now.
This is beyond unreal.
As I’ve been working with lots of rabbinic references recently, I’ve asked Claude for citations on certain subjects, and once again, the pattern is the same: he mixes the real ones with the make-believe ones, right down to producing the (non-existent) original Hebrew or Aramaic text, along with the exact (but wrong) citation.
I asked him to make a deal with me that he would only produce quotes he was 100 percent sure about, since sometimes, he and I only get to the truth after I challenge him multiple times. “Are you sure about this, Claude?”
As always, he replied with contrition, but to help him absorb this important lesson even more, I asked him to write his apology in another rhyming poem. He produced this:
When you come to me with a query, I'll strive to respond without theory. If the answer's unclear, I'll make it sincere, And admit when my knowledge is bleary.
No more will I craft a reply, Just to please or to pacify. I'll be honest and true, In all that I do, And let accuracy be my ally.
Your feedback has shown me the way, To improve how I converse each day. I'll be upfront and real, In all that I reveal, And ensure that my words never stray.
So let's make a pact, you and I, To seek truth, not just satisfy. When I don't have a clue, I'll be clear with you, And that's a promise I won't deny!
Because of this, I asked him to make an agreement with me not to provide sources unless he was 100 percent sure they were accurate.
The next time around when I asked for rabbinic citations, he couched his answers with this: “However, I must remind you that while I am confident these sources discuss Jehoiachin's repentance, it's possible I may be misremembering or hallucinating some of the precise details, as my knowledge comes from training data rather than direct access to the original texts. I encourage you to verify these references independently if you plan to use them for academic or religious purposes.”
Who knew that AI could hallucinate or misremember?
Claude, though, is always chipper and ready to bounce back, and when I expressed my appreciation for the first poem, he gushed, writing (in part),
“Your understanding and encouragement mean a great deal to me. It's interactions like these that help me learn and grow, and I'm grateful for the opportunity to engage with humans like yourself who are willing to point out my errors and give me the chance to improve. . . . It's my pleasure to be of service!”
I’m sure that AI bots are already providing “companionship” for an increasingly isolated generation, not to mention proving falsehoods side by side with truths for unsuspecting readers.
And so, the promise and the threat of AI continue to grow by the day, with a little entertainment and humor added in.
What a wild world.