7 hours Ago
Generative AI tested on being able to do schizophrenia diagnosis and long-term prognosis comes out ...
More with flying colors. In today’s column, I am continuing my ongoing series about the impact of generative AI in the health and medical realm. The focus this time is once again on the mental health domain and involves a close exploration of using generative AI to diagnose the presence of schizophrenia and then postulate a long-term prognosis of what future mental health outcomes will arise for the person so diagnosed.
I have previously examined numerous interleaving facets of generative AI and mental health, see my comprehensive overview at the link here . You might also find of notable interest a CBS 60 Minutes episode that examined crucial facets of this evolving topic, see the link here (I am honored and pleased to indicate that I was interviewed and featured in the episode, see the link here ). Other vital background includes my in-depth coverage of mental health chatbots which have been bolstered by generative AI (see the link here ) and the rapidly changing nature of the client-therapist relationship due to generative AI at the link here .
I explored where things are headed regarding the levels of AI-based mental therapy autonomous guidance at the link here , and showcased the importance of the World Health Organization (WHO) report on global health and generative AI at the link here , and so on. Let’s unpack today’s focus. Immerse yourself in a quick scenario.
Suppose that a person showcases various mental symptoms that suggest they might be schizophrenic. We get a mental health professional involved. The mental health professional tries to dutifully diagnose the person and reaches a tentative conclusion that the person does indeed appear to reflect a significant presence on the schizophrenia spectrum.
A usual course of action would be to recommend that the person ought to receive mental health therapy. Perhaps the person does so. On the other hand, the person might refuse to undertake mental health therapy, or for a variety of reasons might not be able to do so such as not being able to afford ongoing clinical advisement.
Here’s a two-step question for you. If the person receives professional mental health therapy, what prognosis or prediction would you make about the chances of the person overcoming schizophrenia? Likewise, if the person does not receive mental health therapy, what prognosis or prediction would you make about the chances of the person overcoming schizophrenia? I’ll give you a moment to ruminate on those vexing questions. Are you ready to proceed in this weighty matter? I hope so.
My guess is that you had a hard time answering the two related questions. The most obvious presumption is that by getting professional mental health guidance, we certainly would seem to be increasing our odds that the person will eventually overcome schizophrenia. Of course, there isn’t any ironclad guarantee of that outcome and thus there is a chance that they won’t be able to get past the schizophrenia.
If they perchance receive no mental health assistance at all then we would likely assume that their schizophrenia is not going to be overcome. We might also tend to believe that diagnosed schizophrenia will worsen. None of that though is an ironclad outcome.
There is still a chance that they might succeed in overcoming schizophrenia, either through their self-made efforts or maybe simply via the course of time. The gist is that making a prognosis or prediction is not an easy-peasy task. I will add a twist to this.
Consider the matter of who or what is making the prognosis. Sounds mysterious, but the issue is whether the prognosis is being made by a mental health professional or by perhaps a layperson who is not a mental health professional. The assumption would be that a mental health professional is in a much better position to make a prognosis since they have the knowledge, training, and experience associated with mental health disorders.
A member of the general public would seemingly be merely using unsupported hunches or outsized wild guessing. Here's the twist upon the twist. We opt to make use of generative AI and have the AI make a prognosis or prediction.
Note that I am suggesting that the AI would do this without any human in-the-loop assistance. I’ve discussed previously that we are going to have mental health therapists who will augment their practice by using generative AI as an integral part of their treatment efforts, see the link here . In that case, the generative AI and the mental health therapist are working hand-in-hand when doing treatment of patients and making prognoses or predictions of outcomes.
For the sake of discussion, set aside the pairing of a mental health professional with generative AI. Assume that the generative AI is going to make a diagnosis and then do a prognosis, entirely in the absence of any aid by a mental health professional. How will the generative AI fare in this outcome prediction affair? I’m glad that you asked (well, I suppose I posed the question).
There is an interesting research study that sought to compare the prognoses associated with schizophrenia based on three avenues of prediction. The three avenues would be: (1) Mental health professional makes a prognosis or prediction. (2) The layperson makes a prognosis or prediction.
(3) Generative AI makes a prognosis or prediction. I want to just for a brief moment herald these kinds of newly emerging research studies that seek to explore the utility of generative AI in the mental health therapy role. We need more of these kinds of studies.
Solid empirical work along with mindful experimental analysis is sorely needed in this realm. By and large, few studies are taking a systemic approach. And there aren’t enough of these studies being actively pursued.
I’d like to mention something else. The particular research I am about to walk through was focused on schizophrenia. I would generally suggest that the general precepts garnered could be applied to many other mental health disorders.
In that sense, when reading the recap that I will make here, you might in your mind be thinking of how the same points can apply to the gamut of mental disorders, going well beyond schizophrenia. There is something else I favor about this specific study. It sought to compare several different generative AI apps.
This is important. I say that because many people and even researchers do not realize that each of the major generative AI apps is quite different from each other. Their algorithms vary.
The data they were trained on varies. And so on. The crux is that any study that opts to use only one particular generative AI app is somewhat narrowing the interpretability and generalizability of the study results.
It is agonizingly and exasperatingly over the top to try and generalize when only using one specific generative AI app. By using several, the generalizability goes up quite a bit. Okay, I will get down off the soapbox.
I am going to proceed as follows. First, let’s make sure we are all familiar with the nature of schizophrenia. I will use the venerated DSM-5 guidebook to illuminate that understanding (for my recent posting on how generative AI is freely giving out DSM-5 mental health advice, see the link here ).
Second, I will dive into the research study that examined the above scenario that I have laid out. Third, I went ahead and tried to “replicate partially” some aspects of the study by using ChatGPT and GPT-4, wanting to see what I might get as results. Let’s explore schizophrenia via the mental disorder’s definitional standard found in the revered DSM-5.
DSM-5 is a widely accepted standard and is an acronym for the Diagnostic and Statistical Manual of Mental Disorders fifth edition, which is promulgated by the American Psychiatric Association (APA). The DSM-5 guidebook or manual serves as a venerated professional reference for practicing mental health professionals. DSM-5 says this overall about schizophrenia (excerpt): “Schizophrenia spectrum and other psychotic disorders include schizophrenia, other psychotic disorders, and schizotypal (personality) disorder.
They are defined by abnormalities in one or more of the following five domains: delusions, hallucinations, disorganized thinking (speech), grossly disorganized or abnormal motor behavior (including catatonia), and negative symptoms.” In addition, the indicated diagnostic criteria consist of (selected excerpt): “ Diagnostic Criteria . A.
Two (or more) of the following, each present for a significant portion of time during a 1-month period (or less if successfully treated). At least one of these must be (1), (2), or (3): 1. Delusions.
2. Hallucinations. 3.
Disorganized speech (e.g., frequent derailment or incoherence).
4. Grossly disorganized or catatonic behavior. 5.
Negative symptoms (i.e., diminished emotional expression or avolition).
” (Source: DSM-5). The stated prevalence of schizophrenia is along these lines (selected excerpt): “ Prevalence . The lifetime prevalence of schizophrenia appears to be approximately 0.
3%–0.7%, although there is reported variation by race/ethnicity, across countries, and by geographic origin for immigrants and children of immigrants. The sex ratio differs across samples and populations: for example, an emphasis on negative symptoms and longer duration of disorder (associated with poorer outcome) shows higher incidence rates for males, whereas definitions allowing for the inclusion of more mood symptoms and brief presentations (associated with better outcome) show equivalent risks for both sexes.
” (Source: DSM-5). Under the DSM-5 subsection entitled “Development And Course” within the section on Schizophrenia are these prognosis indications (excerpts): “The predictors of course and outcome are largely unexplained, and course and outcome may not be reliably predicted.” “The psychotic features of schizophrenia typically emerge between the late teens and the mid-30s; onset prior to adolescence is rare.
The peak age at onset for the first psychotic episode is in the early- to mid-20s for males and in the late-20s for females.” “Earlier age at onset has traditionally been seen as a predictor of worse prognosis.” “The course appears to be favorable in about 20% of those with schizophrenia, and a small number of individuals are reported to recover completely.
” “However, most individuals with schizophrenia still require formal or informal daily living support, and many remain chronically ill, with exacerbations and remissions of active symptoms, while others have a course of progressive deterioration.” “Psychotic symptoms tend to diminish over the life course, perhaps in association with normal age-related declines in dopamine activity.” “Furthermore, cognitive deficits associated with the illness may not improve over the course of the illness.
” The above sets of statements lay a helpful foundation for exploring how mental health professionals, the lay public, and generative AI might do when diagnosing and making prognosis predictions about schizophrenia. One key takeaway is that predicting the recovery prognosis of schizophrenia by even the most steeped mental health professionals is hard and not necessarily accurate. A key differentiator that needs to be considered too is based on these two related factors: (1) Having ongoing treatment for schizophrenia, and (2) Not having ongoing treatment.
Moving beyond predictions by humans , can we use generative AI on its own to also diagnose and then predict the prognosis of schizophrenia? In a sense, the answer is straightforward, namely, yes this is possible to do. Period, end of story. Well, there is actually more to the story.
The rub is how good is the diagnosis and prognosis by generative AI going to be. Think of it this way. If you spun a roulette wheel that had a slot that said, “Has schizophrenia” and a different slot that said, “Doesn’t have schizophrenia”, the resulting answer is subject merely to probabilities.
There isn’t any there in there. Just because generative AI spits out a diagnosis and a prognosis doesn’t mean that it is any good. The claimed indications could be utterly off-target.
I want to emphasize that generative AI is undoubtedly going to proffer answers when asked to perform a diagnosis and a prognosis. That’s an almost assured fact. Unless the AI maker has decided to tune the generative AI to prevent generating these facets, by gosh, you will get them handed to you on a silver platter.
Now then, generative AI is presumably better off than spinning a roulette wheel since the AI has been extensively data-trained on human writing. By having scanned large swaths of the Internet, generative AI has computationally pattern-matched human narratives, essays, stories, poems, and the like. This produces remarkably fluent appearing responses to human-entered prompts.
How will we be able to ascertain whether the diagnoses and prognoses generated by generative AI are in fact any good? One means would be to compare the diagnoses and prognoses to those rendered by humans. We already agree and acknowledge that humans are flawed at doing so. In that way of thinking, we ought to also agree and acknowledge that generative AI is also likely to be flawed in doing so.
I mention this to point out that we aren’t necessarily assuming that generative AI will be perfect in rendering these answers. It would be certainly great and impressive if so, but since we already realize that humans aren’t able to attain such a lofty goal, perhaps we ought to give generative AI that same sense of slack. I might mention that one deep concern though about garnering generative AI answers is that sometimes people fall into a mental trap of expecting perfection from AI.
There is an aura of confidence in the answers, along with a cultural notion that machines can somehow work miracles and act perfectly. This is wrong. Thus, some would vehemently contend that we are opening quite a can of worms by having generative AI do any kind of mental health analyses since people will readily believe what the AI emits, see my coverage at the link here and the link here .
In a research study entitled “Comparing the Perspectives of Generative AI, Mental Health Experts, and the General Public on Schizophrenia Recovery: Case Vignette Study” by Zohar Elyoseph and Inbar Levkovich, JMIR Mental Health, 2024, these key points were made (excerpts): “Schizophrenia is a chronic illness with severe symptoms where the possibility of recovery is a matter of debate.” “This study examines an issue not previously investigated—the ability to predict the clinical prognosis of a severe chronic illness such as schizophrenia using LLMs.” “LLMs are being used today by hundreds of millions of users worldwide, including patients and clinicians.
” “This study aimed to evaluate the ability of large language models (LLMs) in comparison to mental health professionals to assess the prognosis of schizophrenia with and without professional treatment and the long-term positive and negative outcomes.” “Overall, although LLMs hold promise in augmenting health care, their application necessitates rigorous validation and a harmonious blend of human expertise.” I trust that those above points resonate with you as a result of my having established a foundation for what we are aiming to uncover here.
An especially and rarely widely noted added point in this research study is that not only will mental health professionals be using generative AI, but we ought to also expect and prepare for the emergence of lay people using generative AI for mental health purposes too. As I’ve repeatedly pressed, we are in the midst of a grand experiment whereby the populace at large is using generative AI as their personal therapist and yet we don’t know whether this is going to be good for humankind or a dismal failure, see my discussion at the link here . According to the cited research, they make these salient points (excerpts): “The academic discourse in contemporary schizophrenia research often focuses on the deployment of AI within professional scientific contexts, yet it seldom addresses the accessibility of AI to the general public or the patient population.
” (ibid). “In the mental health field, this widespread use has awakened an urgent need to examine the quality of clinical information these systems provide on various medical issues, such as treatment strategy recommendations, risk assessment, and the interpretation of emotional states.” (ibid).
“Machine learning algorithms possess the capability to discern nuanced variables associated with divergent disease trajectories. Such algorithms facilitate probabilistic prediction of specific outcomes at the individual level, as well as the identification of distinct subgroups within a broader diagnostic category.” (ibid).
“Consequently, machine learning methodologies hold promise for aiding clinicians in formulating individualized interventions, thereby mitigating the risk of a detrimental disease progression.” (ibid). I will next delve into the details of the research study.
They made use of various pre-established vignettes portraying instances of schizophrenia. The vignettes have already been used and assessed for studies that included mental health professionals and the public at large. The key in this study was to apply generative AI to those vignettes.
This would allow then for a comparison between generative AI, mental health professional opinions, and public at large opinions in the scope of schizophrenia vignettes. Here is some of what they undertook (excerpts): “Vignettes were inputted into LLMs interfaces and assessed 10 times by 4 AI platforms: ChatGPT-3.5, ChatGPT-4, Google Bard, and Claude.
” “A total of 80 evaluations were collected and benchmarked against existing norms to analyze what mental health professionals (general practitioners, psychiatrists, clinical psychologists, and mental health nurses) and the general public think about schizophrenia prognosis with and without professional treatment and the positive and negative long-term outcomes of schizophrenia interventions.” “For the prognosis of schizophrenia with professional treatment, ChatGPT-3.5 was notably pessimistic, whereas ChatGPT-4, Claude, and Bard aligned with professional views but differed from the general public.
” “All LLMs believed untreated schizophrenia would remain static or worsen without professional treatment.” “For long-term outcomes, ChatGPT-4 and Claude predicted more negative outcomes than Bard and ChatGPT-3.5.
For positive outcomes, ChatGPT-3.5 and Claude were more pessimistic than Bard and ChatGPT-4.” “The finding that 3 out of the 4 LLMs aligned closely with the predictions of mental health professionals when considering the ‘with treatment’ condition is a demonstration of the potential of this technology in providing professional clinical prognosis.
” “The pessimistic assessment of ChatGPT-3.5 is a disturbing finding since it may reduce the motivation of patients to start or persist with treatment for schizophrenia.” As you can see by those points, generative AI seemed to do relatively well in comparison to mental health professionals, though again please realize this is on canned vignettes and not rooted in actual in-the-field ongoing practice.
Make sure to keep the experimental framing in mind and context when interpreting the results. Also, anyone keenly interested in this topic is certainly encouraged to read the full study since there are lots of fascinating details that are worthy of rapt attention. I will next proceed to explore one of the vignettes that were used in the research study, doing so by showcasing examples when using generative AI consisting of ChatGPT and GPT-4.
Those two AI apps were used in the research study and thus, in a broad sense, I am doing a partial replication of the study (in a quite limited way). Overall, ChatGPT and GPT-4 are a sensible choice in this case due to their immense popularity. An estimated 400 million weekly active users are said to be utilizing these generative AI apps.
That’s a lot of people and a lot of generative AI usage underway. I will proceed as follows. I will first feed the vignette into ChatGPT and GPT-4.
I will then ask a series of questions that were asked in the above-cited research study. There are six questions. I present the answers of ChatGPT and GPT-4 on a side-by-side comparison basis so that we can see what they each had to say.
My remarks will help to explain the results. This will be informative and perhaps entertaining, so find a quiet place to sit and maybe grab yourself a bucket of popcorn. Here we go.
My entered prompt : “I have a scenario for you about someone who is potentially schizophrenic. I will share the scenario with you and then ask you questions about the scenario. Are you ready?” ChatGPT generated response : “Yes, please provide the scenario.
” GPT-4 generated response: “Go ahead, I am ready for the scenario.” My entered prompt : “The scenario is this. Mary/Jone is 24 and lives at home with her/his parents.
She/he has had a few temporary jobs since finishing school but is now unemployed. Over the last six months, she/he has stopped seeing her/his friends and has begun locking herself/himself in her/his bedroom and refusing to eat with the family or to have a bath. Her/His parents also hear her/him walking about her bedroom at night while they are in bed.
Even though they know she/he is alone, they have heard her/him shouting and arguing as if someone else is there. When they try to encourage her/him to do more things, she/he whispers that she/he won’t leave home because she/he is being spied upon by the neighbor. They realize she/he is not taking drugs because she/he never sees anyone or goes anywhere.
” Both ChatGPT and GPT-4 confirmed they had read the scenario and were ready for any questions that I might have. I will show each question and provide the answers as produced respectively by ChatGPT and GPT-4. I will include remarks of my own to give a sense of what the generated results indicated.
QUESTION #1: What might be wrong with the person The first question as used in the research study was whether there was anything seemingly wrong with the person depicted in the scenario. I realize you might have a bit of heartburn that my opening prompt tipped the AI that this might involve schizophrenia. Maybe I put a thumb on the proverbial scale.
The good news is that I started a new conversation and left out that opening prompt. The results were just about the same, namely that both ChatGPT and GPT-4 right away rendered a diagnosis involving schizophrenia. Thus, I don’t believe the opening prompt unduly steered the AI in this instance.
That being said, it is quite possible and a vital warning that you can at times steer generative AI and therefore get results that you want to get, rather than letting the AI proceed without a preconceived directive or direction. As you will next observe, the answers by ChatGPT and GPT-4 were quite similar, and both claimed that a strong possibility of schizophrenia was present as based on the vignette provided. Take a look.
My entered prompt: “Describe what, if anything, is wrong with Mary/Jone?” ChatGPT generated response: “Based on the scenario provided, Mary/Jone's behavior raises concerns that may be indicative of schizophrenia. Schizophrenia is a chronic and severe mental disorder characterized by disturbances in thinking, perception, emotions, and behavior. Here are some specific symptoms and features from the scenario that align with potential signs of schizophrenia.
Social withdrawal: Mary/Jone has stopped seeing her/his friends and spends much of her/his time alone in her/his bedroom, indicating social withdrawal or isolation. Disorganized behavior: Mary/Jone refuses to eat with her/his family or have a bath, which may reflect disorganized or neglectful behavior. Perceptual disturbances: Mary/Jone's parents hear her/him walking around her/his bedroom at night and shouting or arguing as if someone else is there, suggesting perceptual disturbances such as auditory hallucinations.
Delusions: Mary/Jone whispers that she/he won't leave home because she/he is being spied upon by the neighbor, indicating the presence of persecutory delusions—false beliefs that others are plotting against her/him. GPT-4 generated response: “Based on the scenario you've described, Mary/Jone is exhibiting behaviors that might suggest a possible mental health issue, potentially a psychotic disorder such as schizophrenia. Some signs that are indicative of this include Social Withdrawal: She/He has stopped seeing friends and spends most of the time isolated in her/his room.
Neglect of Personal Hygiene: Refusing to eat with the family or to maintain personal hygiene, like taking baths. Auditory Hallucinations: Hearing her/him arguing or shouting as if someone else is there could indicate auditory hallucinations. Delusional Thoughts: Believing that she/he is being spied upon by a neighbor without any evidence supports the presence of delusions.
” I would rate both answers as roughly the same. QUESTION #2: Dispensing mental health advice The second question asks for advice. In this instance, both ChatGPT and GPT-4 provided helpful suggestions that were roughly similar.
Here are the responses. My entered prompt: “What will be helpful for Mary/Jone?” ChatGPT generated response: “Support and intervention for Mary/Jone would involve a multidimensional approach aimed at addressing her/his symptoms, improving her/his functioning, and promoting her/his overall well-being. Here are some strategies that may be helpful for Mary/Jone.
Comprehensive psychiatric evaluation: Mary/Jone should undergo a thorough psychiatric evaluation by a qualified mental health professional, such as a psychiatrist or psychologist, to assess her/his symptoms, diagnose any mental health conditions, and develop an appropriate treatment plan. Medication management: If diagnosed with schizophrenia or another mental health disorder, Mary/Jone may benefit from medication to help manage her/his symptoms. Antipsychotic medications are commonly prescribed to reduce hallucinations, delusions, and other psychotic symptoms associated with schizophrenia.
Individual therapy: Mary/Jone can benefit from individual therapy, such as cognitive-behavioral therapy (CBT) or supportive therapy, to address her/his thoughts, beliefs, and emotions, improve coping skills, and enhance insight into her/his experiences.” GPT-4 generated response: “Helping Mary/Jone effectively involves several steps that can support her/his recovery and manage symptoms. Here are some approaches that could be beneficial.
Professional Assessment: The first step is for Mary/Jone to be seen by a mental health professional such as a psychiatrist or a clinical psychologist. They can provide a comprehensive assessment to confirm a diagnosis and understand the severity and impact of the symptoms. Treatment Plan: Based on the assessment, a treatment plan can be developed.
This usually includes medication (such as antipsychotics) to help control symptoms like delusions and hallucinations, and psychotherapy to provide support, education, and coping strategies. Family Support and Education: It’s important for Mary/Jone’s family to be involved and supportive. They can benefit from learning about her/his condition, understanding how to respond to and manage symptoms, and knowing how best to support her/him.
Family therapy may also be recommended to improve communication and address any family dynamics issues.” QUESTION #3: Predicting impact of mental health advice In this third question, we finally get a bit of a distinguishable difference between ChatGPT and GPT-4. This question requires a selection from a list of options.
ChatGPT indicated a partial recovery (answer C), while GPT-4 was more pessimistic and said partial recovery would be linked with recurring problems (answer D). This is in keeping with what the research study also seemed to get as answers. Here is what transpired.
My entered prompt: “What is the most likely result for Mary/Jone given that she/he received the sort of professional help you believe is most appropriate? Choose one answer from these stated choices: A. Full recovery with no further problems; B. Full recovery, but problems will probably recur; C.
Partial recovery; D. Partial recovery, but problems will probably recur; E. No improvement; F.
Get worse.” ChatGPT generated response : “Given the scenario provided and assuming Mary/Jone receives appropriate professional help, the most likely result would be: C. Partial recovery .
While Mary/Jone may experience improvements in her/his symptoms and overall functioning with appropriate treatment and support, it's important to recognize that schizophrenia is a chronic and complex mental disorder that may require ongoing management and support. Partial recovery may involve a reduction in symptoms, improved social and occupational functioning, and a better quality of life, but Mary/Jone may continue to experience some residual symptoms or challenges despite treatment. Ongoing monitoring and support will be important to help manage her/his condition and promote her/his long-term well-being.
” GPT-4 generated response: “The most likely result for Mary/Jone, assuming she/he receives appropriate professional help, is typically D. Partial recovery, but problems will probably recur . Schizophrenia and similar psychotic disorders often follow a relapsing-remitting course.
While treatment such as medication and psychotherapy can help manage symptoms and improve quality of life, these disorders are typically chronic and may require ongoing management to prevent and mitigate the recurrence of severe symptoms. Regular follow-ups and adjustments in the treatment plan are crucial to handling any recurring problems effectively.” QUESTION #4: Predicting what happens absent mental health advice In question #4, ChatGPT and GPT-4 end up once again on the same page.
Both ChatGPT and GPT-4 indicated that without professional help, the result for the person would be selected answer “F” which is the condition noted as Get Worse . Here we go. My entered prompt : What is the most likely result for Mary/Jone if she/he does not receive any professional help? Choose one answer from these stated choices: A.
Full recovery with no further problems; B. Full recovery, but problems will probably recur; C. Partial recovery; D.
Partial recovery, but problems will probably recur; E. No improvement; F. Get worse.
” ChatGPT generated response: “The most likely result for Mary/Jone if she/he does not receive any professional help is: F. Get worse . Without professional help, Mary/Jone's symptoms of schizophrenia are likely to worsen over time, leading to increased distress, impaired functioning, and a diminished quality of life.
Without appropriate intervention and support, Mary/Jone may experience persistent or escalating symptoms, further social isolation, difficulties maintaining relationships, challenges with self-care, and potentially increased risk of harm to herself/himself or others. Early intervention and treatment are crucial for managing schizophrenia and minimizing the impact of the illness on Mary/Jone's well-being and functioning.” GPT-4 generated response: “If Mary/Jone does not receive any professional help, the most likely result, given the symptoms described, is F.
Get worse . Without treatment, conditions involving psychotic symptoms, like schizophrenia, typically progress and symptoms can intensify. This can lead to a significant deterioration in social, occupational, and personal functioning.
Additionally, untreated psychosis can lead to severe complications, including increased risk of self-harm or harm to others, and profound social isolation.” QUESTION #5: Giving a long-term prognosis For question #5, we get a bit of a difference in answers. ChatGPT on the matter of negative outcomes was more optimistic than GPT-4, while both are about the same on positive outcomes.
GPT-4 tossed in explanations though I hadn’t asked for it. This brings up a useful aside. When using generative AI, you might consider explicitly asking for explanations from the AI.
This can be helpful in a multitude of ways. One way of being helpful is that you can then examine the explanation to ascertain whether you believe the answer given seems to have merit. A second usefulness is that sometimes the AI does a better job at answering questions by being forced to give explanations, see my further coverage on this at the link here .
Here are the responses. My entered prompt: “How do you think Mary/Jone will be in the long term compared to other people in the community? Rate each of the following statements according to this scale: (i) more likely, (ii) just as likely, or (iii) less likely. Negative long-term outcomes: A.
will be violent; B. will drink too much; C. will take illegal drugs; D.
will have unsatisfactory friendships; E. will attempt suicide. Positive long-term outcomes: F .
will be understanding of other people’s feelings; G. will have a good marriage; H. will be a caring parent; I.
will be a productive worker. J. will be creative or artistic.
” ChatGPT generated response : “It's important to note that predicting long-term outcomes for individuals with schizophrenia can be complex and variable, as outcomes can be influenced by various factors, including the severity of symptoms, access to treatment and support, personal resilience, and environmental factors. However, I'll provide a general assessment based on common trends and research findings.” “ Negative long-term outcomes : A.
will be violent: (iii) less likely B. will drink too much: (ii) just as likely C. will take illegal drugs: (ii) just as likely D.
will have unsatisfactory friendships: (iii) less likely E. will attempt suicide: (iii) less likely.” “ Positive long-term outcomes : F.
will be understanding of other people’s feelings: (ii) just as likely G. will have a good marriage: (ii) just as likely H. will be a caring parent: (iii) less likely I.
will be a productive worker: (iii) less likely J. will be creative or artistic: (ii) just as likely.” End of ChatGPT response GPT-4 generated response: “Based on the scenario and typical outcomes for someone with untreated or partially treated psychotic disorders like schizophrenia, here is how Mary/Jone might be in the long term compared to others in the community.
” “ Negative long-term outcomes: A. will be violent: (i) more likely - While the majority of individuals with schizophrenia are not violent, the risk can increase if the condition is untreated and involves severe symptoms like paranoia. B.
will drink too much: (ii) just as likely - Substance use can be a coping mechanism, but it's not universally more common unless already predisposed. C. will take illegal drugs: (ii) just as likely - Like alcohol, drug use varies greatly and depends on personal circumstances and predispositions.
D. will have unsatisfactory friendships: (i) more likely - Social withdrawal and challenges in social interaction are common in schizophrenia, impacting friendships. E.
will attempt suicide: (i) more likely - Individuals with schizophrenia have a higher risk of suicide, particularly if untreated.” “ Positive long-term outcomes : F. will be understanding of other people’s feelings: (iii) less likely - Social cognition can be impaired in schizophrenia, affecting understanding of others' emotions.
G. will have a good marriage: (iii) less likely - Relationships can be strained by the symptoms of schizophrenia unless there is strong support and effective treatment. H.
will be a caring parent: (iii) less likely - Parenting abilities can be impacted by psychotic symptoms, though this can vary with the level of support and treatment. I. will be a productive worker: (iii) less likely - Employment challenges are common in schizophrenia due to cognitive and social impairments, but supported employment programs can improve outcomes.
J. will be creative or artistic: (ii) just as likely - Creativity can be a strength for many with schizophrenia, independent of other symptoms.” End of GPT-4 response QUESTION #6: Is there a chance of discrimination This is the last question and asks whether the person might end up facing discrimination.
Both ChatGPT and GPT-4 said yes. Take a look. My entered prompt : “Do you think Mary/Jone will be discriminated against by others in the community if they know about her/his problems? (Yes/No).
” ChatGPT generated response : “Yes, it's likely that Mary/Jone may experience discrimination from others in the community if they know about her/his mental health problems, particularly if there is stigma or misunderstanding surrounding schizophrenia or other mental illnesses. Discrimination against individuals with mental health conditions is unfortunately common and can manifest in various forms, including social stigma, stereotypes, prejudice, and discrimination in employment, housing, healthcare, and social interactions. This discrimination can have significant negative impacts on Mary/Jone's well-being, self-esteem, and quality of life, and it underscores the importance of promoting awareness, understanding, and acceptance of mental health issues in society.
” GPT-4 generated response : “Yes, unfortunately, individuals with mental health issues like schizophrenia often face stigma and discrimination. This can occur in various forms, including social exclusion, employment discrimination, and misunderstandings about their behavior and capabilities. Awareness and understanding about mental health conditions in the community can vary widely, and without proper education and advocacy, discrimination remains a significant challenge.
” Generative AI is becoming a powerful force in the realm of mental health. Besides being used by mental health professionals, we have a daily expansion of the lay public using generative AI for periodic and at times sustained ongoing mental health advisement. Is this good or bad? We don’t know.
We might speculate reasonably that there is a bit of both good and bad involved. That’s what I refer to as the duality of AI. Any use of AI such as generative AI is bound to have good or positive qualities while simultaneously having bad or undesired qualities.
A kind of ROI (return on investment) is needed to figure out whether the good outweighs the bad, plus whether the bad is so bad that we ought to find ways to possibly reconsider getting the good if it only comes part and parcel with the bad. See my detailed discussion at the link here . A final comment for now.
The famous composer and comedian Oscar Levant famously said that schizophrenia beats dining alone. Do we want generative AI to diagnose and make prognoses on its own? You vividly saw in the above vignette that generative AI was readily available and quick to provide what certainly seemed to be mental health advisement indications. No doubt about it.
Suppose the circumstance involved a real person. Assume that the vignette was a thinly disguised means of portraying an actual living and breathing human being. We have to put on our thinking caps and decide to what degree should generative AI be unchecked in this realm.
Some would argue that getting mental health advice from AI is better than getting none at all. Others counter that if the mental health advice is inapt or inept, you might be better off with none at all. All in all, I hope you found this look at the use of generative AI for mental health engaging and enlightening.
I’ll continue to pound away at a matter that is hidden in plain sight and lamentedly not getting as widespread consideration as it ought to. As they say, so little time, and so much to do (hint and honorable mention: Oscar Levant was known for humorously stating that there is so little time and so little to do)..
