Critical Analysis – Hints for Critical Analysis in Class Activities and Daily Life
by Jerry Mertens, Dan Scriver, and Dan Knip, St. Cloud State University

As of Oct. 17, 2017

The objective of this material is to help you develop a critical analysis repertoire that can be used to analyze the critical analysis events we typically use at the start of class; even more importantly, it is intended that these exercises develop critical analysis skills for use in daily life. False explanations, inadequate analysis of human behavior, extraordinary claims, fraudulent activities, etc. all must be carefully analyzed in order to find the variables at work in a given situation. Deceptive practices are used frequently. The number one source of white collar crime in 1994 was financial scams, according to Channel 9 News in Minneapolis, as reported on May 5, 1994.

Students in class witness events that are presented as psychic events. The student’s job is to become a detective and offer real world explanations for these supposed psychic events. This handout is intended to help students develop and utilize some critical analysis skills.

1. Utilizing a Useful Definition of Magic in Critical Thinking:
When one really analyzes magic, it is the use of scientific principles to make it look like scientific principles are not at work. This definition of magic, developed by the first author, is useful in the critical analysis of events performed at the start of class, as well as when critical analysis is needed in daily life. Hobby and stage magicians do exactly this. They use scientific principles to make it appear as though there is NO use of scientific principles in the performance of the trick or look like a different principle at work. When the stage magician says that they will pull a rabbit out of an empty hat, the magician does NOT pull a rabbit out of nowhere. The magician uses principles which make it look like they are pulling the rabbit out of an apparently empty hat. In another trick, the magician is not really sawing someone in half. When they use a good set of scientific principles, it just looks like the event is done by “magic.”

A con artist does this also. The following is often heard after someone has been made a fool of by a con artist: “Joe Blow was such a nice person. I just can’t imagine that this person was involved in such a scam.” Of course, if the person did appear like a con artist you would not buy into the scam. You would not expect the person to say, “Joe Blow looked like a crook from the moment I first saw him/her. I knew that it was a scam, but I still gave them all my money. I just like to give money to people who are trying ineffectively to perpetuate a scam.”

Approaching the unknown, examining extraordinary claims, questioning inferential explanations, etc. is necessary. This definition of magic encourages the search for the scientific principles being used. What other principle(s) of science could account for what appears to be a stunt that defies the laws of science?

2. Magic is the Exception in Daily Life Where Society Approves of ANNOUNCED Deceit, Fraud, Lies, Cheats, etc.:
The role magic plays in our society is another concept that is useful in developing and using critical thinking. Magic as entertainment is society’s approved way of using fraud, cheating, etc. to fool people. There is a bar in Chicago that features bartenders doing magic. The marquee outside the bar says “It is fun to be fooled.” The problem is that not all who use magic technology are announcing that they are tricking you. There exists a large commercial multimillion dollar industry out there that is working on inventing new things, manufacturing items to deceive, marketing devices with good sales pitches, selling sophisticated technology, organizing groups, providing a lot of literature (propaganda) with well orchestrated sources of information, and so on. This is not simply passing on the secrets of the art form of “magic”; it can be personal and secret information. People have tied some of the deceptive methods of the art form of “magic” to strong ideological cult-like efforts and have organized groups to perpetuate this cause. In short, there are many people abusing the art form of “magic” in order to gain money and power. This may be verbal deception or magic gimmicks. If you are going to fall for some extraordinary claim, do it with the knowledge of the technology and methods available. Better yet, don’t fall for it at all.

As far as the in-class activity, the nature of the “fraud” producing industry means that there will continue to be a constant new supply of stunts for the student to guess at what principles are at work. Therefore, these will be stunts that students have a minimum of experience with. This condition maximizes the students’ need to compare the environment against their knowledge to increase the chances of coming up with an adequate explanation.

3. Source of the Claim (person making claim):
One needs to question the source from which a claim has come. What do you know about the source? Is it reputable? Be careful. Just because a name sounds good or scientific does not prove that good techniques are being used. The less familiar you are with the source, the more reservations you should have about the validity of the claim. As in the class procedure, you should know it is a trick. In real life, you are not warned before a fraud, con, scam, hoax, skullduggery, swindle, or flimflam takes place. What is the nature of the source of evidence?

4. Determine the Source of Experimental Scientific Data (publishing source):
Are the experimental findings that are in question reported in reputable journals? What values or personal opinions are evident in the source of the findings? Does it appear in a journal published to promote a specific interest or the position of a group? Does the finding appear in a “refereed journal” (i.e. a journal with a panel of reputable authorities in the field to evaluate the studies that are submitted for publication)? Is this panel as unbiased as possible? Is it a good scientific source? Science is a buzz word, so be careful. Some of the worst incorrect inferential fictions stated as supposed causes of behavior have, and will be, offered as good science. You must be able to recognize false claims.

5. The use of Independent and Dependent Variables in Experimentation:
Were the findings based on independent variable experimentation? How were the relevant variables controlled? If it was an experimental design, was it a single-subject design or a group design the study used? Learn the role, value, and shortcomings of both of these research designs.

6. Has the Claim Been Replicated by Independent Sources?
Has the finding been replicated by a number of independent sources using good experimental methods? For example, the concept of positive reinforcement has been replicated numerous times by independent, credible sources, making it a useful concept. If fraud had been found in these labs, e.g. for research on reinforcement, if a string had been used to pull the animal through the procedure or false data (the so called “dry run”) was found to be used, we should question the reinforcement notion as much as we question extraordinary claims.

7. Forget Those Supposed “Gems” You Have Learned:
At some time during their education, many students are taught things like ”Everyone selects the color blue” (or yellow or green etc.) or ”People are attracted to bright colors, and will always select the brightest one.” “A certain position is always selected.” “People will go to the right all the time.” “The hand is quicker than the eye.” These sorts of “gems” have no use in predicting what a person will do in a given situation.

8. What Someone Says vs. What The Person Does:
Listen carefully to what a person says he or she is doing, then analyze what the person is actually doing. Are the two consistent? Listen and look for any inconsistencies. If the person said, “you have a random choice,” do you? In the case of “fear,” for example, which of the following statements are relevant:
a. Does the person say he/she is studying fear, but actually gives the subjects a paper and pencil test?
b. Does the person use a physiological measurement (heart rate, respiratory rate, a GSR – Galvanic Skin Resistance, etc.) and call it “fear?”
c. Does the person use a sample of fear related verbal statements, and call that “fear?”
d. Did the person just conjecture (their judgment call) one’s “fear?”
These are all different examples. Know what a person’s statement is based on.

9. Being Lucky vs. Using Coincidence:
One may take advantage of luck when it happens. If the real odds of an event happening are very bad, the psychic is not using luck in a given prediction. Yet, many individuals can take advantage of others when a low probability event happens. They can make the event appear much bigger or stronger than it actually is. There is a difference between these two. A psychic is not using 1 in 52 probability to predict your selected card. Those odds work for “post” diction, not prediction! However, if luck goes his or her way, a good magician or psychic will take advantage of it and can make a low probability event look like a bigger miracle.

10. Is The Situation Really Controlled?
Supposed psychics claim the situation is controlled to eliminate the possibility of any foul play, when in reality, the relevant variable is not sufficiently controlled. A self-professed psychic could set up a situation that appears controlled. For example, if he or she claimed to be able to rotate a dollar bill on a stick pin using only the power of his or her mind, the “supposed” controlled setting he or she could create might be to enclose the dollar bill on a pin under an upside-down fish tank. It would seem that the tank would eliminate the possibility of foul play involving moving air (like the supposed psychic’s own breath), but in a case exposed by Koran, it actually did not prevent air from going under the tank and above the table where the tank had been placed. How could she/he manipulate the dollar bill in this situation? Is the tank imperfect in some subtle way? Will it rest flush (air tight) against the table when it is upside-down? Could there be a gap between the upside-down tank and the table it sits on which could allow air to flow under the tank to reach the dollar bill? Be observant during situations that are supposed to be controlled. You need to question the controls in every situation.

11. Be on Guard For Misdirection:
Misdirection can be verbal or physical, and is intended to distract your attention. Here are some examples to look out for:
a) Repeated pairings with words or actions, e.g. continuously repeating an action, motion, etc. This will cause your attention to be dulled so you can be more easily fooled.
b) Pointing, and any other body movements that get you to look off in the wrong direction. Ask yourself what variables are used to misdirect.
c) Cards, book pages, or virtually any object can be secretly manipulated by the supposed psychic. If the first sheet of paper is shown as blank, that doesn’t mean a second sheet of paper introduced is also blank, or that he/she didn’t discreetly write something on it after revealing it was blank.
d) One may be misdirected as easily by verbal comments as by movements, gestures, looking in a certain direction, eye contact, etc.
e) Gaps in a verbal account are left for you to fill in for yourself, leading you astray. For example, a person opens up a “new” box of playing cards and pulls out a stack of apparently 52 regular playing cards. Before you get led astray, you might ask yourself a question, “Is this supposed fresh sealed deck a misdirection used to get one off track?”

12. Is the Explanation You Came Up With Feasible?
Is today’s technology in the specific area in question as advanced as your supposed explanation? After one of two coins appears to pass through a solid rubber mat, you might think, “The coin breaks through the mat, then the mat instantaneously heals/seals itself without a blemish, and this all happens before a penny that was sitting on top of the quarter can make it through the mat.” This explanation is probably not feasible.
Could the psychic have placed 30 different viable predictions all over the building so as to cover all prediction possibilities, and make it still appear as psychic ability? For example, one in the waste basket, one upstairs, one 2 offices down the hall, one under the table, one inside of a book in the hall, one downstairs, etc. Would this be an effective way of performing a stunt with a supposed prediction? Probably not. That would appear too phony. Is there a more feasible explanation, or is there a way for you to refine or readjust your guess so as to make it usable?
Did the psychic touch the item, or was it handled by audience members? Could there be confederates in the audience?
Assume there is an ordinary explanation to an extraordinary event. Often, you will not be aware of the scientific principle(s) used. This only means that the supposed psychic or magician has done a good job at keeping the truth hidden.

13. Realize Who is in Control:
The routine was created by the psychic performer, so he or she is the one in control of what happens. What are they still able to control? Has she or he told you ahead of time what is going to happen, or what she or he will do? If so, look for small changes the person might make in the routine as it goes along. Changes may occur depending on what you do.

14. Task Analysis:
By task analysis, one breaks down what was seen or what is happening into component parts. To do this, you must pay close attention to everything that happens during the episode. To miss one small but crucial part of the stunt could lead you in the wrong direction. In your task analysis, start with the first thing and work forward through the whole routine. If the purpose of a task seems to be to distract you, you might ask yourself what it is that you are not supposed to notice. If the presentation remains a mystery after analyzing each task or component part, reconsider if you may still be missing part of what happened.

15. Question Your Own Observations:
a) Question what is said instead of being absorbed into it.
b) Examine all the facts presented and be selective of which facts you believe to be true.
c) Consider all the alternatives to your tentative solution.
d) Look for contradictions in the facts being presented to you.
e) Be open to change your explanation for the observed mystery. Be careful not to become so set on an explanation that you continue to hold on to it even in light of alternate evidence.
f) Evaluate your own biases and work toward being more objective.
g) Realize if your conclusions are based on correlation. Events that are related or correlated are not necessarily causative. Look for an independent variable at work in the situation.
h) Push yourself to consider, “What physically possible ways exist for each of the component parts to change in order to accomplish the questioned happening? What in the physical world has to take place for the observed event to be an outcome?”
i) If all facts are not available, wait for them before making a final assessment. It is better to be empty (in need of answers) than to be full of garbage (wrong answers)! One needs to accept questions and not faulty answers.

16. Multiple Causation:
A reaction or result can be obtained by more than one means. You may know one way a fraud, mystery, trick, etc. is perpetuated, but that does not mean it is the only way. In other words, there is more than one way to bend metal, levitate objects, know a selected object, etc.

17. Be Alert:
Did you tune into the current problem late? Did you start looking soon enough? Are you currently seeing all of what is taking place? Have there been some events which have taken place prior to what you are now watching:
a. this could have been the same two people meeting.
b. a confederate and the person now involved having minimal contact.
c. information obtained from friends in a variety of situations.
d. information received by the supposed psychic, but not known to you. For example, overhearing a conversation in the lobby, street, entrance, rest room, public records, etc. Or perhaps all in the audience were asked to write something, but some were asked to write something different from others, etc. It has been stated that what you don’t know is not a problem. What is a problem is when what you know is incorrect knowledge.
Scientists can be vulnerable to paranormal claims and extraordinary events if they are not careful. Often, the scientist waits too long to begin questioning the situation. This questioning must occur continually. The task analysis needs to start with the very first task which may occur a long time before the stunt is even performed.
The next time you receive a notice in the mail that you have won a prize for some contest you never entered, and that all you need to do is order some product or send in a “small” amount of money to receive your “big” prize, remember that the company or organization may only be concerned with making a profit. The prize you supposedly won may be worthless, and receiving it is probably contingent on you doing something that will benefit the agent.

18. Life Can be A Stage:
Even if you may not be theatrically inclined to strange exaggeration in the way you present the world, realize others may be very theatrical in the way they tell you about the world. For example, there is documented scientific literature that good theatrical suggestions have led a non attention seeker to confess to a murder that the person had not committed. Oftentimes, when people describe a previous event, they tend to exaggerate certain aspects. It is more enjoyable to tell a shocking or interesting story even if it requires stretching the truth.