Inquiry & Assessment in Science

TESTTEST

What is Inquiry?How do we assess it?

Arthur Eisenkraftwww.COSMIC.umb.eduwww.COSMIC.umb.edu

The Guillotine StoryThe Guillotine Story

The Kaila FactorThe Kaila Factor

numberatomic 1 = solid(1,2,3)matter of phase

numberatomic Factor Kaila =

A

1 = solid2 = liquid3 = gas

φκ A=

•Helium

•Carbon

•Mercury

•Zirconium

Compare and ContrastCompare and Contrast

• Density study in most textbooksDensity study in most textbooks

D it i i ti it• Density as an inquiry activity– What do we want students to know about

d it ?density?

Density study in most textbooks

Density as an inquiry activityfrom

Active Chemistry

A recipe for inquiryA recipe for inquiry7E instructional model

• Engage• ElicitElicit• Explore• Explain

Evaluate

• Elaborate• Extend

Enhancing the 5E model: gThe Science Teacher (9/03)Available at

www.cosmic.umb.eduwww.cosmic.umb.edu

Why ABC (Activity Before Concept)E iExperience

• Experience is required – science isExperience is required science is experiments

• Learn baseball without ever seeing the gameLearn baseball without ever seeing the game• Learn baseball without ever having equipment• Knitting without instructiong

• Why do we think students can learn science by watching?science by watching?

Why ABC (Activity Before Concept)C E iCommon Experience

• Television viewing habits of Black andTelevision viewing habits of Black and White Americans

• 10 top programs• 10 top programs– Only 2 programs in common

?• What are the common programs?• What is the lesson?

– Every time I made a reference to television, I was disenfranchising some group of students.

Why ABC (Activity Before Concept)C E iCommon Experience

• There is no common experienceThere is no common experience– TV, music, food, vacation, movies, travel, home– Can someone imagine what a mango tastes like?g g– TRADITIONAL BOOKS ARE FILLED WITH “You

know; Imagine”lik t th• … like waves at the ocean

• …like a recipe for making brownie

– Examples from your teachingp y g– Examples from literature or movies

Why ABC (Activity Before Concept)Mi iMisconceptions

• Misconceptions ResearchMisconceptions Research– You’re asking people to change the way they

look at the world. – They must have some evidence and some

experience.– They must have a reason to change.

• Phase of the moon• Colder in the winter• Acorn mighty oak

I wish you “the courage for inquiry”I wish you the courage for inquiry

• Inquiry requires patienceInquiry requires patience

P ti i• Patience requires courage

• Have the courage to pursue inquiry!

Assessing InquiryAssessing Inquiry

• If we value inquiry we must test forIf we value inquiry, we must test for inquiry.

Are we listening?Are we listening?

• The optometristThe optometrist

Th D ll titi• The Duracell competition

• Two objects falling in a vacuum

• The cord of wood

Brief History of AssessmentBrief History of Assessment

• When did it all begin?When did it all begin?

IQ t t– IQ tests– PhD orals

S– Socrates

And God said, “Let there be light” … and it was good

And God created humans and it was very goodAnd God created humans ….. and it was very good

How do we assess?How do we assess?

• List 3 ways you assess student’sList 3 ways you assess student s knowledge?

Classroom AssessmentClassroom Assessment The Grade Book

TestsQ iQuizzesHomeworkClass participation (?)Lab reportsLab reportsAttendance (X)Projects

The Final ExamThe Final ExamIndividual vs. department wide

These are often treated as summative though they do g yinform as formative.

Other formative assessments include:Q ti i l–Questions in class

–Practice tests

Challenges

•Identify who are we testing•Identify who are we testingoStudentsoTeachersoSchools and districts

Challenges

• Identify for what purpose(from Classroom Assessment and the NSES, NRC)

Challenges

oHelp students learn oTo illustrate and articulate the standards for quality workoTo inform teaching oTo guide curriculum selectionoTo guide curriculum selectionoTo monitor programsoTo provide a basis for reporting concrete accomplishments to interested parties

F t bilitoFor accountabilityoCertification

•Reporting individual achievement•Gradingg•Placement•Promotion•Accountability

oParents to taxpayersoParents to taxpayers(from High Stakes Assessments, NRC)

Challenges

•Are we trying to use ONE instrument•for all (students, teachers, schools)? f ll ?•for all purposes?

•Understanding vs. beliefoMazur student taking FCI

Classroom Assessment and the National Science Education

Standards (2001)Center for Education

www.nap.edu

Knowing What Students Know: The Science and Design of Design of Educational Assessment(2001)Center for Education

Today’s DiscussionToday s Discussion

• Assessment as a tool for helping peopleAssessment as a tool for helping people learn

Assessment should be more about what we– Assessment should be more about what we want to know

– Assessment is difficultAssessment is difficult– Assessment should focus on what we value

and think is importantand think is important

What do we ask students?What do we ask students?

• Please discussPlease discuss

What do we ask friends?What do we ask friends?

• Please discussPlease discuss

School Questions vs R l Q iReal Questions

• What is your favorite color?What is your favorite color?

• How do I get to the wedding?

Conclusion 1Conclusion 1

• Students do not want to give the “wrong”Students do not want to give the wrong answer.

• Ask questions because you want to know• Ask questions because you want to know what students think and why they think that way not because you want to testthat way – not because you want to test them.

Simplify the fraction:

16

p y

64

ANSWER A:

116464

=

Simplify the fraction:

16

p y

64

ANSWER B:ANSWER A:

116

ANSWER B:

11616464

=46464

==

Review results of surveyReview results of survey

• Student A vs Student B (5 points)Student A vs. Student B (5 points)• Student C vs. Student D (5 points)

• Student E vs. Student F (15 points)• Student G vs. Student H (15 points)

• What do these results tell us?

What Goes Wrong?g

Tests that do not correlate with understanding•Force Concept Inventory (FCI)Force Concept Inventory (FCI)•Regents exam question on moving galaxies•Private Universe videotapes

What Goes Wrong?

Tests that do not correlate with understandingTests that do not correlate with understanding•Force Concept Inventory (FCI)•Regents exam question on moving galaxies•Private Universe videotapes

We’re not testing what we teachg•Harris cartoon of mouse and maze

Wh t G W ?What Goes Wrong?

Tests that do not correlate with understandingg•Force Concept Inventory (FCI)•Regents exam question on moving galaxies

We’re not testing what we teach•Harris cartoon of mouse and maze

We’re not teaching what we test•“Waldo” phenomenon

What Goes Wrong?

Tests that do not correlate with understanding•Force Concept Inventory (FCI)y ( )•Regents exam question on moving galaxies

We’re not testing what we teachWe re not testing what we teach•Harris cartoon of mouse and maze

W ’ t t hi h t t tWe’re not teaching what we test•“Waldo” phenomenon

What can we do? What are our next steps?

Conclusion 2Conclusion 2

• Assessment is difficultAssessment is difficult– 16/64

Test what is valuable– Test what is valuable– Listen to their response (Waldo phenomenon)

The Assessment TriangleThe Assessment Triangle

cognition observation and interpretation—cognition, observation, and interpretationmust be explicitly connected and designed as a coordinated whole If not theas a coordinated whole. If not, the meaningfulness of inferences drawn from the assessment will be compromisedthe assessment will be compromised.

C O

I

The Assessment TriangleThe Assessment Triangle

cognition observation and interpretation—cognition, observation, and interpretation

Th O t t i t StThe Optometrist Story

C O

I

Assessment #1Assessment #1

• Question: What was the date of the battleQuestion: What was the date of the battle of the Spanish Armada?

Answer: 1588 [correct]– Answer: 1588 [correct].• Question: What can you tell me about

what this meant?what this meant?– Answer: Not much. It was one of the dates I

memorized for the exam Want to hear thememorized for the exam. Want to hear the others?

Assessment #2Assessment #2

• Question: What was the date of the battle of theQuestion: What was the date of the battle of the Spanish Armada?– Answer: It must have been around 1590.

• Question: Why do you say that?– Answer: I know the English began to settle in Virginia

just after 1600, not sure of the exact date. They wouldn’t have dared start overseas explorations if Spain still had control of the seas. It would take a littleSpain still had control of the seas. It would take a little while to get expeditions organized, so England must have gained naval supremacy somewhere in the late 1500s1500s.

ComparisonComparison• Most people would agree that the second p p g

student showed a better understanding of the Age of Colonization than the first, but too many examinations would assign the first student aexaminations would assign the first student a better score.

• When assessing knowledge one needs toWhen assessing knowledge, one needs to understand how the student connects pieces of knowledge to one another. Once this is known, th t h t t i ththe teacher may want to improve the connections, showing the student how to expand his or her knowledge.s o e o edge

Not in vainNot in vain

• When studying about veins and arteriesWhen studying about veins and arteries, for example, students may be expected to remember thatremember that – arteries are

• thicker than veins• thicker than veins,• more elastic, and • carry blood from the heart; y ;

– veins carry blood back to the heart.

Sample test itemSample test item

• ArteriesArteries– a. Are more elastic than veins

b Carry blood that is pumped from the heart– b. Carry blood that is pumped from the heart– c. Are less elastic than veins

d Both a and b– d. Both a and b– e. Both b and c

The new science of learningThe new science of learning

• does not deny that facts are important fordoes not deny that facts are important for thinking and problem solving.

• Research on expertise in areas such asResearch on expertise in areas such as chess, history, science,and mathematics demonstrate that experts ’ abilities to think pand solve problems depend strongly on a rich body of knowledge about subject

tt ( Ch d Si 1973 Chi tmatter (e.g.,Chase and Simon,1973;Chi et al.,1981;deGroot,1965).

Facts are not enoughFacts are not enough

• However,the research also shows clearly thatHowever,the research also shows clearly that “usable knowledge ” is not the same as a mere list of disconnected facts.

• Experts’ knowledge is – connected and organized around important concepts

(e.g., Newton ’s second law of motion); – “conditionalized” to specify the contexts in which it is

applicable;applicable;– supports understanding and transfer (to other

contexts) rather than only the ability to remember.

Vein and artery expertsVein and artery experts• Know the facts in the mc question• also understand why veins and arteries have particular

properties.– They know that blood pumped from the heart exits in spurts – That the elasticity of the arteries helps accommodate pressure

changes.– They know that blood from the heart needs to move upward (to

the brain) as well as downward and that the elasticity of an arterythe brain) as well as downward and that the elasticity of an artery permits it to function as a one-way valve that closes at the end of each spurt and prevents the blood from flowing backward.

• They are better able to transfer– Design an artificial artery strong enough to handle pressure with

or without elasticity (Bransford and Stein,1993).

Questions that foster deep understanding rather than ti th t k f titi f i d i f tiquestions that ask for repetition of memorized information

and conclusions. (Jim Minstrell and Emily van Zee)

• Ask or promote questions to open an inquiry and elicit students’ initial understanding and reasoning.

• Ask or promote questions to interpret and make sense of data in order to generate new knowledge and understanding.

• Ask or promote questions to clarify or elaborate on observations and inferences.inferences.

• Ask or promote questions to encourage learners to justify their answers and conclusions or to explain their reasoning to go beyond a mere stating of an answer.A k t ti t t d l l d id• Ask or promote questions to extend or apply learned ideas.

• Ask or promote questions that help learners monitor their own learning.

Creation of question.Creation of question.

What are our goals i h ?as science teachers?

• Higher student achievementHigher student achievement• Engaged students

A i ti (l ?) f i i th• Appreciation (love?) of science in the world

• Critical thinking

• Five years later: what is science?– e g Chemistry responses– e.g. Chemistry responses

Bloom’s TaxonomyBloom s Taxonomy

• Knowledge (define describe identify )Knowledge (define, describe, identify…)

• Comprehension (give examples, summarize…)

A li ti• Application (demonstrate, predict, solve…)

• Analysis (analyze, compare & contrast, …)

• Synthesis (modify, create …)

• Evaluation (evaluate conclude )Evaluation (evaluate, conclude…)

4 Q Assessment ModelTh E i l Q iThe Essential Questions

• What does it mean?» Newton’s 2nd Law: F = ma» Energy is conserved» The Earth goes around the Sun» The atom has a nucleus» Mitochondria are the power house of the cell» H2O is a polar molecule (angle of 104)

4 Q Assessment ModelTh E i l Q iThe Essential Questions

• What does it mean?

• How do we know? » Newton’s 2nd Law: F = ma» Energy is conserved» Energy is conserved» The Earth goes around the Sun» The atom has a nucleus» Mitochondria are the power house of the cell» H2O is a polar molecule (angle of 104)

4 Q Assessment ModelTh E i l Q iThe Essential Questions

• What does it mean?• How do we know?

• Why should I believe? (When do I not believe?)N t ’ 2nd L F» Newton’s 2nd Law: F = ma

» Energy is conserved» The Earth goes around the Sun» The atom has a nucleus» Mitochondria are the power house of the cell» H2O is a polar molecule (angle of 104)» H2O is a polar molecule (angle of 104)

4 Q Assessment ModelTh E i l Q iThe Essential Questions

• What does it mean?• How do we know?

Wh h ld I b li ?• Why should I believe?

• Why should I care?Why should I care?» Newton’s 2nd Law: F = ma» Energy is conserved» The Earth goes around the Sun» The Earth goes around the Sun» The atom has a nucleus» Mitochondria are the power house of the cell» H O is a polar molecule (angle of 104)» H2O is a polar molecule (angle of 104)

Shavelson & LiShavelson & Li

• Declarative knowledge (knowing that)Declarative knowledge (knowing that)• Procedural knowledge (knowing how)

S h ti k l d (k i h )• Schematic knowledge (knowing why)• Strategic knowledge (knowing when and

where to apply knowledge)

NAEP 2009-2019NAEP 2009 2019

• Identifying scientific principlesIdentifying scientific principles• Using scientific principles

U i i tifi i i• Using scientific inquiry• Using technology

4 Q Assessment Model4 Q Assessment Model

• What does it mean?

H d k ?• How do we know?

• Why should I believe?Why should I believe?

• Why should I care?

Post these in front of the room and reap unanticipated benefits.p

Conclusion 3Conclusion 3

• Assessment can drive the curriculumAssessment can drive the curriculum• Assessment should focus on what we

value and think is importantvalue and think is important– Essential Questions

SummarySummary

• The assessment triangle: cognition,The assessment triangle: cognition, observation, interpretation– Confusion over what we assess (optometrist)– Difficulty in assessing (16/64)

• What do we want students to know– Essential Questions and organizing principles

• Obstacles to be aware of– Are we testing what we teach? (Mice in maze)– Are we teaching what we test? (Waldo phenomenon)

An Inquiry TaskAn Inquiry Task

• Count the number of passes from aCount the number of passes from a person with a white shirt to a person in a white shirtwhite shirt.

InquiryInquiry

• When we are told what to look for weWhen we are told what to look for, we often miss things of interest.

• How to ruin another good activity!• How to ruin another good activity!– You are going to see an example of selective

attentionattention…

Where is the knowledge we haveWhere is the knowledge we have lost in information…

Where is the wisdom we have lost in knowledge?lost in knowledge?

T.S. Eliot

What is Inquiry?What is Inquiry?

Center of Science and Math In Context

COSMICCOSMIC• Please check out ourPlease check out our

website: www.cosmic.umb.edufor a copy of the power point.

Or email with questions: arthur.eisenkraft@umb.edu